You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@beam.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2020/06/07 00:49:23 UTC
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #596
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/596/display/redirect>
Changes:
------------------------------------------
[...truncated 293.96 KB...]
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:164)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Jun 07, 2020 12:45:20 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Jun 07, 2020 12:45:20 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Jun 07, 2020 12:45:20 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Jun 07, 2020 12:45:20 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Jun 07, 2020 12:45:20 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Jun 07, 2020 12:45:20 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Jun 07, 2020 12:45:21 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Jun 07, 2020 12:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 209 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Jun 07, 2020 12:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil$PackageAttributes forFileToStage
INFO: Staging custom dataflow-worker.jar as beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-aUzaT5i-X0a9y-Rodd37ZM9bDxXxRgGt70HSpwTLUtw.jar
Jun 07, 2020 12:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.23.0-SNAPSHOT-sDX1AtvIi2FKauG6zqX-ns7qzRXl-L_w6h_cgKsc50g.jar
Jun 07, 2020 12:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT-QkknYK0dMIBl4VNON8UrYkVGBvErRZqTHdmA9daN2vo.jar
Jun 07, 2020 12:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-tests-yRd5lN4v-ptUwIcotNzxf5TCVTjGr2KGUAqxilIEKVY.jar
Jun 07, 2020 12:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-tests-qSFbXbO__-5eklH6O3_0cmmlqnqXG0IKXfivXHgk3lw.jar
Jun 07, 2020 12:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-2Hsj2FuCjenZmvi9hdjDj3qK5YBfymbW8OpiT9IDoGo.jar
Jun 07, 2020 12:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /tmp/test9103518950024609139.zip to gs://temp-storage-for-perf-tests/loadtests/staging/test-pF0EmUd0-knVDwIL_hoi7fW0sRHLlYyy6MQ6QGzPXCk.jar
Jun 07, 2020 12:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-vIrT1juK7Ymcy736Fu1orQq62HaZEf4RgtqzOw8ykSQ.jar
Jun 07, 2020 12:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-IM8zUJspeecfjWOwGT7f4r_KdeJM3wjClQiQuntxACU.jar
Jun 07, 2020 12:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.23.0-SNAPSHOT-lV_hqSg10QSIsbw0X4GJ4rdKgo2ikL0SukPmB_VG0Os.jar
Jun 07, 2020 12:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tests-iq6S_mHNNOmma0faPhJI2QhLWtBRLhgaCZ9mS1lxa0s.jar
Jun 07, 2020 12:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT-jojWqn5rReG2KPoLMPQEOVJkqKLLCQqqptFyFFkrQb0.jar
Jun 07, 2020 12:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.23.0-SNAPSHOT-_nER_ModuiesYj2r7lkHWZFyciogAicwJubdzJvFeZ0.jar
Jun 07, 2020 12:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-aUzaT5i-X0a9y-Rodd37ZM9bDxXxRgGt70HSpwTLUtw.jar
Jun 07, 2020 12:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-tests-pqucZzoSx6Nrp8heFyS8JdO3wvrFXzAC_fidonswoG8.jar
Jun 07, 2020 12:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-tests-4wJ7lyKs4CpOiqRpbLuPc16Yf76qarSVGnLU_UXbBbM.jar
Jun 07, 2020 12:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-unshaded-34uREE_OiuHinXaT4fGMDGKiY5wxuvin_ks7ElbjlwQ.jar
Jun 07, 2020 12:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-fn-execution-2.23.0-SNAPSHOT-xyQhhEY2wN3x_hN1-O_Y_h--IcIWMzkpVcQpgdmbnmQ.jar
Jun 07, 2020 12:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/java-fn-execution/build/libs/beam-runners-java-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-java-fn-execution-2.23.0-SNAPSHOT-82cPjDR_X1SkCfBKY54-1gtizp5EdvBfVW5RN55D4bI.jar
Jun 07, 2020 12:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-Yy6jkEgfpMdocwF7MXgqWY2BmhKR2IkL2GfSJP6phHw.jar
Jun 07, 2020 12:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.23.0-SNAPSHOT-ADlkMcTpkOpE6fAMREa0WYJkf3kjXw8EFa1dvswEA4k.jar
Jun 07, 2020 12:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-PrynFjE67NvxmtxM6UCRmVjTgdrHmUdTQRuXrc8cU6g.jar
Jun 07, 2020 12:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-TRMI8t4CJ0YSRAS7LdTjFR0I7t0lD50grNPSPBRMe84.jar
Jun 07, 2020 12:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-tests-qMGLsop42vvYe-8lGtwBkOhKXqZaSXB7YAWHl4KqfZA.jar
Jun 07, 2020 12:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.23.0-SNAPSHOT-rePcoysH6XXPnk2dnHAAp2mSBlUApvekrDFh4O2RkLo.jar
Jun 07, 2020 12:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-tests-c6jxhIA-85QwDDN7a2a5V3tWF6Bdw0Ub8wOUrSyaKQk.jar
Jun 07, 2020 12:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.23.0-SNAPSHOT-tests-qVW3JT16ckFV567FdhyuAlKR9ExgfoSi-VYErreyKZo.jar
Jun 07, 2020 12:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-l_QFFWWWv-GjeSYiZcNW-VsCINsRqHT9cpNbMmqJQLI.jar
Jun 07, 2020 12:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-aUzaT5i-X0a9y-Rodd37ZM9bDxXxRgGt70HSpwTLUtw.jar
Jun 07, 2020 12:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.23.0-SNAPSHOT-CfhjAVXiARrYZFu3aNLbFkUrDARdQUVqYjZ4BVxoF-Y.jar
Jun 07, 2020 12:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.23.0-SNAPSHOT-mkZZL_t3gGYEEYbvl2vmWitRGzM4GUqXU-zjgmKGauI.jar
Jun 07, 2020 12:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.23.0-SNAPSHOT-1xz4LrZ_CuSSKo-9mpu6ruMvkJjLVMxd30smPXCa97U.jar
Jun 07, 2020 12:45:24 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-aUzaT5i-X0a9y-Rodd37ZM9bDxXxRgGt70HSpwTLUtw.jar&uploadType=resumable&upload_id=AAANsUmycB9dddLav_GxeAPEGTYDallN4gW0SlP2fPDwJzcAxv4SlJypu6pX4xCpcHFL_mdEz0gM50O8YhuvQKPApKSPbNGjUQ.
Jun 07, 2020 12:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar>
java.io.IOException: Upload failed for 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-aUzaT5i-X0a9y-Rodd37ZM9bDxXxRgGt70HSpwTLUtw.jar'
at com.google.cloud.hadoop.util.BaseAbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(BaseAbstractGoogleAsyncWriteChannel.java:297)
at com.google.cloud.hadoop.util.BaseAbstractGoogleAsyncWriteChannel.close(BaseAbstractGoogleAsyncWriteChannel.java:214)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:221)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:168)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:152)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:136)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Jun 07, 2020 12:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-aUzaT5i-X0a9y-Rodd37ZM9bDxXxRgGt70HSpwTLUtw.jar
Jun 07, 2020 12:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 31 files newly uploaded in 7 seconds
Jun 07, 2020 12:45:30 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Jun 07, 2020 12:45:30 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Jun 07, 2020 12:45:30 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Jun 07, 2020 12:45:30 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Jun 07, 2020 12:45:30 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Jun 07, 2020 12:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <91292 bytes, hash 2f4a083624bd200049d26e62ce42004da8a3361f6d7211b441a2ccbdaa11589c> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-L0oINiS9IABJ0m5izkIATaijNh9tchG0QaLMvaoRWJw.pb
Jun 07, 2020 12:45:31 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.23.0-SNAPSHOT
Jun 07, 2020 12:45:32 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-06-06_17_45_31-7964275073213879033?project=apache-beam-testing
Jun 07, 2020 12:45:32 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-06-06_17_45_31-7964275073213879033
Jun 07, 2020 12:45:32 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-06-06_17_45_31-7964275073213879033
Jun 07, 2020 12:45:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-06-07T00:45:31.616Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Jun 07, 2020 12:45:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-07T00:45:38.625Z: Worker configuration: n1-standard-1 in us-central1-a.
Jun 07, 2020 12:45:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-07T00:45:39.295Z: Expanding CoGroupByKey operations into optimizable parts.
Jun 07, 2020 12:45:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-07T00:45:39.336Z: Expanding GroupByKey operations into optimizable parts.
Jun 07, 2020 12:45:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-07T00:45:39.369Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jun 07, 2020 12:45:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-07T00:45:39.450Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jun 07, 2020 12:45:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-07T00:45:39.484Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Jun 07, 2020 12:45:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-07T00:45:39.521Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Jun 07, 2020 12:45:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-07T00:45:39.555Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Jun 07, 2020 12:45:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-07T00:45:40.038Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Jun 07, 2020 12:45:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-07T00:45:40.116Z: Starting 5 workers in us-central1-a...
Jun 07, 2020 12:46:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-06-07T00:46:06.586Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jun 07, 2020 12:46:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-07T00:46:11.279Z: Autoscaling: Raised the number of workers to 1 based on the rate of progress in the currently running stage(s).
Jun 07, 2020 12:46:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-07T00:46:11.310Z: Resized worker pool to 1, though goal was 5. This could be a quota issue.
Jun 07, 2020 12:46:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-07T00:46:16.656Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Jun 07, 2020 12:46:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-07T00:46:38.219Z: Workers have started successfully.
Jun 07, 2020 12:46:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-07T00:46:38.257Z: Workers have started successfully.
Jun 07, 2020 12:47:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-07T00:47:14.709Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Jun 07, 2020 12:47:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-07T00:47:14.915Z: Cleaning up.
Jun 07, 2020 12:47:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-07T00:47:15.013Z: Stopping worker pool...
Jun 07, 2020 12:49:13 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-07T00:49:13.395Z: Autoscaling: Resized worker pool from 5 to 0.
Jun 07, 2020 12:49:13 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-07T00:49:13.431Z: Worker pool stopped.
Jun 07, 2020 12:49:20 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-06-06_17_45_31-7964275073213879033 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): e658dbec-11f2-4682-a234-ab2ff4463c8f and timestamp: 2020-06-07T00:49:20.465000000Z:
Metric: Value:
read_time 17.526
fields_read 4375276.0
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Jun 07, 2020 12:49:20 AM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.023 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.032 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 4,5,main]) completed. Took 4 mins 8.682 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 0s
104 actionable tasks: 68 executed, 36 from cache
Publishing build scan...
https://gradle.com/s/7nuvfqirmgfia
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #637
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/637/display/redirect?page=changes>
Changes:
[github] [BEAM-10217] CALL_FUNCTION and CALL_METHOD fixes (#11966)
------------------------------------------
[...truncated 293.80 KB...]
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Jun 17, 2020 6:45:22 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Jun 17, 2020 6:45:22 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery writeDisposition is set to: WRITE_EMPTY
Jun 17, 2020 6:45:22 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Jun 17, 2020 6:45:22 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Jun 17, 2020 6:45:22 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery writeDisposition is set to: WRITE_EMPTY
Jun 17, 2020 6:45:22 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Jun 17, 2020 6:45:22 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Jun 17, 2020 6:45:22 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Jun 17, 2020 6:45:23 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Jun 17, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 209 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Jun 17, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil$PackageAttributes forFileToStage
INFO: Staging custom dataflow-worker.jar as beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-0QZFFciQzdJX8dfWGVzE-N7ITjiJqpQlu28rVxGCJ_g.jar
Jun 17, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.23.0-SNAPSHOT-e4EuSZVsDYC2TYpT_ssd-FLbfGMzQFGwOozdC4E-Bjg.jar
Jun 17, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-tests-KAX93fRF_naNjV4cqeR9WdrT1QsvJH40fH9TZ8y3cIs.jar
Jun 17, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-tests-Y1MgXp96btago3dVxDI9g2hq1UuJBmuDNSGNiU_A8xg.jar
Jun 17, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-QB17NPicCgaoMMMpVVRPyfCeEyNKcLpqa7LPDhSf_Sk.jar
Jun 17, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT-E1bmAQUNqyKe5v-xCa0VBRJPX6V8DYeYoDJcmOGdKw0.jar
Jun 17, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.23.0-SNAPSHOT-2VWW5cQwVGVqcx9sVjkHXJ8bxFBted-Tufoa1kK5I7Y.jar
Jun 17, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-L-BIUKaxpufiwc-RiDlaoDxHfmK0jrHyI0sTsV0aznU.jar
Jun 17, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-tests-EXma9p7QqDHKJ0Iy6ZgZ8Kpi_MI0MGc7VnFDfZ28kYg.jar
Jun 17, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-tests-rvheiEyrC7-4jJC9xobBzQHF96FpV_sL3pT_ctKW3v0.jar
Jun 17, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.23.0-SNAPSHOT-6NHoUJFalbLy_-ShymG9LPlcZo0tprIdG6oI6_XgT0U.jar
Jun 17, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-xW3vurl8IW9Ve8tpriZGh4kanUO9Jf0fDpbNXxH7m6g.jar
Jun 17, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /tmp/test4196560194431367712.zip to gs://temp-storage-for-perf-tests/loadtests/staging/test-R8LcFafqjoSEmpoOq-UFakDqx_AXEDyGhOtrTgj59qc.jar
Jun 17, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT--_cqUsdOt93wQOH4xCDQ2d6i5epRyVx1nr3ghsdw_wY.jar
Jun 17, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-ZulyVysXlBx9RhKcbXQaBhzqzxzpcYLr6bp0wQ9XhSo.jar
Jun 17, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.23.0-SNAPSHOT-O50il_1EZ3IAaKjmKkE15FduYvijwqS_6R_xbxP9Pn8.jar
Jun 17, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.23.0-SNAPSHOT-tests-Y3bMUk7HDELw3DKW6qwckI6sTpI12ydoyId3KMaxCYM.jar
Jun 17, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-0QZFFciQzdJX8dfWGVzE-N7ITjiJqpQlu28rVxGCJ_g.jar
Jun 17, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tests-kErA6TJyKCOzcbRV90FMbxDvcYZj-SdGhQe3WeSRzhY.jar
Jun 17, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-unshaded-jeqhiSvHbZNh6LuT8TI_WrFqa39H71hojYLWbhe6Ick.jar
Jun 17, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-cY-pxEAdAuGrFqEBq-8MH6sRaFe3uqUybsvrdtGDqg4.jar
Jun 17, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-xLgjeFSfnEvmIZ-K8xAoL1cjyvxM69J-MCTpVXUS4zA.jar
Jun 17, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.23.0-SNAPSHOT-lW_h3HFEx6cdMoP0tytYONEifRDflx4mO6f8G6pLFLU.jar
Jun 17, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/java-fn-execution/build/libs/beam-runners-java-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-java-fn-execution-2.23.0-SNAPSHOT-HenSoULppAINBqZ6XKJG8sIxbl7PFGWBDHi1HHz4AVw.jar
Jun 17, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-rrh8hBwWZMuavTDULVj-vfOBUJCTNRgm7RTumc2hYdI.jar
Jun 17, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-tests-JO-nogNjXBPq_AjxSjh8z1LcNpPEVjJqU0fXtg52hVk.jar
Jun 17, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-tests-JgSBQZ50tUE7JYkW-sqhID9roiA0p4-K7HiSXETm2U4.jar
Jun 17, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-fn-execution-2.23.0-SNAPSHOT-i3CU9E1ZZy9-fTRP1SO9D8TnU2zcRy5hrvLX6akdXrY.jar
Jun 17, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-0QZFFciQzdJX8dfWGVzE-N7ITjiJqpQlu28rVxGCJ_g.jar
Jun 17, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.23.0-SNAPSHOT-u1P760bvRk8wQwbQtlKQIbQ7PSF1MYg_DLfThAhWV_w.jar
Jun 17, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.23.0-SNAPSHOT-O3wlYcV7wBNzqSa5GhhS8g0G7mI71yqZF2DG-O7v-LI.jar
Jun 17, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.23.0-SNAPSHOT-aRLkXkBqCLomxzRYP_tzE7_2PmR0TqbZKIP9Mh4bKM8.jar
Jun 17, 2020 6:45:26 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-0QZFFciQzdJX8dfWGVzE-N7ITjiJqpQlu28rVxGCJ_g.jar&uploadType=resumable&upload_id=AAANsUkSRZMyAMVUywifJf8jQ89pHVns7b7cqkYlW35dQUFiqUdMfrwlZzBY8gpIQmbJtgS9hYZtyh07F6pfh_WsC9IhMu4JXQ.
Jun 17, 2020 6:45:26 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar>
java.io.IOException: Upload failed for 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-0QZFFciQzdJX8dfWGVzE-N7ITjiJqpQlu28rVxGCJ_g.jar'
at com.google.cloud.hadoop.util.BaseAbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(BaseAbstractGoogleAsyncWriteChannel.java:297)
at com.google.cloud.hadoop.util.BaseAbstractGoogleAsyncWriteChannel.close(BaseAbstractGoogleAsyncWriteChannel.java:214)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:221)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:168)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:152)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:136)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Jun 17, 2020 6:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-0QZFFciQzdJX8dfWGVzE-N7ITjiJqpQlu28rVxGCJ_g.jar
Jun 17, 2020 6:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 31 files newly uploaded in 8 seconds
Jun 17, 2020 6:45:33 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Jun 17, 2020 6:45:34 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Jun 17, 2020 6:45:34 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Jun 17, 2020 6:45:34 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Jun 17, 2020 6:45:34 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Jun 17, 2020 6:45:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <91369 bytes, hash 649406986a4b85d90a0b3972e19f3317cdce6ef6d73d7109464f20f6dc178d8b> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-ZJQGmGpLhdkKCzly4Z8zF83ObvbXPXEJRk8g9twXjYs.pb
Jun 17, 2020 6:45:34 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.23.0-SNAPSHOT
Jun 17, 2020 6:45:35 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-06-16_23_45_34-15960701559817935774?project=apache-beam-testing
Jun 17, 2020 6:45:35 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-06-16_23_45_34-15960701559817935774
Jun 17, 2020 6:45:35 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-06-16_23_45_34-15960701559817935774
Jun 17, 2020 6:45:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-06-17T06:45:34.705Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Jun 17, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-17T06:45:43.479Z: Worker configuration: n1-standard-1 in us-central1-a.
Jun 17, 2020 6:45:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-17T06:45:44.314Z: Expanding CoGroupByKey operations into optimizable parts.
Jun 17, 2020 6:45:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-17T06:45:44.351Z: Expanding GroupByKey operations into optimizable parts.
Jun 17, 2020 6:45:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-17T06:45:44.386Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jun 17, 2020 6:45:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-17T06:45:44.462Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jun 17, 2020 6:45:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-17T06:45:44.488Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Jun 17, 2020 6:45:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-17T06:45:44.520Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Jun 17, 2020 6:45:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-17T06:45:44.554Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Jun 17, 2020 6:45:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-17T06:45:44.900Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Jun 17, 2020 6:45:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-17T06:45:44.979Z: Starting 5 workers in us-central1-a...
Jun 17, 2020 6:45:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-06-17T06:45:57.941Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jun 17, 2020 6:46:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-17T06:46:21.602Z: Autoscaling: Raised the number of workers to 1 based on the rate of progress in the currently running stage(s).
Jun 17, 2020 6:46:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-17T06:46:21.644Z: Resized worker pool to 1, though goal was 5. This could be a quota issue.
Jun 17, 2020 6:46:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-17T06:46:27.048Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Jun 17, 2020 6:46:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-17T06:46:41.865Z: Workers have started successfully.
Jun 17, 2020 6:46:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-17T06:46:41.900Z: Workers have started successfully.
Jun 17, 2020 6:47:15 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-17T06:47:13.283Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Jun 17, 2020 6:47:15 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-17T06:47:13.486Z: Cleaning up.
Jun 17, 2020 6:47:15 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-17T06:47:13.564Z: Stopping worker pool...
Jun 17, 2020 6:48:10 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-17T06:48:09.089Z: Autoscaling: Resized worker pool from 5 to 0.
Jun 17, 2020 6:48:10 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-17T06:48:09.134Z: Worker pool stopped.
Jun 17, 2020 6:48:15 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-06-16_23_45_34-15960701559817935774 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): c422ebc0-c328-466b-aeb5-b31d0b85e453 and timestamp: 2020-06-17T06:48:15.651000000Z:
Metric: Value:
fields_read 4375276.0
read_time 15.938
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Jun 17, 2020 6:48:16 AM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.023 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.031 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 9,5,main]) completed. Took 3 mins 2.976 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 3m 56s
104 actionable tasks: 68 executed, 36 from cache
Publishing build scan...
https://gradle.com/s/fwc4whyxggege
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #636
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/636/display/redirect?page=changes>
Changes:
[heejong] [BEAM-10208] add cross-language KafkaIO integration test
[rionmonster] added kotlin katas release blog post with associated images
[rionmonster] fixed up verbiage
[kcweaver] [BEAM-9852] Do not create data channel for empty timer descriptor.
[kcweaver] Fix state handler for missing service descriptor.
[github] Merge pull request #11838 from [BEAM-9322] Modify the TestStream to
[github] [BEAM-10251] Adds transform id to TestStream step (#12003)
[github] [BEAM-7672] Increase the set of acceptable Python wheels in Beam Python
[github] Merge pull request #11790 from [BEAM-9926] Programming guide - Fix typos
[github] [BEAM-9679] Update Stepik course information (#12018)
[github] [BEAM-10169] ParDo functions with correct output N in their error
------------------------------------------
[...truncated 293.96 KB...]
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Jun 17, 2020 12:45:29 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Jun 17, 2020 12:45:29 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery writeDisposition is set to: WRITE_EMPTY
Jun 17, 2020 12:45:29 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Jun 17, 2020 12:45:29 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Jun 17, 2020 12:45:29 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery writeDisposition is set to: WRITE_EMPTY
Jun 17, 2020 12:45:29 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Jun 17, 2020 12:45:29 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Jun 17, 2020 12:45:29 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Jun 17, 2020 12:45:30 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Jun 17, 2020 12:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 209 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Jun 17, 2020 12:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil$PackageAttributes forFileToStage
INFO: Staging custom dataflow-worker.jar as beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-pk1vyZn5Y76Fzugk4kQeklhqhlG2hjLy-CjdZg4odEw.jar
Jun 17, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-tests-MgHhu5bO4Ym86uK2YNP0tK2Z9rOLWGtvqzuQ0FSfAsU.jar
Jun 17, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-DgWUFo6KbZU9bLGXE-WRFaS3g123UGXDJKHjN3z2Jt0.jar
Jun 17, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /tmp/test8084354935468521649.zip to gs://temp-storage-for-perf-tests/loadtests/staging/test-u_-VFziB_GBqtxs_PMPphNMB1fA2tHdZsfWGG627C-U.jar
Jun 17, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-tests-yZg2iUrcRZ-liB7tQudCs_Rl3ZqEHKeW5Cb1uVSVDdQ.jar
Jun 17, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-tests-nN91jfU0U9TdxPIzh4yQTzSEoV-g0yWhDrkV1nbQsk0.jar
Jun 17, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-vrETBPn-R3TzhDs-RS6Gop7sQy4jRufbrBxthqs4TO4.jar
Jun 17, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-tests-A3juOHXn6qiYCRRcw0V2nEwUytO5cTECetj8D-pUjVY.jar
Jun 17, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.23.0-SNAPSHOT-nCbgUiIMtkZp0dsbiHkFhaUqMUHIe-e4MXSO8yKCPbg.jar
Jun 17, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-cae3ROz7OkddCaTNAliD_rpW31Yh_94Lgs40DS63WlU.jar
Jun 17, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tests-rj-CxGi2gV7DbiZ97WOlC99UCHQacoUm3vjlTjoHCf0.jar
Jun 17, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.23.0-SNAPSHOT-zGmUg9L2MUur0O5ifps8sFUW3awp0QJZx6uyyTMtVq8.jar
Jun 17, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-fn-execution-2.23.0-SNAPSHOT-HmXlXJ6UVwIlXYD5-BE33qr0L4wl8_qe5LsRnxPyovI.jar
Jun 17, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-W5dtnX4sSK0gRnVKviNio8qHw5VJBCvA38iWmkcOUWk.jar
Jun 17, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-fN0r9M-Rojz8XCU68CnPNZjLI2_58wXuun3fiNObR8c.jar
Jun 17, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-Oclbbj0iMf2tNSMmBqfeKkyfjytvrwDsA3HWGWZv7x0.jar
Jun 17, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT-Z9NWbqlKijdS-iOddda4-ZlKu9CxK6e7SdNQvh4x1zA.jar
Jun 17, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.23.0-SNAPSHOT-qy5Mpa2xN4zYH3019wH1B3fVzddJkFkwWAYYD28RKPY.jar
Jun 17, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-unshaded-56sz3cG_p4OlI8JD7kLci1fHoDcslo5CQN0e_-5tp_E.jar
Jun 17, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-tests-Q7oxkwd1BjQMZm7hVI5V3VamWecLvyJiT6_LyZ2JkiY.jar
Jun 17, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT-aJ67vgoO2o7qX9H9OuO-lwQZXk2ay0je4d3CbtNFMmY.jar
Jun 17, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-22g7qwIMdRWxVCuEvqvOYe9CjcUMD0hii7xb4guN-48.jar
Jun 17, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-pk1vyZn5Y76Fzugk4kQeklhqhlG2hjLy-CjdZg4odEw.jar
Jun 17, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.23.0-SNAPSHOT-0E1RCKHk0fgDJeABIRIyX5d_Vxn8zhH9V0WCzeF2bsU.jar
Jun 17, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-tests-l6p3I6fSekBLgtEkENujHZQkzxjCAeDFu0nWAgVIfSk.jar
Jun 17, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.23.0-SNAPSHOT-tests-qTG-sA8zuLX9s216pqM5Ox0wTiUPEf8pdJeZD3x2YfI.jar
Jun 17, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.23.0-SNAPSHOT-1bIjHkYzK1Gt6shqmunz92vDrJkBst_tPO7h8G1NNYk.jar
Jun 17, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/java-fn-execution/build/libs/beam-runners-java-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-java-fn-execution-2.23.0-SNAPSHOT-5rHClUPFwn6duq0SVLOYg_6YL5zNTDmw6v-5JDLKVec.jar
Jun 17, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-pk1vyZn5Y76Fzugk4kQeklhqhlG2hjLy-CjdZg4odEw.jar
Jun 17, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.23.0-SNAPSHOT-DariNt4Jxl-2dOOVvrVb_ctnQ8IEuPJH7AMP1ktCZDw.jar
Jun 17, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.23.0-SNAPSHOT-KODobDNpaKuGX4bNftQ6-F49g2JPxwh0Fh_fp3frv9k.jar
Jun 17, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.23.0-SNAPSHOT-GS6agjpKhaYMc0D2-UazbP7v-aJflE40agU0kNdYEZg.jar
Jun 17, 2020 12:45:34 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-pk1vyZn5Y76Fzugk4kQeklhqhlG2hjLy-CjdZg4odEw.jar&uploadType=resumable&upload_id=AAANsUnFVT2ilcvr24Bv2RUOc1mbK0iSjbS5uHR3pCRmgHvmQQX1xhwwEe9eDqhMvv7UqCShKzP2157tTF0ZcpAKEQhwFk0xkA.
Jun 17, 2020 12:45:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar>
java.io.IOException: Upload failed for 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-pk1vyZn5Y76Fzugk4kQeklhqhlG2hjLy-CjdZg4odEw.jar'
at com.google.cloud.hadoop.util.BaseAbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(BaseAbstractGoogleAsyncWriteChannel.java:297)
at com.google.cloud.hadoop.util.BaseAbstractGoogleAsyncWriteChannel.close(BaseAbstractGoogleAsyncWriteChannel.java:214)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:221)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:168)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:152)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:136)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Jun 17, 2020 12:45:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-pk1vyZn5Y76Fzugk4kQeklhqhlG2hjLy-CjdZg4odEw.jar
Jun 17, 2020 12:45:39 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 31 files newly uploaded in 6 seconds
Jun 17, 2020 12:45:39 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Jun 17, 2020 12:45:39 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Jun 17, 2020 12:45:39 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Jun 17, 2020 12:45:39 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Jun 17, 2020 12:45:39 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Jun 17, 2020 12:45:39 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <91368 bytes, hash 35af90a782b868be4ebb3bad82ffd3a1058b5558d0553701b67cfe94574f2f4a> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-Na-Qp4K4aL5Ouzutgv_ToQWLVVjQVTcBtnz-lFdPL0o.pb
Jun 17, 2020 12:45:39 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.23.0-SNAPSHOT
Jun 17, 2020 12:45:41 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-06-16_17_45_39-1626291154219099259?project=apache-beam-testing
Jun 17, 2020 12:45:41 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-06-16_17_45_39-1626291154219099259
Jun 17, 2020 12:45:41 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-06-16_17_45_39-1626291154219099259
Jun 17, 2020 12:45:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-06-17T00:45:39.875Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Jun 17, 2020 12:45:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-17T00:45:47.567Z: Worker configuration: n1-standard-1 in us-central1-a.
Jun 17, 2020 12:45:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-17T00:45:48.823Z: Expanding CoGroupByKey operations into optimizable parts.
Jun 17, 2020 12:45:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-17T00:45:48.867Z: Expanding GroupByKey operations into optimizable parts.
Jun 17, 2020 12:45:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-17T00:45:48.890Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jun 17, 2020 12:45:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-17T00:45:48.976Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jun 17, 2020 12:45:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-17T00:45:49.010Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Jun 17, 2020 12:45:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-17T00:45:49.044Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Jun 17, 2020 12:45:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-17T00:45:49.076Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Jun 17, 2020 12:45:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-17T00:45:49.600Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Jun 17, 2020 12:45:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-17T00:45:49.675Z: Starting 5 workers in us-central1-a...
Jun 17, 2020 12:46:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-06-17T00:46:20.790Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jun 17, 2020 12:46:24 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-17T00:46:23.469Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running stage(s).
Jun 17, 2020 12:46:24 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-17T00:46:23.542Z: Resized worker pool to 4, though goal was 5. This could be a quota issue.
Jun 17, 2020 12:46:29 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-17T00:46:28.915Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Jun 17, 2020 12:46:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-17T00:46:44.976Z: Workers have started successfully.
Jun 17, 2020 12:46:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-17T00:46:45.007Z: Workers have started successfully.
Jun 17, 2020 12:47:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-17T00:47:19.227Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Jun 17, 2020 12:47:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-17T00:47:19.448Z: Cleaning up.
Jun 17, 2020 12:47:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-17T00:47:19.523Z: Stopping worker pool...
Jun 17, 2020 12:48:05 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-17T00:48:04.107Z: Autoscaling: Resized worker pool from 5 to 0.
Jun 17, 2020 12:48:05 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-17T00:48:04.164Z: Worker pool stopped.
Jun 17, 2020 12:48:09 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-06-16_17_45_39-1626291154219099259 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 1393fc77-2602-47a4-b9ba-25a81166989a and timestamp: 2020-06-17T00:48:09.795000000Z:
Metric: Value:
read_time 16.444
fields_read 4375276.0
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Jun 17, 2020 12:48:10 AM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.02 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.032 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 3,5,main]) completed. Took 2 mins 52.134 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 3m 48s
104 actionable tasks: 68 executed, 36 from cache
Publishing build scan...
https://gradle.com/s/n2i5p7erijhaw
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #635
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/635/display/redirect?page=changes>
Changes:
[mxm] [BEAM-10260] Fix continuation token support with statecache
[mxm] [BEAM-10260] Remove is_cached parameter from CachingStateHandler
[github] Merge pull request #11086 from [BEAM-8910] Make custom BQ source read
------------------------------------------
[...truncated 292.50 KB...]
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Jun 16, 2020 6:45:24 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Jun 16, 2020 6:45:24 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery writeDisposition is set to: WRITE_EMPTY
Jun 16, 2020 6:45:24 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Jun 16, 2020 6:45:24 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Jun 16, 2020 6:45:25 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery writeDisposition is set to: WRITE_EMPTY
Jun 16, 2020 6:45:25 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Jun 16, 2020 6:45:25 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Jun 16, 2020 6:45:25 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Jun 16, 2020 6:45:25 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Jun 16, 2020 6:45:27 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 209 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Jun 16, 2020 6:45:27 PM org.apache.beam.runners.dataflow.util.PackageUtil$PackageAttributes forFileToStage
INFO: Staging custom dataflow-worker.jar as beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-MThHi9L7k2cl2yeFQCDz68TSTVnierUjjFa05WxlYZs.jar
Jun 16, 2020 6:45:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.23.0-SNAPSHOT-3jOwAbk2cYVdK-6pMF_pevpmUO3o__aKTaMaY7xqyTQ.jar
Jun 16, 2020 6:45:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-fn-execution-2.23.0-SNAPSHOT-9TgLL-s_cOT1iqLiqYQHop_gqaEWGWgxwagAw99C2wg.jar
Jun 16, 2020 6:45:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT-Ma62pnjY06DcLM646rZPpedA--jGHxcVPcABHib9M-Y.jar
Jun 16, 2020 6:45:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-tests-x0XwBqPaR3h5TuqC4VilOXehOue4HG9F6FDMgI5MJ1I.jar
Jun 16, 2020 6:45:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-tests-J3g1PHitcdI6F6vDy36veVyx72k9UT-T2swUWgxB0ck.jar
Jun 16, 2020 6:45:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT-EMUHYOeEbXYLeXmpGcpptBaNsLiW62Xts6WkWXikink.jar
Jun 16, 2020 6:45:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-6SsFtMZUeH3NmeVuXw0SdlFoehsHoZhwVtMHRBSldzw.jar
Jun 16, 2020 6:45:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-JaGKBblHzjSXM5unYY-0dx08qxu3hFpKHnuRld7BHhU.jar
Jun 16, 2020 6:45:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-tests-V88IN9pY7kqvLgs8okUp8rpjI7XenlRH8nIEJOG8qcI.jar
Jun 16, 2020 6:45:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.23.0-SNAPSHOT-GAkphSvTzsHFxO_3sqAx8UHchJt7aZ4zQssn9waVG6Q.jar
Jun 16, 2020 6:45:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tests-IrB-BHd5rrnlPHVEe2ZcZ2kCZBfN8lITxE79ZfIsxO4.jar
Jun 16, 2020 6:45:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-NmNawG4UTF3-wC42tz7Wif5mbpLqhRAIBRja9QmgUsQ.jar
Jun 16, 2020 6:45:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-3Y0HL_p6ZHK-SjhHJLGJhWwde4Rtcd2a-jnVg5IERrM.jar
Jun 16, 2020 6:45:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.23.0-SNAPSHOT-xttgSG953NBN4G5GHkWdOHhQJUQHlYRNxATIHn7wlt8.jar
Jun 16, 2020 6:45:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.23.0-SNAPSHOT-tests-MMuMOHn72FrK8ejv9ct7aEXhlq9ttyYOMZX7Hqi8pGU.jar
Jun 16, 2020 6:45:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-tests-nF1sIYVHnuWMsFEcqGkEcYU3M0ja88HdfB73Msr73pM.jar
Jun 16, 2020 6:45:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-8LpnwWs1yqyNIDhPsJgsXpuRWK6a9-ITPIomQjTgglM.jar
Jun 16, 2020 6:45:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.23.0-SNAPSHOT-ve0VRlLvDPArRmkWpuOTQAkLk-ojhZn1p61hvkrjZjg.jar
Jun 16, 2020 6:45:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-MThHi9L7k2cl2yeFQCDz68TSTVnierUjjFa05WxlYZs.jar
Jun 16, 2020 6:45:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.23.0-SNAPSHOT-m4cGuBZY-7eSru5jtopv8paNqJNoqi5W9g6Hu34PL1E.jar
Jun 16, 2020 6:45:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-dUKyFY9GTP93lVj1AMlNrLLuj95lIiljDr2fQazgiEg.jar
Jun 16, 2020 6:45:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-tests-hgdMcabWALPaxzYoHdHRKk1ZdKlsG4L2rwOZRLM9Gjk.jar
Jun 16, 2020 6:45:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-tests-ts3gBdPbJc6AjTyKvPJs4aD3g43S1J9ZzLg0wBuewNY.jar
Jun 16, 2020 6:45:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/java-fn-execution/build/libs/beam-runners-java-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-java-fn-execution-2.23.0-SNAPSHOT-wlQt4LWhfNJVPZy_PxHoiVXLTU0ipRAgpQ-3vSN0nA0.jar
Jun 16, 2020 6:45:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /tmp/test1450548537234301188.zip to gs://temp-storage-for-perf-tests/loadtests/staging/test-ZtwjgXIM_WOf5KAZSdVuZbckYnaWc6efv_2LweRJkFY.jar
Jun 16, 2020 6:45:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-unshaded-cCZ0OLe7WUYBqEGl0dKx8cKB3BfyA58uMRd3kdf1_Hg.jar
Jun 16, 2020 6:45:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT--VHu6bfgZOoijqNC0GLDA3OgImzNOrf8-UkFssmUp9s.jar
Jun 16, 2020 6:45:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-MThHi9L7k2cl2yeFQCDz68TSTVnierUjjFa05WxlYZs.jar
Jun 16, 2020 6:45:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.23.0-SNAPSHOT-sLbq90L-HffMI8UrQ8jnb9RVEL-z-xDRZjwTL5C1Ebk.jar
Jun 16, 2020 6:45:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.23.0-SNAPSHOT-eCHy0C_fg1sX7s0yqsrz9Zo_Sr1AdWhPKqmAyYXV624.jar
Jun 16, 2020 6:45:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.23.0-SNAPSHOT-psGKsewIwzQzAluLIhVpKetrkrKNC733ThJTPc0On3Y.jar
Jun 16, 2020 6:45:28 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-MThHi9L7k2cl2yeFQCDz68TSTVnierUjjFa05WxlYZs.jar&uploadType=resumable&upload_id=AAANsUl-S4LKfzywUUJbH5xvwVZu-lAJd2qRQJxpQse6PMbtutOjL73hwi7CcmHJkiETAI8soMo2gwsj7v7VnkHwxz73nykopg.
Jun 16, 2020 6:45:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar>
java.io.IOException: Upload failed for 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-MThHi9L7k2cl2yeFQCDz68TSTVnierUjjFa05WxlYZs.jar'
at com.google.cloud.hadoop.util.BaseAbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(BaseAbstractGoogleAsyncWriteChannel.java:297)
at com.google.cloud.hadoop.util.BaseAbstractGoogleAsyncWriteChannel.close(BaseAbstractGoogleAsyncWriteChannel.java:214)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:221)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:168)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:152)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:136)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Jun 16, 2020 6:45:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-MThHi9L7k2cl2yeFQCDz68TSTVnierUjjFa05WxlYZs.jar
Jun 16, 2020 6:45:37 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 31 files newly uploaded in 9 seconds
Jun 16, 2020 6:45:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Jun 16, 2020 6:45:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Jun 16, 2020 6:45:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Jun 16, 2020 6:45:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Jun 16, 2020 6:45:37 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Jun 16, 2020 6:45:37 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <91368 bytes, hash 42b1bfa1a11ea36617cc7c22379cf0e3f3da78223d32fc30f94016621e7670d9> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-QrG_oaEeo2YXzHwiN5zw4_PaeCI9Mvww-UAWYh52cNk.pb
Jun 16, 2020 6:45:37 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.23.0-SNAPSHOT
Jun 16, 2020 6:45:38 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-06-16_11_45_37-15354772517132809839?project=apache-beam-testing
Jun 16, 2020 6:45:38 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-06-16_11_45_37-15354772517132809839
Jun 16, 2020 6:45:38 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-06-16_11_45_37-15354772517132809839
Jun 16, 2020 6:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-06-16T18:45:37.767Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Jun 16, 2020 6:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-16T18:45:44.472Z: Worker configuration: n1-standard-1 in us-central1-a.
Jun 16, 2020 6:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-16T18:45:45.306Z: Expanding CoGroupByKey operations into optimizable parts.
Jun 16, 2020 6:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-16T18:45:45.344Z: Expanding GroupByKey operations into optimizable parts.
Jun 16, 2020 6:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-16T18:45:45.386Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jun 16, 2020 6:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-16T18:45:45.471Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jun 16, 2020 6:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-16T18:45:45.502Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Jun 16, 2020 6:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-16T18:45:45.536Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Jun 16, 2020 6:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-16T18:45:45.570Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Jun 16, 2020 6:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-16T18:45:45.984Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Jun 16, 2020 6:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-16T18:45:46.066Z: Starting 5 workers in us-central1-a...
Jun 16, 2020 6:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-06-16T18:46:01.977Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jun 16, 2020 6:46:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-16T18:46:13.729Z: Autoscaling: Raised the number of workers to 3 based on the rate of progress in the currently running stage(s).
Jun 16, 2020 6:46:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-16T18:46:13.769Z: Resized worker pool to 3, though goal was 5. This could be a quota issue.
Jun 16, 2020 6:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-16T18:46:19.155Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Jun 16, 2020 6:46:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-16T18:46:32.224Z: Workers have started successfully.
Jun 16, 2020 6:46:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-16T18:46:32.257Z: Workers have started successfully.
Jun 16, 2020 6:47:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-16T18:47:00.703Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Jun 16, 2020 6:47:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-16T18:47:00.912Z: Cleaning up.
Jun 16, 2020 6:47:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-16T18:47:01.003Z: Stopping worker pool...
Jun 16, 2020 6:48:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-16T18:48:02.419Z: Autoscaling: Resized worker pool from 5 to 0.
Jun 16, 2020 6:48:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-16T18:48:02.454Z: Worker pool stopped.
Jun 16, 2020 6:48:10 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-06-16_11_45_37-15354772517132809839 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 81daf7f7-f447-4244-9bc5-ef16e7d31c24 and timestamp: 2020-06-16T18:48:10.940000000Z:
Metric: Value:
fields_read 4375276.0
read_time 14.256
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Jun 16, 2020 6:48:11 PM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.023 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.03 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 9,5,main]) completed. Took 2 mins 54.392 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 3m 44s
104 actionable tasks: 68 executed, 36 from cache
Publishing build scan...
https://gradle.com/s/mogo7yrggy4ea
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #634
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/634/display/redirect>
Changes:
------------------------------------------
[...truncated 294.00 KB...]
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:164)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Jun 16, 2020 12:45:36 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Jun 16, 2020 12:45:36 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery writeDisposition is set to: WRITE_EMPTY
Jun 16, 2020 12:45:37 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Jun 16, 2020 12:45:37 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Jun 16, 2020 12:45:37 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery writeDisposition is set to: WRITE_EMPTY
Jun 16, 2020 12:45:37 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Jun 16, 2020 12:45:37 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Jun 16, 2020 12:45:37 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Jun 16, 2020 12:45:38 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Jun 16, 2020 12:45:39 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 209 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Jun 16, 2020 12:45:40 PM org.apache.beam.runners.dataflow.util.PackageUtil$PackageAttributes forFileToStage
INFO: Staging custom dataflow-worker.jar as beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-fyJrwmFqniQxNZdi0JmBzDeu697hnUWIJLCk77uFvpQ.jar
Jun 16, 2020 12:45:40 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-kQRBqyYvemgd37q70qYuO_2FPIp5EXkc7Alwss79zQg.jar
Jun 16, 2020 12:45:40 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /tmp/test2905504233453814478.zip to gs://temp-storage-for-perf-tests/loadtests/staging/test-G380AIqWslrOJJ5LSP4s2wqA7HewkhE6RYWnIvNogZE.jar
Jun 16, 2020 12:45:40 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-tests-FbmXWAJNf4PbVvRvxrAn4gcNHetRk08Ja0_UhZ2Oms8.jar
Jun 16, 2020 12:45:40 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-tests-gqAlSWoHrMlVWqMmkc-gm5IYJFtAkkOEs04gNWsf0jc.jar
Jun 16, 2020 12:45:40 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT-8hhNnOJmFAqm0pfyNiJp6d0eq9N3mbtctTjrcvLjT6k.jar
Jun 16, 2020 12:45:40 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tests-h9iENZQi6HyZjrlojWukjhHB6T07addGdp8XKLu6pFI.jar
Jun 16, 2020 12:45:40 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-MjhDHAsTSB-SOep0MrjSC9ku8VtLb0p1Mfyj8_zBZio.jar
Jun 16, 2020 12:45:40 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.23.0-SNAPSHOT-eNBgI03MY5Fa3BM5VLkoAVtMqWGw6okeIzwHsVXyttw.jar
Jun 16, 2020 12:45:40 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.23.0-SNAPSHOT-W4DrGhLwe-PiB4MRZFvNKbkH9D1xeI0XRdENI3lgPRg.jar
Jun 16, 2020 12:45:40 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-tests-SRZm0_p6STEgCILAscRjFoipnJsju0YztPNF67S5Emo.jar
Jun 16, 2020 12:45:40 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-fyJrwmFqniQxNZdi0JmBzDeu697hnUWIJLCk77uFvpQ.jar
Jun 16, 2020 12:45:40 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-53AxH60uE1eVTp4rpZlAaJmkVnopKINJYpiY390TnQM.jar
Jun 16, 2020 12:45:40 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-1tuRF9qwz8GWhHor2tFt8WUr4o47lQX3PiZQRr7-Frc.jar
Jun 16, 2020 12:45:40 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.23.0-SNAPSHOT-jqHANPZZkgWDIkhPPS6MouU-YiBiXvjbfa3_cXjLLlg.jar
Jun 16, 2020 12:45:40 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-tests-lz8onNvm5zOauOeuWYhMm0QMXv2TAomRW0H30TIlVLg.jar
Jun 16, 2020 12:45:40 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.23.0-SNAPSHOT-tests-AUgVb9R_tPUY3BY22rX08dyZns_UgREUE6oycx8cQFc.jar
Jun 16, 2020 12:45:40 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT-EKtaRCWCi-pVPkMNmaNjChU4vUgu1WCC0QmCE4S4ApE.jar
Jun 16, 2020 12:45:40 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-tests-fP_Ckiqu3W4Fz824OUQ7X9_m1mpCDswy2_UqX4FumyE.jar
Jun 16, 2020 12:45:40 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.23.0-SNAPSHOT-K2-MQ8MSvDL3MEEIVlzjuOiBccU8BtrnXKAcRdPhfOM.jar
Jun 16, 2020 12:45:40 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-jMml8XgZ7VK2hgQCzZQ70FLxH5-lvLEsee0bGTWXXNU.jar
Jun 16, 2020 12:45:40 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-XhMYT2D2KQiT62tliMdiU3x2FCP55iSaArzEh0y-rjI.jar
Jun 16, 2020 12:45:40 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-42D1p6VRITTRLHXCIJFbMyzuOqTbUJq1Gkvb2sX9mIU.jar
Jun 16, 2020 12:45:40 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.23.0-SNAPSHOT-qsS7zTBO2YnLuft812w6najgCch9ll4DARVG0f1chvk.jar
Jun 16, 2020 12:45:40 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-tests-BsIeXb5ZAEs-UQTQL0IVANBdyLWuzbRzJuQvZp1u3GA.jar
Jun 16, 2020 12:45:40 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-unshaded-4RvotwHxIexgkYZR3VMT-Jm9JOZXMT04KGDgOyWlZlg.jar
Jun 16, 2020 12:45:40 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-fn-execution-2.23.0-SNAPSHOT-2OrVTa8yivXfW3h02lMS00fqT_jr6ZhKiZxBu6nwkVA.jar
Jun 16, 2020 12:45:40 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/java-fn-execution/build/libs/beam-runners-java-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-java-fn-execution-2.23.0-SNAPSHOT-EQPsKX8VCPbqbxyCY_ONFxn6TrDEJ2xRMqJPtCE6q4M.jar
Jun 16, 2020 12:45:40 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-fyJrwmFqniQxNZdi0JmBzDeu697hnUWIJLCk77uFvpQ.jar
Jun 16, 2020 12:45:40 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.23.0-SNAPSHOT-M9kFOTizuhq56PVUcqcFbduPgko3TOINBRi-dgQakf8.jar
Jun 16, 2020 12:45:41 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.23.0-SNAPSHOT-l1Nck0jf96w0x3BzLg1jYRc_93rfQIpLPumP1VPvf2I.jar
Jun 16, 2020 12:45:41 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.23.0-SNAPSHOT-23cDq7YMjsAPr12g9dljcOhHpJ3i9VbiMVRZUIS0qyg.jar
Jun 16, 2020 12:45:41 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-fyJrwmFqniQxNZdi0JmBzDeu697hnUWIJLCk77uFvpQ.jar&uploadType=resumable&upload_id=AAANsUmXKygIONfHIcSTYC3j45QtI-I4IsFDyYm2ll29JEmyJ5OnZj6UF7oT62Dw4-lczYu1Mb0_8GwfkjuAK0BNW_o.
Jun 16, 2020 12:45:41 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar>
java.io.IOException: Upload failed for 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-fyJrwmFqniQxNZdi0JmBzDeu697hnUWIJLCk77uFvpQ.jar'
at com.google.cloud.hadoop.util.BaseAbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(BaseAbstractGoogleAsyncWriteChannel.java:297)
at com.google.cloud.hadoop.util.BaseAbstractGoogleAsyncWriteChannel.close(BaseAbstractGoogleAsyncWriteChannel.java:214)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:221)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:168)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:152)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:136)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Jun 16, 2020 12:45:45 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-fyJrwmFqniQxNZdi0JmBzDeu697hnUWIJLCk77uFvpQ.jar
Jun 16, 2020 12:45:46 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 31 files newly uploaded in 6 seconds
Jun 16, 2020 12:45:46 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Jun 16, 2020 12:45:46 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Jun 16, 2020 12:45:46 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Jun 16, 2020 12:45:46 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Jun 16, 2020 12:45:46 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Jun 16, 2020 12:45:46 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <91369 bytes, hash 1b5842fb6343874bc399f45ee15d674e1d0755a644193eb1696a4284a9ae49de> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-G1hC-2NDh0vDmfRe4V1nTh0HVaZEGT6xaWpChKmuSd4.pb
Jun 16, 2020 12:45:46 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.23.0-SNAPSHOT
Jun 16, 2020 12:45:47 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-06-16_05_45_46-8302130818230928812?project=apache-beam-testing
Jun 16, 2020 12:45:47 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-06-16_05_45_46-8302130818230928812
Jun 16, 2020 12:45:47 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-06-16_05_45_46-8302130818230928812
Jun 16, 2020 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-06-16T12:45:46.861Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Jun 16, 2020 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-16T12:45:52.918Z: Worker configuration: n1-standard-1 in us-central1-a.
Jun 16, 2020 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-16T12:45:53.654Z: Expanding CoGroupByKey operations into optimizable parts.
Jun 16, 2020 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-16T12:45:53.974Z: Expanding GroupByKey operations into optimizable parts.
Jun 16, 2020 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-16T12:45:54.005Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jun 16, 2020 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-16T12:45:54.092Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jun 16, 2020 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-16T12:45:54.134Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Jun 16, 2020 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-16T12:45:54.171Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Jun 16, 2020 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-16T12:45:54.206Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Jun 16, 2020 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-16T12:45:54.565Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Jun 16, 2020 12:45:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-16T12:45:54.662Z: Starting 5 workers in us-central1-a...
Jun 16, 2020 12:46:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-06-16T12:46:09.520Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jun 16, 2020 12:46:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-16T12:46:30.458Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Jun 16, 2020 12:46:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-16T12:46:48.547Z: Workers have started successfully.
Jun 16, 2020 12:46:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-16T12:46:48.585Z: Workers have started successfully.
Jun 16, 2020 12:47:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-16T12:47:18.106Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Jun 16, 2020 12:47:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-16T12:47:18.293Z: Cleaning up.
Jun 16, 2020 12:47:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-16T12:47:18.383Z: Stopping worker pool...
Jun 16, 2020 12:49:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-16T12:49:16.394Z: Autoscaling: Resized worker pool from 5 to 0.
Jun 16, 2020 12:49:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-16T12:49:16.443Z: Worker pool stopped.
Jun 16, 2020 12:49:22 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-06-16_05_45_46-8302130818230928812 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): f14dd896-8403-4f39-b43a-e930aad16a3a and timestamp: 2020-06-16T12:49:22.154000000Z:
Metric: Value:
fields_read 4375276.0
read_time 13.825
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Jun 16, 2020 12:49:23 PM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.07 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.068 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 8,5,main]) completed. Took 3 mins 55.774 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 1s
104 actionable tasks: 68 executed, 36 from cache
Publishing build scan...
https://gradle.com/s/xurdvz2rt3nfc
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #633
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/633/display/redirect>
Changes:
------------------------------------------
[...truncated 292.89 KB...]
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Jun 16, 2020 6:45:18 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Jun 16, 2020 6:45:18 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery writeDisposition is set to: WRITE_EMPTY
Jun 16, 2020 6:45:18 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Jun 16, 2020 6:45:18 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Jun 16, 2020 6:45:18 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery writeDisposition is set to: WRITE_EMPTY
Jun 16, 2020 6:45:18 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Jun 16, 2020 6:45:18 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Jun 16, 2020 6:45:18 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Jun 16, 2020 6:45:19 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Jun 16, 2020 6:45:20 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 209 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Jun 16, 2020 6:45:20 AM org.apache.beam.runners.dataflow.util.PackageUtil$PackageAttributes forFileToStage
INFO: Staging custom dataflow-worker.jar as beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-_NiFA_SzUYFcNBnm3KnJmLrqoIkZUi2p_eJksde3ZKs.jar
Jun 16, 2020 6:45:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.23.0-SNAPSHOT-7KB1ZkTyBvYWZnkfRvo_dfO5GLS-urJxgH5daKaT9cA.jar
Jun 16, 2020 6:45:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-8G5YGy_avdI3vHDmZDTEDNlpQJSXajt3VOy--IvOgyk.jar
Jun 16, 2020 6:45:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-1pzckp1Et5bQu35tnQ1KCD3NGQKEwfmfSl9mRE08Z9g.jar
Jun 16, 2020 6:45:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tests-Cp2D-eelwr7il8B-PP3CplAtiGmKiP-tybo6ps5cjRE.jar
Jun 16, 2020 6:45:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.23.0-SNAPSHOT-eMLkI6zk7mGNzW3I_7x6opUH0e0vrdwPfmaeNq7cbBQ.jar
Jun 16, 2020 6:45:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT-CT0ZQa7eiaZWTDp9qxr4X3gVub3sNKb4YsW4swrIleg.jar
Jun 16, 2020 6:45:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-tests-sytjlDnJ_3Klp9cm3uiNTFZA0bJZBKYYFEvWb8vZpPE.jar
Jun 16, 2020 6:45:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-7pWB6ZvP9mECvtAHYK1T2TV7O83YvE46Mk23xMobR7c.jar
Jun 16, 2020 6:45:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-dnbo69ehaSKQPrgL3tmJbHfgoAQYzG_YxUMtO9B8jFU.jar
Jun 16, 2020 6:45:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-tests-OcNSVq7tg1Xb94Rmk1bomW_BSHk-Styfm0okepZdRfE.jar
Jun 16, 2020 6:45:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.23.0-SNAPSHOT-tests-cVg-31WW1Qu5WK6oSRmIB7DaE77g5z9Aypuenp7Fzzo.jar
Jun 16, 2020 6:45:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/java-fn-execution/build/libs/beam-runners-java-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-java-fn-execution-2.23.0-SNAPSHOT-r1qEeBTdg9ZFnMXptXNEFlqd3gOAmJRMoiTqlLn3_oY.jar
Jun 16, 2020 6:45:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.23.0-SNAPSHOT-lL7TNFsbOmxp2da1h_Q406FNqZC7dFUp8H_IWRGd424.jar
Jun 16, 2020 6:45:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-tests-qTowwjGDfSh7h7XOfRki51EXPaCxswPdygHVp3zZLhs.jar
Jun 16, 2020 6:45:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.23.0-SNAPSHOT--E2LOoxDSmjeDnpWHGgEfwh64jwjco_wLwIxcV8gQas.jar
Jun 16, 2020 6:45:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-tests-Q67ao54-Jld3QLpYx_7fZVaQ_lJnsqxuKEPRfCvJ9cM.jar
Jun 16, 2020 6:45:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-tests-6rX53XoMxjOfkoe62aCPNbeaLNXRI1Q1jI2iPdK01Uw.jar
Jun 16, 2020 6:45:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.23.0-SNAPSHOT-zNGAr3CGF9svDwiRjXcsB5ahLXy8Q4EafMbpnLyHsZA.jar
Jun 16, 2020 6:45:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-_NiFA_SzUYFcNBnm3KnJmLrqoIkZUi2p_eJksde3ZKs.jar
Jun 16, 2020 6:45:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-4yM2Nhpt4AlYAMOzezPqK7iH4hVgVfsoDyigulUJULs.jar
Jun 16, 2020 6:45:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.23.0-SNAPSHOT-RkgiFRvMn7hhLwKjO65KlXCAUdwEP2tr15KR9D-2mo4.jar
Jun 16, 2020 6:45:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-fn-execution-2.23.0-SNAPSHOT-iZr0ZKq2Jn7Zke7g7MgM2__yBcgftYjKNLiBQ7P1pMw.jar
Jun 16, 2020 6:45:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /tmp/test5777873280444244219.zip to gs://temp-storage-for-perf-tests/loadtests/staging/test-eQMaS5CT6zZIm1LjaS0lkG4KDCuPVF-ih0Mx31yFtvI.jar
Jun 16, 2020 6:45:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT-I7IlV-m8WIfXsKbBPiWYaWcow8RZ958ThW3LxCVZ8uk.jar
Jun 16, 2020 6:45:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-UgsDL2-XVPYk_DQT7cd38dGZsO9OdojiBVOSCtD8Udc.jar
Jun 16, 2020 6:45:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-tests-AKZ2EpeAisE9MsKizvQ_fuRsoyP3AK6pErFmIlbY8yI.jar
Jun 16, 2020 6:45:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-unshaded-eNRfXIi9PD4ubPngiyi-9Z9sOLYjZ2edev2jvu2p-Oc.jar
Jun 16, 2020 6:45:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-_NiFA_SzUYFcNBnm3KnJmLrqoIkZUi2p_eJksde3ZKs.jar
Jun 16, 2020 6:45:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.23.0-SNAPSHOT-dacQnD13cT1nZryl1KDDur6PCc7fUyCaxew3HH_rTBs.jar
Jun 16, 2020 6:45:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.23.0-SNAPSHOT-jkhCXEdOqUH3vuCEc-5p_oLawrf7sowURTBzLLON_og.jar
Jun 16, 2020 6:45:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.23.0-SNAPSHOT-jTVUDHWEHy_7LSPAGBnjsSeEWwxqn9vjWoThIPP_UPU.jar
Jun 16, 2020 6:45:22 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-_NiFA_SzUYFcNBnm3KnJmLrqoIkZUi2p_eJksde3ZKs.jar&uploadType=resumable&upload_id=AAANsUkBPBqVEe7xTe2MWH7u41qodA7JcnvSgp2SF7pOyU4Z3qgHvaFVgXZL3HQkLo69qjjnXy0wLNFEv9R1IgDEpro.
Jun 16, 2020 6:45:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar>
java.io.IOException: Upload failed for 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-_NiFA_SzUYFcNBnm3KnJmLrqoIkZUi2p_eJksde3ZKs.jar'
at com.google.cloud.hadoop.util.BaseAbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(BaseAbstractGoogleAsyncWriteChannel.java:297)
at com.google.cloud.hadoop.util.BaseAbstractGoogleAsyncWriteChannel.close(BaseAbstractGoogleAsyncWriteChannel.java:214)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:221)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:168)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:152)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:136)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Jun 16, 2020 6:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-_NiFA_SzUYFcNBnm3KnJmLrqoIkZUi2p_eJksde3ZKs.jar
Jun 16, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 31 files newly uploaded in 9 seconds
Jun 16, 2020 6:45:30 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Jun 16, 2020 6:45:30 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Jun 16, 2020 6:45:30 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Jun 16, 2020 6:45:30 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Jun 16, 2020 6:45:30 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Jun 16, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <91368 bytes, hash fcc290a45e1c53de42e46d4671614f0c11d93c527b0aaffb6e8f324fbd3a8304> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-_MKQpF4cU95C5G1GcWFPDBHZPFJ7Cq_7bo8yT706gwQ.pb
Jun 16, 2020 6:45:31 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.23.0-SNAPSHOT
Jun 16, 2020 6:45:32 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-06-15_23_45_31-16236925608730554007?project=apache-beam-testing
Jun 16, 2020 6:45:32 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-06-15_23_45_31-16236925608730554007
Jun 16, 2020 6:45:32 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-06-15_23_45_31-16236925608730554007
Jun 16, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-06-16T06:45:31.397Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Jun 16, 2020 6:45:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-16T06:45:40.461Z: Worker configuration: n1-standard-1 in us-central1-a.
Jun 16, 2020 6:45:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-16T06:45:41.407Z: Expanding CoGroupByKey operations into optimizable parts.
Jun 16, 2020 6:45:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-16T06:45:41.439Z: Expanding GroupByKey operations into optimizable parts.
Jun 16, 2020 6:45:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-16T06:45:41.484Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jun 16, 2020 6:45:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-16T06:45:41.563Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jun 16, 2020 6:45:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-16T06:45:41.597Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Jun 16, 2020 6:45:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-16T06:45:41.638Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Jun 16, 2020 6:45:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-16T06:45:41.670Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Jun 16, 2020 6:45:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-16T06:45:42.025Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Jun 16, 2020 6:45:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-16T06:45:42.098Z: Starting 5 workers in us-central1-a...
Jun 16, 2020 6:46:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-16T06:46:08.280Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running stage(s).
Jun 16, 2020 6:46:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-16T06:46:08.318Z: Resized worker pool to 4, though goal was 5. This could be a quota issue.
Jun 16, 2020 6:46:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-06-16T06:46:11.252Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jun 16, 2020 6:46:15 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-16T06:46:13.670Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Jun 16, 2020 6:46:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-16T06:46:26.189Z: Workers have started successfully.
Jun 16, 2020 6:46:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-16T06:46:26.220Z: Workers have started successfully.
Jun 16, 2020 6:47:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-16T06:47:03.738Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Jun 16, 2020 6:47:05 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-16T06:47:03.944Z: Cleaning up.
Jun 16, 2020 6:47:05 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-16T06:47:04.029Z: Stopping worker pool...
Jun 16, 2020 6:48:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-16T06:48:37.886Z: Autoscaling: Resized worker pool from 5 to 0.
Jun 16, 2020 6:48:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-16T06:48:37.937Z: Worker pool stopped.
Jun 16, 2020 6:48:45 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-06-15_23_45_31-16236925608730554007 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 8f5aaa96-389d-4a69-93d8-a3f07af6654b and timestamp: 2020-06-16T06:48:45.826000000Z:
Metric: Value:
fields_read 4375276.0
read_time 20.91
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Jun 16, 2020 6:48:46 AM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.027 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.039 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 4,5,main]) completed. Took 3 mins 36.975 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 26s
104 actionable tasks: 68 executed, 36 from cache
Publishing build scan...
https://gradle.com/s/bgqfq4upy6pms
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #632
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/632/display/redirect?page=changes>
Changes:
[davidyan] [BEAM-10247] Pin google-api-core to 1.17.0, because otherwise the pulled
[davidyan] Bumping grpcio version to 1.29.0 to be compatible with
[ningk] Update screen_diff deps and goldens as stable Chrome version advances.
[robinyqiu] Add zetaSqlValueToJavaObject() with unknown target type
[daniel.o.programmer] [BEAM-9951] Fixing some lint bugs.
[stuart.m.perks] BEAM-10221: Add in four tests cases of base on the java equivalent for
[davidyan] added rsa<4.1 for python2
[bhulette] Lump together PMC-only steps
[github] Clarify release guide for publishing release notes to GitHub (#12015)
------------------------------------------
[...truncated 294.94 KB...]
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:164)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Jun 16, 2020 12:56:18 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Jun 16, 2020 12:56:18 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery writeDisposition is set to: WRITE_EMPTY
Jun 16, 2020 12:56:18 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Jun 16, 2020 12:56:18 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Jun 16, 2020 12:56:18 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery writeDisposition is set to: WRITE_EMPTY
Jun 16, 2020 12:56:18 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Jun 16, 2020 12:56:19 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Jun 16, 2020 12:56:19 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Jun 16, 2020 12:56:19 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Jun 16, 2020 12:56:22 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 209 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Jun 16, 2020 12:56:22 AM org.apache.beam.runners.dataflow.util.PackageUtil$PackageAttributes forFileToStage
INFO: Staging custom dataflow-worker.jar as beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-hpLtTGzu-RLXQ26WpbDQOCW5NQClv4iMn2faOCasmZQ.jar
Jun 16, 2020 12:56:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.23.0-SNAPSHOT-AV1LAQ9wsCmnvJbE2lkOGqxhkWtKWkjlZ7vGu6K2Yz0.jar
Jun 16, 2020 12:56:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /tmp/test6681815561811096882.zip to gs://temp-storage-for-perf-tests/loadtests/staging/test-h7PQXyZF9o7gcAZE2S0fPcBtf13FIA03ToaSY1frKSY.jar
Jun 16, 2020 12:56:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-ski1g-wMG9TNit-i2BlZtCB68zUgqf4ihV-W1M2leG0.jar
Jun 16, 2020 12:56:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-tests-LZGiiZ0-vet2ctqc0FqsVrr-q3yi5-0m061AbQpxUZk.jar
Jun 16, 2020 12:56:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-nPS_3WYjbqI186dtdytJAph42uyXnji44llhn9Ma4Q0.jar
Jun 16, 2020 12:56:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-kpdjMbWyKP6oFUiJsKynXb0SoPUwEX7cGdWONp0S-Os.jar
Jun 16, 2020 12:56:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-ztu8mEtWBi9qf73T7AuBG8edARihLfDRi8iIsviqwTY.jar
Jun 16, 2020 12:56:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-tests-Akn09u9TK5rRB8nr02UfUx5ql-nMoKCp0PjNpRC2UCQ.jar
Jun 16, 2020 12:56:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT-8hopGAhyPCcj7tBokeZitcc2GxQnawjSZaBHwvKae2E.jar
Jun 16, 2020 12:56:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.23.0-SNAPSHOT-k-O8CVg0knh754JeJFKNQSZ6ibennTDDKo1cpWV-wuE.jar
Jun 16, 2020 12:56:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tests-tTSkXAeo747D8smY2448n3tpwuZ4hbm-0Qp_4rXYUZw.jar
Jun 16, 2020 12:56:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-RAOvCfqv7pnmWWB6Ekq4gZAwuvG4q4iPdnrhcCZCZB4.jar
Jun 16, 2020 12:56:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/java-fn-execution/build/libs/beam-runners-java-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-java-fn-execution-2.23.0-SNAPSHOT-6uiGIaVB1GupJGTtl7W_OSAGTXz4oXcB0qu7xxpXUXU.jar
Jun 16, 2020 12:56:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-fn-execution-2.23.0-SNAPSHOT-804xF6xAFYgL7IwOczZ4M8aCELHJRgGg4GWcMmiAjWw.jar
Jun 16, 2020 12:56:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-tests-l2hj4xTPxaAYEjf6PZtmSrSLribVtA-ky4j_E12hG4M.jar
Jun 16, 2020 12:56:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-tests-2UX9UnxfIwXCY9wa2zsRUXGddqEaNiM0AmqV4dbBnyc.jar
Jun 16, 2020 12:56:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT-e8uYqK7TuBkj2bM_nSoxpT8U7u5ljHCyC7CC3REEdXo.jar
Jun 16, 2020 12:56:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-OLt6XQNQlmrGh3JBqFzi2JzWZSihNf22hhuIOItZGns.jar
Jun 16, 2020 12:56:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-hpLtTGzu-RLXQ26WpbDQOCW5NQClv4iMn2faOCasmZQ.jar
Jun 16, 2020 12:56:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.23.0-SNAPSHOT-VRhaEhdb4yjq_cJwDFxhI90nkJGVz_SMWSCuyiLzw-Q.jar
Jun 16, 2020 12:56:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.23.0-SNAPSHOT-tests-awtUKuDdBvrqqnHCZiwDDQ0CAJohHfbOBS9BMQ26PCI.jar
Jun 16, 2020 12:56:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-unshaded-aa9QTEE4gI1H7YquGvcCAYFVXe6eJ7QyH5I30z3ixrM.jar
Jun 16, 2020 12:56:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.23.0-SNAPSHOT-57XEDMFKcxU0ni2GH_as2LKTVpghvqjn16MjPUuduiw.jar
Jun 16, 2020 12:56:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-1MxC2YzZ7S_M5JJ-lRUcJ56LH4WFOk1VGIUG67pUr6o.jar
Jun 16, 2020 12:56:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-tests-b6Qw8KCQihz3RBvY8YpsmWrM69w-mM9dwKSd_OzfVgA.jar
Jun 16, 2020 12:56:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.23.0-SNAPSHOT-Tdp46UyF2fkQxSzY3FKw66MmvLX6AMIk5J6iyCe4sds.jar
Jun 16, 2020 12:56:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-tests-hN9fKopbhW8et8NFKD1RsJbnb0XFTevQFNl4ZHhADGI.jar
Jun 16, 2020 12:56:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-hpLtTGzu-RLXQ26WpbDQOCW5NQClv4iMn2faOCasmZQ.jar
Jun 16, 2020 12:56:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.23.0-SNAPSHOT-Gpwa1lAWgra1uxYCC3YwaZspZ3ESfK2-b7KW6Z0fW2A.jar
Jun 16, 2020 12:56:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.23.0-SNAPSHOT-0E2sCGyD57pkZVQhI0xKoIFXainO09aUsXx6Pdj7rSk.jar
Jun 16, 2020 12:56:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.23.0-SNAPSHOT-t4Mj8S4odFPT8epg3TE5cd9kMyntGCQBQ9dAWLxdBeA.jar
Jun 16, 2020 12:56:23 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-hpLtTGzu-RLXQ26WpbDQOCW5NQClv4iMn2faOCasmZQ.jar&uploadType=resumable&upload_id=AAANsUkyjE-_GSb0HP70M-4Y-S8KmOF6Bl4gqsvhJU_I2p_kLsLgL1S7LHIOS_Vxd9BB09w4FnqM0kulvAH1LlDo8o4.
Jun 16, 2020 12:56:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar>
java.io.IOException: Upload failed for 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-hpLtTGzu-RLXQ26WpbDQOCW5NQClv4iMn2faOCasmZQ.jar'
at com.google.cloud.hadoop.util.BaseAbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(BaseAbstractGoogleAsyncWriteChannel.java:297)
at com.google.cloud.hadoop.util.BaseAbstractGoogleAsyncWriteChannel.close(BaseAbstractGoogleAsyncWriteChannel.java:214)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:221)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:168)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:152)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:136)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Jun 16, 2020 12:56:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-hpLtTGzu-RLXQ26WpbDQOCW5NQClv4iMn2faOCasmZQ.jar
Jun 16, 2020 12:56:30 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 31 files newly uploaded in 8 seconds
Jun 16, 2020 12:56:31 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Jun 16, 2020 12:56:31 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Jun 16, 2020 12:56:31 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Jun 16, 2020 12:56:31 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Jun 16, 2020 12:56:31 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Jun 16, 2020 12:56:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <91368 bytes, hash 1e57f2cae72de1587ec231fec11f1f4a0b85d34fbcdd8eae00531fed83985f93> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-Hlfyyuct4Vh-wjH-wR8fSguF00-83Y6uAFMf7YOYX5M.pb
Jun 16, 2020 12:56:31 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.23.0-SNAPSHOT
Jun 16, 2020 12:56:32 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-06-15_17_56_31-7323517975836051540?project=apache-beam-testing
Jun 16, 2020 12:56:32 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-06-15_17_56_31-7323517975836051540
Jun 16, 2020 12:56:32 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-06-15_17_56_31-7323517975836051540
Jun 16, 2020 12:56:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-06-16T00:56:31.729Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Jun 16, 2020 12:56:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-16T00:56:39.655Z: Worker configuration: n1-standard-1 in us-central1-a.
Jun 16, 2020 12:56:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-16T00:56:40.565Z: Expanding CoGroupByKey operations into optimizable parts.
Jun 16, 2020 12:56:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-16T00:56:40.636Z: Expanding GroupByKey operations into optimizable parts.
Jun 16, 2020 12:56:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-16T00:56:40.687Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jun 16, 2020 12:56:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-16T00:56:40.833Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jun 16, 2020 12:56:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-16T00:56:40.888Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Jun 16, 2020 12:56:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-16T00:56:40.927Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Jun 16, 2020 12:56:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-16T00:56:40.960Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Jun 16, 2020 12:56:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-16T00:56:41.548Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Jun 16, 2020 12:56:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-16T00:56:41.637Z: Starting 5 workers in us-central1-a...
Jun 16, 2020 12:56:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-06-16T00:56:47.292Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jun 16, 2020 12:57:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-16T00:57:08.106Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Jun 16, 2020 12:57:28 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-16T00:57:27.107Z: Workers have started successfully.
Jun 16, 2020 12:57:28 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-16T00:57:27.142Z: Workers have started successfully.
Jun 16, 2020 12:57:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-16T00:57:58.903Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Jun 16, 2020 12:58:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-16T00:57:59.241Z: Cleaning up.
Jun 16, 2020 12:58:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-16T00:57:59.380Z: Stopping worker pool...
Jun 16, 2020 12:59:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-16T00:59:51.537Z: Autoscaling: Resized worker pool from 5 to 0.
Jun 16, 2020 12:59:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-16T00:59:51.580Z: Worker pool stopped.
Jun 16, 2020 12:59:57 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-06-15_17_56_31-7323517975836051540 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 2d199eca-0b0e-4d4b-bc5d-10606c5d67e8 and timestamp: 2020-06-16T00:59:57.366000000Z:
Metric: Value:
fields_read 4375276.0
read_time 15.443
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Jun 16, 2020 12:59:57 AM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 2 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.022 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.029 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':',5,main]) completed. Took 3 mins 48.783 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 57s
104 actionable tasks: 70 executed, 34 from cache
Publishing build scan...
https://gradle.com/s/nvbiel77mc5lg
Stopped 1 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #631
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/631/display/redirect?page=changes>
Changes:
[mxm] [BEAM-10249] Populate state cache with initial values before appending
------------------------------------------
[...truncated 294.25 KB...]
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:164)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Jun 15, 2020 7:01:09 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Jun 15, 2020 7:01:09 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery writeDisposition is set to: WRITE_EMPTY
Jun 15, 2020 7:01:09 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Jun 15, 2020 7:01:09 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Jun 15, 2020 7:01:09 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery writeDisposition is set to: WRITE_EMPTY
Jun 15, 2020 7:01:09 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Jun 15, 2020 7:01:09 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Jun 15, 2020 7:01:09 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Jun 15, 2020 7:01:10 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Jun 15, 2020 7:01:12 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 209 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Jun 15, 2020 7:01:12 PM org.apache.beam.runners.dataflow.util.PackageUtil$PackageAttributes forFileToStage
INFO: Staging custom dataflow-worker.jar as beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-6VKMms7ekxBLNFatb3jdwIg66fbBEhSs3pIhyKySpYE.jar
Jun 15, 2020 7:01:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-tests-CygyqAkAVhWS7nisGrSwuqoDNuCtwjNKWjQcp-sRBJk.jar
Jun 15, 2020 7:01:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-ZGdfw3YVXKw-ofukFckzyZQTLNZ3QHJLKj4VFFdurYo.jar
Jun 15, 2020 7:01:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-tests-RjvCDFYlQhhs3txaCdKmjR_Wq3KkONq6GcKKD2dIJVU.jar
Jun 15, 2020 7:01:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT-5VhNF0WZT_KzklQK6y2nHlksxtU6eRLxkPGIIU0PqN0.jar
Jun 15, 2020 7:01:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/java-fn-execution/build/libs/beam-runners-java-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-java-fn-execution-2.23.0-SNAPSHOT-WN7BwxMilesttpCAPtfLINal1oINpTOdBFAvVSxsE4Y.jar
Jun 15, 2020 7:01:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-zgjz_lhTt03nRvNmB3vCFns0Q5XTfOGWG0otz9r2S9o.jar
Jun 15, 2020 7:01:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-CFCV7n_R1r7_Ym8-SEFooYONBuPzC0TOKBIaZ25eVaU.jar
Jun 15, 2020 7:01:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-tests-rQMF9fPhCt0KKSVBQn4l0cfB6zHlM38CeYV3A9dZUSY.jar
Jun 15, 2020 7:01:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-tests-xUvWxrQkPFO0ZohWrdLJqgt80BjzlYnljlfW72o7GxA.jar
Jun 15, 2020 7:01:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.23.0-SNAPSHOT-tests-96q_99MFAlffYoZ1MMl-a8bVNHeA1c4YpxFvhMv_quY.jar
Jun 15, 2020 7:01:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-6VKMms7ekxBLNFatb3jdwIg66fbBEhSs3pIhyKySpYE.jar
Jun 15, 2020 7:01:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.23.0-SNAPSHOT-gEzOp3NgjDer3-PUgoX7pyj6y2-4sIjtEIMC3F9uPJY.jar
Jun 15, 2020 7:01:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-tests-6BjM2JwTL_APmXJ0-DkBWY9xxxYb_7T6UoRcUNwT1Kw.jar
Jun 15, 2020 7:01:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-fQ_TMBDhafBDAwgXXQaIJMbLgNP5Wd1YLiGJweh_o3w.jar
Jun 15, 2020 7:01:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-kP8vNY14ZjSUx_ItCqZ-EniezDZlUh9XN7f-DryB2-Q.jar
Jun 15, 2020 7:01:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-mRWhwpmQBLauldInqhpq1JZWm78oW_vQ2B8vLhU2JMo.jar
Jun 15, 2020 7:01:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tests-7gw2ODHUN6UMmkGpj5D1oc6lV5k82i6qlFg38YK6jd4.jar
Jun 15, 2020 7:01:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.23.0-SNAPSHOT-vYISozhyRgQvkaQYxka0i-A-CcYkpGEUe-GH-2Ggo7k.jar
Jun 15, 2020 7:01:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /tmp/test812673620175011487.zip to gs://temp-storage-for-perf-tests/loadtests/staging/test-XuHSsVB187EQ7AWbUd1FGv4xH-AtSGc4V_uJHRI8fj8.jar
Jun 15, 2020 7:01:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.23.0-SNAPSHOT-3D4fTMVnmWaXflOht8Ffjs7w9cb9s00-dg7evizXvMM.jar
Jun 15, 2020 7:01:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.23.0-SNAPSHOT-TQMZaQF40ntQAvOtSO73Uklh6o-wAkfGAszpV49xJtg.jar
Jun 15, 2020 7:01:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT-cUDEjemu9-EZZJbxd6oRmvtoAlAjXkrV1QDUkOX8Oys.jar
Jun 15, 2020 7:01:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-fn-execution-2.23.0-SNAPSHOT-B8ZPPBIC2IGkneFmqDfGuWdlg8XWw3DTW9nGjk-OPrw.jar
Jun 15, 2020 7:01:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.23.0-SNAPSHOT-3OEqRVVOIJ5zXby91xaogWCq4LpC89s-j6qZas9M-eY.jar
Jun 15, 2020 7:01:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-tests-Q7fOXLw_X2NJWUjKYfMNESM-HakrE0frkkcNzdLlTRM.jar
Jun 15, 2020 7:01:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-unshaded-4WmeE09Zf6Np-w8wto4qNAqVvwyb2gIeplcjCAqr8BY.jar
Jun 15, 2020 7:01:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-8RLYeeF_YPGcQWcvBZylKqir8mfn6jw6DwDlK9S0KSk.jar
Jun 15, 2020 7:01:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-6VKMms7ekxBLNFatb3jdwIg66fbBEhSs3pIhyKySpYE.jar
Jun 15, 2020 7:01:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.23.0-SNAPSHOT-tvW8FQWXOrxPEKIX-hjTpcdSO2RqPCVtFzRHQ0zoIKM.jar
Jun 15, 2020 7:01:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.23.0-SNAPSHOT-wRpa2LcQ7mmkRcu4dXnENwFGhZb_ssjFIR9VYKBBGWo.jar
Jun 15, 2020 7:01:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.23.0-SNAPSHOT-WvJj86HGsiNVrpXTWtP6Ji-AO2fFeeV0riEDxh5yfA4.jar
Jun 15, 2020 7:01:14 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-6VKMms7ekxBLNFatb3jdwIg66fbBEhSs3pIhyKySpYE.jar&uploadType=resumable&upload_id=AAANsUkjbg4IVJsyG5dwjp7tg2j_B7xiG9CaqiPSn6vebpggyoZ30GYwUaMc7tUjb1s1lNWBfpSTek9KVX1cpGHuPqX11Jrnhw.
Jun 15, 2020 7:01:14 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar>
java.io.IOException: Upload failed for 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-6VKMms7ekxBLNFatb3jdwIg66fbBEhSs3pIhyKySpYE.jar'
at com.google.cloud.hadoop.util.BaseAbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(BaseAbstractGoogleAsyncWriteChannel.java:297)
at com.google.cloud.hadoop.util.BaseAbstractGoogleAsyncWriteChannel.close(BaseAbstractGoogleAsyncWriteChannel.java:214)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:221)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:168)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:152)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:136)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Jun 15, 2020 7:01:18 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-6VKMms7ekxBLNFatb3jdwIg66fbBEhSs3pIhyKySpYE.jar
Jun 15, 2020 7:01:19 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 31 files newly uploaded in 7 seconds
Jun 15, 2020 7:01:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Jun 15, 2020 7:01:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Jun 15, 2020 7:01:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Jun 15, 2020 7:01:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Jun 15, 2020 7:01:19 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Jun 15, 2020 7:01:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <91367 bytes, hash a33e09750fd9ea80441eff9fa5732af64b52b42203121b7be8fdaf04bff3cb56> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-oz4JdQ_Z6oBEHv-fpXMq9ktStCIDEht76P2vBL_zy1Y.pb
Jun 15, 2020 7:01:20 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.23.0-SNAPSHOT
Jun 15, 2020 7:01:21 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-06-15_12_01_20-11883618630077928799?project=apache-beam-testing
Jun 15, 2020 7:01:21 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-06-15_12_01_20-11883618630077928799
Jun 15, 2020 7:01:21 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-06-15_12_01_20-11883618630077928799
Jun 15, 2020 7:01:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-06-15T19:01:20.217Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Jun 15, 2020 7:01:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-15T19:01:27.788Z: Worker configuration: n1-standard-1 in us-central1-a.
Jun 15, 2020 7:01:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-15T19:01:28.706Z: Expanding CoGroupByKey operations into optimizable parts.
Jun 15, 2020 7:01:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-15T19:01:28.744Z: Expanding GroupByKey operations into optimizable parts.
Jun 15, 2020 7:01:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-15T19:01:28.785Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jun 15, 2020 7:01:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-15T19:01:28.880Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jun 15, 2020 7:01:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-15T19:01:28.913Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Jun 15, 2020 7:01:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-15T19:01:28.963Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Jun 15, 2020 7:01:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-15T19:01:29.003Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Jun 15, 2020 7:01:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-15T19:01:29.410Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Jun 15, 2020 7:01:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-15T19:01:29.567Z: Starting 5 workers in us-central1-a...
Jun 15, 2020 7:01:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-06-15T19:01:44.169Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jun 15, 2020 7:01:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-15T19:01:57.331Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Jun 15, 2020 7:02:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-15T19:02:13.485Z: Workers have started successfully.
Jun 15, 2020 7:02:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-15T19:02:13.523Z: Workers have started successfully.
Jun 15, 2020 7:02:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-15T19:02:50.469Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Jun 15, 2020 7:02:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-15T19:02:50.733Z: Cleaning up.
Jun 15, 2020 7:02:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-15T19:02:50.824Z: Stopping worker pool...
Jun 15, 2020 7:04:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-15T19:04:57.684Z: Autoscaling: Resized worker pool from 5 to 0.
Jun 15, 2020 7:04:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-15T19:04:57.730Z: Worker pool stopped.
Jun 15, 2020 7:05:02 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-06-15_12_01_20-11883618630077928799 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 10ca86c3-69f2-4833-ab07-3cde36afe52f and timestamp: 2020-06-15T19:05:03.015000000Z:
Metric: Value:
fields_read 4375276.0
read_time 17.839
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Jun 15, 2020 7:05:03 PM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 2 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.038 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.063 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 7,5,main]) completed. Took 4 mins 2.195 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 6s
104 actionable tasks: 70 executed, 34 from cache
Publishing build scan...
https://gradle.com/s/3edzmjcj5rc5o
Stopped 1 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #630
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/630/display/redirect>
Changes:
------------------------------------------
[...truncated 293.88 KB...]
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:164)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Jun 15, 2020 12:45:21 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Jun 15, 2020 12:45:21 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery writeDisposition is set to: WRITE_EMPTY
Jun 15, 2020 12:45:21 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Jun 15, 2020 12:45:21 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Jun 15, 2020 12:45:21 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery writeDisposition is set to: WRITE_EMPTY
Jun 15, 2020 12:45:21 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Jun 15, 2020 12:45:21 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Jun 15, 2020 12:45:21 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Jun 15, 2020 12:45:22 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Jun 15, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 209 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Jun 15, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil$PackageAttributes forFileToStage
INFO: Staging custom dataflow-worker.jar as beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-vkoZO5fFksaawKWb1JOW5ch_1NJwo6R2JIe3JeVBqt0.jar
Jun 15, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-7anj-zNzJbvkp6JhmykIKTs640wEpTq6-ZocyN_PGzk.jar
Jun 15, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.23.0-SNAPSHOT-DH-_i0o4WSOHUnr44qJg5U2VKX4M07MW2Ta7QVnt8Fs.jar
Jun 15, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-vkoZO5fFksaawKWb1JOW5ch_1NJwo6R2JIe3JeVBqt0.jar
Jun 15, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-tests-nsISz_JeUg69e3a3drSwMsMS_VsCKVjmapC6cQLcnqs.jar
Jun 15, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-tests-2w2iUb_ZicsZ0kwq30cnqOc5YUx8kY26OeVix22nb6Q.jar
Jun 15, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.23.0-SNAPSHOT-P2lSw3H_qtHFqDQhRdl045xGT3WagVLJhvWECiJOME8.jar
Jun 15, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-fn-execution-2.23.0-SNAPSHOT-Sb8uQudD2Z04X_jlwLg07HUDyxiNuXOvHEZS7pz15UY.jar
Jun 15, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /tmp/test9078386645714202862.zip to gs://temp-storage-for-perf-tests/loadtests/staging/test-79vgMd8nSf78xDNWmvnfcBAC51Zoy9qrZNNbLof8JsI.jar
Jun 15, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/java-fn-execution/build/libs/beam-runners-java-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-java-fn-execution-2.23.0-SNAPSHOT-U_QMvIevf5hpfTXSfowdrKjqBjw_NSag9uXbZzylvzk.jar
Jun 15, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT-KmcZcK1xymP7Ex0m0R-ncNCdT79EYTNrJICaR8zzDfM.jar
Jun 15, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tests-IOfTk66Y4tgw1zm298ZyKvomXsjK4CLEe_npzh2uIQo.jar
Jun 15, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-tests-u24sTBMIdGo48KhTHs9ow2UxKvVozt-Fgsd6zrgbh1I.jar
Jun 15, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-hLVySxkzWpl3su-SfC7DSRw2yBWm9t1y9uxyOU52E9M.jar
Jun 15, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-tests-1FoOG8aNo87Dcee-uXPXE_zCBqLg27DwVh4m7rrWK8A.jar
Jun 15, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT-1cIVJzGXQypnml5Gt0V65SKnl6J388HF4nljY-qpYzQ.jar
Jun 15, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-tests-weMHn8kiUO06Qlzj4jJPasguI3_do04Ct0Jq0W6d7NE.jar
Jun 15, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-4V3mdre_jgVpDrt2vwMBcNCFMuliyfRO0Nl_NEyS7gM.jar
Jun 15, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.23.0-SNAPSHOT-tests--gic6awk0z59gPpLkQdEldZyMe2HoNxCL8i6a7e3XTw.jar
Jun 15, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-LaASqiM-qgPOwBFnL-cb5trXImtqxJfj-UNZZL_o4Z0.jar
Jun 15, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.23.0-SNAPSHOT-xQ94hdxAje5VOe2L8H5lnpPS_-zd2E4VRLzXII3e_dU.jar
Jun 15, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-qiZZjwpl3QROTwiFWIvB9MNCiF3--iXTvLF4MaFbALk.jar
Jun 15, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-jqO0rQZqHMfNK0WuN05Q3LhtuWduEkPniMwJaIh8oqM.jar
Jun 15, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-K25BsUpilEGgFK_1GYy4rAR_ikShLy_JlQJvWyWOL9U.jar
Jun 15, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-tests-dc5wwoZuZ2cTbxvbhZgCWEz9HQFTcmNdOe8Bjx_wvxE.jar
Jun 15, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.23.0-SNAPSHOT-_tCM11xAGnTbZY3maGN8fakub7iE2AfKvqLskNZvWFs.jar
Jun 15, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.23.0-SNAPSHOT-xtlN6ZPWsp9Lq105AoNmo0ukAds4-ivphWZryOfS_rw.jar
Jun 15, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-unshaded-0hHFaABIlPevIUOtKZOwu1HJFF8C2SonvfZ-gvRZ7d0.jar
Jun 15, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-vkoZO5fFksaawKWb1JOW5ch_1NJwo6R2JIe3JeVBqt0.jar
Jun 15, 2020 12:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.23.0-SNAPSHOT-RI61XmgTgRyZmjiYgaW9xiH_eXxgd26ZvfixvGuiLX4.jar
Jun 15, 2020 12:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.23.0-SNAPSHOT-_nUOKXulM5Sy8rOoypdTwqwsXuPjUAkSRq_1q5i-JPo.jar
Jun 15, 2020 12:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.23.0-SNAPSHOT-6HOUJK7Ry1uiCrjVpxAAf_TStaMMnnc4iRxtYuFtnrY.jar
Jun 15, 2020 12:45:25 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-vkoZO5fFksaawKWb1JOW5ch_1NJwo6R2JIe3JeVBqt0.jar&uploadType=resumable&upload_id=AAANsUk92jM_HfcZveceiM1ExxFR7aqfZYSa2O1-U-5Sp_8GbLVEfIcgouLHuBoyz_raBLuR-xVtK0m5QriEhRXqYbI.
Jun 15, 2020 12:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar>
java.io.IOException: Upload failed for 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-vkoZO5fFksaawKWb1JOW5ch_1NJwo6R2JIe3JeVBqt0.jar'
at com.google.cloud.hadoop.util.BaseAbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(BaseAbstractGoogleAsyncWriteChannel.java:297)
at com.google.cloud.hadoop.util.BaseAbstractGoogleAsyncWriteChannel.close(BaseAbstractGoogleAsyncWriteChannel.java:214)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:221)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:168)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:152)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:136)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Jun 15, 2020 12:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-vkoZO5fFksaawKWb1JOW5ch_1NJwo6R2JIe3JeVBqt0.jar
Jun 15, 2020 12:45:31 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 31 files newly uploaded in 7 seconds
Jun 15, 2020 12:45:31 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Jun 15, 2020 12:45:31 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Jun 15, 2020 12:45:31 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Jun 15, 2020 12:45:31 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Jun 15, 2020 12:45:31 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Jun 15, 2020 12:45:31 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <91368 bytes, hash 8c6223e4f6cdac0ea97965a95e30c820abf94732088348def7edd0b81f051090> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-jGIj5PbNrA6peWWpXjDIIKv5RzIIg0je9-3QuB8FEJA.pb
Jun 15, 2020 12:45:32 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.23.0-SNAPSHOT
Jun 15, 2020 12:45:33 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-06-15_05_45_32-17997616388164811137?project=apache-beam-testing
Jun 15, 2020 12:45:33 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-06-15_05_45_32-17997616388164811137
Jun 15, 2020 12:45:33 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-06-15_05_45_32-17997616388164811137
Jun 15, 2020 12:45:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-06-15T12:45:32.201Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Jun 15, 2020 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-15T12:45:39.900Z: Worker configuration: n1-standard-1 in us-central1-a.
Jun 15, 2020 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-15T12:45:40.576Z: Expanding CoGroupByKey operations into optimizable parts.
Jun 15, 2020 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-15T12:45:40.617Z: Expanding GroupByKey operations into optimizable parts.
Jun 15, 2020 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-15T12:45:40.653Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jun 15, 2020 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-15T12:45:40.736Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jun 15, 2020 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-15T12:45:40.773Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Jun 15, 2020 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-15T12:45:40.807Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Jun 15, 2020 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-15T12:45:40.843Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Jun 15, 2020 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-15T12:45:41.224Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Jun 15, 2020 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-15T12:45:41.301Z: Starting 5 workers in us-central1-a...
Jun 15, 2020 12:46:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-15T12:46:08.079Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Jun 15, 2020 12:46:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-06-15T12:46:10.728Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jun 15, 2020 12:46:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-15T12:46:29.787Z: Workers have started successfully.
Jun 15, 2020 12:46:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-15T12:46:29.822Z: Workers have started successfully.
Jun 15, 2020 12:47:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-15T12:47:01.855Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Jun 15, 2020 12:47:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-15T12:47:02.066Z: Cleaning up.
Jun 15, 2020 12:47:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-15T12:47:02.178Z: Stopping worker pool...
Jun 15, 2020 12:48:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-15T12:48:56.912Z: Autoscaling: Resized worker pool from 5 to 0.
Jun 15, 2020 12:48:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-15T12:48:56.964Z: Worker pool stopped.
Jun 15, 2020 12:49:02 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-06-15_05_45_32-17997616388164811137 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 044f8278-de95-4bdb-8057-cf6ba307170e and timestamp: 2020-06-15T12:49:02.990000000Z:
Metric: Value:
fields_read 4375276.0
read_time 13.025
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Jun 15, 2020 12:49:03 PM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.022 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.03 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 8,5,main]) completed. Took 3 mins 50.492 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 43s
104 actionable tasks: 68 executed, 36 from cache
Publishing build scan...
https://gradle.com/s/vle5qk3jj2bom
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #629
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/629/display/redirect>
Changes:
------------------------------------------
[...truncated 293.57 KB...]
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:164)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Jun 15, 2020 6:45:20 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Jun 15, 2020 6:45:20 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery writeDisposition is set to: WRITE_EMPTY
Jun 15, 2020 6:45:20 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Jun 15, 2020 6:45:20 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Jun 15, 2020 6:45:20 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery writeDisposition is set to: WRITE_EMPTY
Jun 15, 2020 6:45:20 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Jun 15, 2020 6:45:20 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Jun 15, 2020 6:45:20 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Jun 15, 2020 6:45:21 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Jun 15, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 209 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Jun 15, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil$PackageAttributes forFileToStage
INFO: Staging custom dataflow-worker.jar as beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-PZq3sbs8YDGX4atX__I1LeemS5KMcSacrVYIcCq5Gl8.jar
Jun 15, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /tmp/test9048063500214795822.zip to gs://temp-storage-for-perf-tests/loadtests/staging/test-SDvy0saARYGU5s2sLGISa6SVtTs24XCScL_kZUyCapc.jar
Jun 15, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-tests-brypqw0Cw4cEEG_ubmY2dRviIfgo9pMNYysxRNvAHbM.jar
Jun 15, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/java-fn-execution/build/libs/beam-runners-java-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-java-fn-execution-2.23.0-SNAPSHOT-v9CHSEz4nKNi0oIT5W8VGQrs28KHxJZBYiCWwPHmBOY.jar
Jun 15, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tests-OjeSfPG2wgA3rE4eHmgj-788_nZwlKWkcCNHKlnwqxQ.jar
Jun 15, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-KLCYMxN0Q9CsWBUdTa-DqrXsJAcz63wX976l3s6jE6o.jar
Jun 15, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-PZq3sbs8YDGX4atX__I1LeemS5KMcSacrVYIcCq5Gl8.jar
Jun 15, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.23.0-SNAPSHOT-tE1Y3OVCvUSfvwnA1mIMI3t0eGAH3q8OB0Uuq-OY9gY.jar
Jun 15, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-bfn3tlC4BjdThWMyhRG5qZyl-982N8LJgwri4CTXxog.jar
Jun 15, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT-6Ai5bdF8AFSEA1-FZ-FU6QbBG2_zYASSHgEpRLN3z5E.jar
Jun 15, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-tests-dvZcKSW5zkV80_4KQhQv3HoxTYSIx3QKcgPoOiB86Uw.jar
Jun 15, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-tests-lpAUw4NeyCNIcvENW2zk7q7ZSJt3CdmwgkP_TMG7LyE.jar
Jun 15, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-IOn2XTwvL-pWLsofV3siGc6fwfDLSEVQab5Mc1zNkgE.jar
Jun 15, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.23.0-SNAPSHOT-boDnJEhSZS87H6QluXg4UJLaE58zcN_BCI-ZJa7ONic.jar
Jun 15, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.23.0-SNAPSHOT-FPZdAPHzUGbdftD6MrCBhtLZiosXd32yZRBHyxk3Jug.jar
Jun 15, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.23.0-SNAPSHOT-Z0KPscmSewJxjTa5fkJwRN00h0xKHNlJFsnIQaFrYyY.jar
Jun 15, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-gsxuqvR1YhO2kuq01vj0Izmv-lAz0_XoeJL8msEDCL8.jar
Jun 15, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-fn-execution-2.23.0-SNAPSHOT-OW7qBaii3RSFzjmB6_68vV0pDe06g3ZnpNubPKl26_w.jar
Jun 15, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-tests-ncYqKy-2ejJb1TQUmetEtFdXjBq4BskJKT4qROzQK_Y.jar
Jun 15, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-tests-p9Mcng60RxCo6VvOI-9O3GHNFYRo8e2BCjQnGzyFw6s.jar
Jun 15, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-tests-c3DQCRSWCuh84xxhBMHTdaJRxAihy-WsMqf4wdffAVw.jar
Jun 15, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT-tp6WGz8tVJQ9-Ita51BSnAJ-RfgxF7XC3qVQLIDgJ9I.jar
Jun 15, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-vf-Qq6Hq4AeRW93t1t3_Z_9h37e2ibVNbrMFOy8tssU.jar
Jun 15, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-e-HbR1pf_29UqQNRgBMvlhqS_fGn5KPAcKZKlWzPsSw.jar
Jun 15, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.23.0-SNAPSHOT-tests-7ze_zuOQTYkId81O1w5QwNwq5WnaDssO0zpzSrT8ZTE.jar
Jun 15, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-unshaded-m4oTe-ga2QDC_JW1c4Tk8NkG_EyYk9QZk6gSuhZ8wAI.jar
Jun 15, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-PZq3sbs8YDGX4atX__I1LeemS5KMcSacrVYIcCq5Gl8.jar
Jun 15, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.23.0-SNAPSHOT-3v4HefZl_WWcAKnKtdEeHS-Qub0PlMlqrEqXrZTbeGA.jar
Jun 15, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-KdbcajywEB1oKkYZlm4Hivf-Bs9tW3k_7-Wf23zl3Pw.jar
Jun 15, 2020 6:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.23.0-SNAPSHOT-NDozGEpHJoiGeB7LtE0s_ENoft5iz2ntgLVygzzongQ.jar
Jun 15, 2020 6:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.23.0-SNAPSHOT-vGHDdSD9VN-36bH7JtoPm6NnvFz8lThOskPUorXBTSM.jar
Jun 15, 2020 6:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.23.0-SNAPSHOT-SozDxyiRqWS6XqVRDcKQTCP2tAHJr3IZ-JAagw7lISw.jar
Jun 15, 2020 6:45:24 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-PZq3sbs8YDGX4atX__I1LeemS5KMcSacrVYIcCq5Gl8.jar&uploadType=resumable&upload_id=AAANsUk-Nzrkqf25nfdRyPzuE_YO9O1ggQ4ykZ1PQlNGxHCUBNDIiDwTDUpNaJo_p_V--4EgOQqTK7ipyR3Oz0ahlA8.
Jun 15, 2020 6:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar>
java.io.IOException: Upload failed for 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-PZq3sbs8YDGX4atX__I1LeemS5KMcSacrVYIcCq5Gl8.jar'
at com.google.cloud.hadoop.util.BaseAbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(BaseAbstractGoogleAsyncWriteChannel.java:297)
at com.google.cloud.hadoop.util.BaseAbstractGoogleAsyncWriteChannel.close(BaseAbstractGoogleAsyncWriteChannel.java:214)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:221)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:168)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:152)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:136)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Jun 15, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-PZq3sbs8YDGX4atX__I1LeemS5KMcSacrVYIcCq5Gl8.jar
Jun 15, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 31 files newly uploaded in 8 seconds
Jun 15, 2020 6:45:32 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Jun 15, 2020 6:45:32 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Jun 15, 2020 6:45:32 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Jun 15, 2020 6:45:32 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Jun 15, 2020 6:45:32 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Jun 15, 2020 6:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <91368 bytes, hash 8730679cc63f244fce00c864cc8fbbd60a403acf5a21fbc525ec42e153d19c21> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-hzBnnMY_JE_OAMhkzI-71gpAOs9aIfvFJexC4VPRnCE.pb
Jun 15, 2020 6:45:32 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.23.0-SNAPSHOT
Jun 15, 2020 6:45:33 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-06-14_23_45_32-5973835179983748696?project=apache-beam-testing
Jun 15, 2020 6:45:33 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-06-14_23_45_32-5973835179983748696
Jun 15, 2020 6:45:33 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-06-14_23_45_32-5973835179983748696
Jun 15, 2020 6:45:34 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-06-15T06:45:32.943Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Jun 15, 2020 6:45:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-15T06:45:40.086Z: Worker configuration: n1-standard-1 in us-central1-a.
Jun 15, 2020 6:45:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-15T06:45:40.836Z: Expanding CoGroupByKey operations into optimizable parts.
Jun 15, 2020 6:45:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-15T06:45:40.877Z: Expanding GroupByKey operations into optimizable parts.
Jun 15, 2020 6:45:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-15T06:45:40.916Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jun 15, 2020 6:45:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-15T06:45:41.004Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jun 15, 2020 6:45:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-15T06:45:41.033Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Jun 15, 2020 6:45:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-15T06:45:41.073Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Jun 15, 2020 6:45:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-15T06:45:41.108Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Jun 15, 2020 6:45:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-15T06:45:41.430Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Jun 15, 2020 6:45:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-15T06:45:41.521Z: Starting 5 workers in us-central1-a...
Jun 15, 2020 6:45:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-06-15T06:45:49.172Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jun 15, 2020 6:46:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-15T06:46:10.958Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Jun 15, 2020 6:46:29 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-15T06:46:27.568Z: Workers have started successfully.
Jun 15, 2020 6:46:29 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-15T06:46:27.605Z: Workers have started successfully.
Jun 15, 2020 6:47:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-15T06:46:58.322Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Jun 15, 2020 6:47:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-15T06:46:58.491Z: Cleaning up.
Jun 15, 2020 6:47:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-15T06:46:58.569Z: Stopping worker pool...
Jun 15, 2020 6:49:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-15T06:49:18.290Z: Autoscaling: Resized worker pool from 5 to 0.
Jun 15, 2020 6:49:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-15T06:49:18.339Z: Worker pool stopped.
Jun 15, 2020 6:49:23 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-06-14_23_45_32-5973835179983748696 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 9644ace6-803f-4a1a-b421-6154be083cf4 and timestamp: 2020-06-15T06:49:23.350000000Z:
Metric: Value:
fields_read 4375276.0
read_time 13.366
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Jun 15, 2020 6:49:23 AM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.028 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.034 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 11,5,main]) completed. Took 4 mins 11.723 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 2s
104 actionable tasks: 68 executed, 36 from cache
Publishing build scan...
https://gradle.com/s/hc57mty63hhio
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #628
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/628/display/redirect>
Changes:
------------------------------------------
[...truncated 294.11 KB...]
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Jun 15, 2020 12:45:18 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Jun 15, 2020 12:45:18 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery writeDisposition is set to: WRITE_EMPTY
Jun 15, 2020 12:45:18 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Jun 15, 2020 12:45:18 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Jun 15, 2020 12:45:18 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery writeDisposition is set to: WRITE_EMPTY
Jun 15, 2020 12:45:18 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Jun 15, 2020 12:45:18 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Jun 15, 2020 12:45:18 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Jun 15, 2020 12:45:19 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Jun 15, 2020 12:45:20 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 209 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Jun 15, 2020 12:45:21 AM org.apache.beam.runners.dataflow.util.PackageUtil$PackageAttributes forFileToStage
INFO: Staging custom dataflow-worker.jar as beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-G1nPjRlQjFexS8Uq-vz8GkvxP8Y2XSYq6s7tNaVH55c.jar
Jun 15, 2020 12:45:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.23.0-SNAPSHOT-21ugql9otS6BMvfUoIDlQjSTOltrBeCl4rwz4bwTPB4.jar
Jun 15, 2020 12:45:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-G1nPjRlQjFexS8Uq-vz8GkvxP8Y2XSYq6s7tNaVH55c.jar
Jun 15, 2020 12:45:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-wVEbBqVwXhkhaQAamP05_BmPFRIZ2l6UorjPw8AE5UU.jar
Jun 15, 2020 12:45:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT-HwFy-jggOLV-l7rgll3W8JYaa_VgeSNAHohqKlpI648.jar
Jun 15, 2020 12:45:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-HvbZ3t7YNmUcoPmwGMXbomDSlU2iDVmVictzAdq0mkA.jar
Jun 15, 2020 12:45:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.23.0-SNAPSHOT-OP8ieu8NCBtWWtL5OjADIlPhqHd_GQec7zD53WlFHnQ.jar
Jun 15, 2020 12:45:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-tests-2D_MCmdfOEZix1YCd6S2nNPDOmfmvUqo4EjwhODF9L4.jar
Jun 15, 2020 12:45:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-tests-FsTVto2G2d5_D-r-EfUhejey_Q_bHQA_eyDE1M_CWt4.jar
Jun 15, 2020 12:45:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.23.0-SNAPSHOT-3S6RxMDRiJFv9Rt6d9du6INLcAtRjv73amvB1vmV8b4.jar
Jun 15, 2020 12:45:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-tests-o5urjCCzPmnLSpDyADTPE1LwOkgQelH8EZQAgIi4wQQ.jar
Jun 15, 2020 12:45:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-fn-execution-2.23.0-SNAPSHOT-4LAzsumW0UzQ7V98EGoZvYnVLpdZ150xdIqgzmukdiI.jar
Jun 15, 2020 12:45:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-tests-SP9qzlVAWCl2KS2lNLeYsKb5VvQcvKJNZEY8xD6MGHI.jar
Jun 15, 2020 12:45:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-pKhuLF4--O2u32k1Ox0LEWsEWhgmc7WzYdrJrUBkQVw.jar
Jun 15, 2020 12:45:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-03ko2or421oqO05LN7mxLxkxg-E_FHU76hNUkzYTGNY.jar
Jun 15, 2020 12:45:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.23.0-SNAPSHOT-tests-M4QQZxl7b15XqpPR-RlyLy2Ko-hChEx9f_du15rlnMc.jar
Jun 15, 2020 12:45:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT-OjsiCCztNI-CR3PVEtJhMly2pQn4xfqAsYzh6q5Zxcc.jar
Jun 15, 2020 12:45:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-pD5RajvVA2sZ6E28oztsiuLMmhaabxtAHYo4JpdGd2k.jar
Jun 15, 2020 12:45:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-unshaded-Ds9X7YlARBY_8kLMfR5xWqK69Uh4S_MCH58ReQ7ginI.jar
Jun 15, 2020 12:45:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-tests-w-HCzOmSt6j694p6fjQE4Uqx0z90r8N1IfvbIvf0oW4.jar
Jun 15, 2020 12:45:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.23.0-SNAPSHOT-v0y3QXbYxi3cZb_bGeYmmZ2ppLmbn2X5Kfr9UTMiXYI.jar
Jun 15, 2020 12:45:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-2xpkkLXn9-lM_te4zPAZHlUO_15IpN3-FtnSgSDXeWI.jar
Jun 15, 2020 12:45:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-GSv-Ud12B0bHcHm0pVyr7aCyrYMpac_uueAm-A4BrC0.jar
Jun 15, 2020 12:45:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /tmp/test450234958329159603.zip to gs://temp-storage-for-perf-tests/loadtests/staging/test-xBxvLYWPRjPUm17_HJinP1aNnemNSqMIQk63Bri0PYM.jar
Jun 15, 2020 12:45:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tests-o4LAuwjU3-myHG7qEM63rxmj_WZGPHNZrL8s1WsXQY0.jar
Jun 15, 2020 12:45:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-tests-e5bB9sJREbaB-3vJd6nOjkaNx_fb2i6_9Hr-0l05ad0.jar
Jun 15, 2020 12:45:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.23.0-SNAPSHOT-BRU8Pk4EFyY_HKiBHuhyREYgA70TVIeksUxkrRo8iYU.jar
Jun 15, 2020 12:45:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/java-fn-execution/build/libs/beam-runners-java-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-java-fn-execution-2.23.0-SNAPSHOT-X2GwzSxcfzvjoTiNJxBxXsJORl-LqrqRd9ls-xAwHaw.jar
Jun 15, 2020 12:45:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-G1nPjRlQjFexS8Uq-vz8GkvxP8Y2XSYq6s7tNaVH55c.jar
Jun 15, 2020 12:45:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.23.0-SNAPSHOT-Ms12lnIcrGPh6FDwZ3g553QlFMyCUr5UkOQB11FPkOA.jar
Jun 15, 2020 12:45:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.23.0-SNAPSHOT-x0TQq3THt02wMxnU-MAfL3UQat4czyWsE0Z3rRUlKjs.jar
Jun 15, 2020 12:45:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.23.0-SNAPSHOT-3-AkFKMnBa6Th9R_1PFEOom2-WJEBrG3AoYZeC_-FAY.jar
Jun 15, 2020 12:45:22 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-G1nPjRlQjFexS8Uq-vz8GkvxP8Y2XSYq6s7tNaVH55c.jar&uploadType=resumable&upload_id=AAANsUkc-FLRn2XDqbWkDoZAAGx21heX2DKhw10g51wahxezOypL63lj8rnt2Yg6a5ebagrS3HW3Re8O27BiYZQ8IwaCMKUyCg.
Jun 15, 2020 12:45:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar>
java.io.IOException: Upload failed for 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-G1nPjRlQjFexS8Uq-vz8GkvxP8Y2XSYq6s7tNaVH55c.jar'
at com.google.cloud.hadoop.util.BaseAbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(BaseAbstractGoogleAsyncWriteChannel.java:297)
at com.google.cloud.hadoop.util.BaseAbstractGoogleAsyncWriteChannel.close(BaseAbstractGoogleAsyncWriteChannel.java:214)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:221)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:168)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:152)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:136)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Jun 15, 2020 12:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-G1nPjRlQjFexS8Uq-vz8GkvxP8Y2XSYq6s7tNaVH55c.jar
Jun 15, 2020 12:45:26 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 31 files newly uploaded in 5 seconds
Jun 15, 2020 12:45:26 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Jun 15, 2020 12:45:26 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Jun 15, 2020 12:45:26 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Jun 15, 2020 12:45:26 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Jun 15, 2020 12:45:26 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Jun 15, 2020 12:45:26 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <91367 bytes, hash dd6f09da6e9be05823a84025c5be73f6f6e4db0b55b18ed5740ce2de202519cc> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-3W8J2m6b4FgjqEAlxb5z9vbk2wtVsY7VdAzi3iAlGcw.pb
Jun 15, 2020 12:45:26 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.23.0-SNAPSHOT
Jun 15, 2020 12:45:27 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-06-14_17_45_26-1513686741457794967?project=apache-beam-testing
Jun 15, 2020 12:45:27 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-06-14_17_45_26-1513686741457794967
Jun 15, 2020 12:45:27 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-06-14_17_45_26-1513686741457794967
Jun 15, 2020 12:45:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-06-15T00:45:26.691Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Jun 15, 2020 12:45:34 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-15T00:45:33.997Z: Worker configuration: n1-standard-1 in us-central1-a.
Jun 15, 2020 12:45:34 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-15T00:45:34.632Z: Expanding CoGroupByKey operations into optimizable parts.
Jun 15, 2020 12:45:37 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-15T00:45:34.671Z: Expanding GroupByKey operations into optimizable parts.
Jun 15, 2020 12:45:37 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-15T00:45:34.708Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jun 15, 2020 12:45:37 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-15T00:45:34.793Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jun 15, 2020 12:45:37 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-15T00:45:34.825Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Jun 15, 2020 12:45:37 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-15T00:45:34.856Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Jun 15, 2020 12:45:37 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-15T00:45:34.891Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Jun 15, 2020 12:45:37 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-15T00:45:35.348Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Jun 15, 2020 12:45:37 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-15T00:45:35.423Z: Starting 5 workers in us-central1-a...
Jun 15, 2020 12:46:02 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-15T00:46:02.256Z: Autoscaling: Raised the number of workers to 2 based on the rate of progress in the currently running stage(s).
Jun 15, 2020 12:46:02 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-15T00:46:02.283Z: Resized worker pool to 2, though goal was 5. This could be a quota issue.
Jun 15, 2020 12:46:10 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-06-15T00:46:07.584Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jun 15, 2020 12:46:10 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-15T00:46:07.721Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Jun 15, 2020 12:46:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-15T00:46:20.648Z: Workers have started successfully.
Jun 15, 2020 12:46:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-15T00:46:20.682Z: Workers have started successfully.
Jun 15, 2020 12:46:57 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-15T00:46:56.942Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Jun 15, 2020 12:46:57 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-15T00:46:57.162Z: Cleaning up.
Jun 15, 2020 12:46:57 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-15T00:46:57.241Z: Stopping worker pool...
Jun 15, 2020 12:48:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-15T00:48:30.498Z: Autoscaling: Resized worker pool from 5 to 0.
Jun 15, 2020 12:48:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-15T00:48:30.544Z: Worker pool stopped.
Jun 15, 2020 12:48:37 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-06-14_17_45_26-1513686741457794967 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 1b8ed581-2ac5-4668-962b-dbe140f1dcdf and timestamp: 2020-06-15T00:48:37.507000000Z:
Metric: Value:
fields_read 4375276.0
read_time 19.049
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Jun 15, 2020 12:48:37 AM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.023 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.032 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 11,5,main]) completed. Took 3 mins 28.463 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 19s
104 actionable tasks: 68 executed, 36 from cache
Publishing build scan...
https://gradle.com/s/xh37vhcobvawk
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #627
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/627/display/redirect>
Changes:
------------------------------------------
[...truncated 294.60 KB...]
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Jun 14, 2020 6:45:22 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Jun 14, 2020 6:45:22 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery writeDisposition is set to: WRITE_EMPTY
Jun 14, 2020 6:45:22 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Jun 14, 2020 6:45:22 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Jun 14, 2020 6:45:22 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery writeDisposition is set to: WRITE_EMPTY
Jun 14, 2020 6:45:22 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Jun 14, 2020 6:45:22 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Jun 14, 2020 6:45:22 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Jun 14, 2020 6:45:23 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Jun 14, 2020 6:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 209 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Jun 14, 2020 6:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil$PackageAttributes forFileToStage
INFO: Staging custom dataflow-worker.jar as beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-L6pHj1nB24y4Tr8D_h2d76vlHlskLc30xVkwaXt3b2w.jar
Jun 14, 2020 6:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.23.0-SNAPSHOT-ml04X4H-zkWrZhLlcfEDT1kaF7sRIj9lmDvEkCgdh0o.jar
Jun 14, 2020 6:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-tests-4oMpg-EDS216OwbO5BnJhV8l9vC2x_uH1oaWtBhuc6s.jar
Jun 14, 2020 6:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-H3yqSl2xeii6hTmY0hHxF42PzEjTotIt9EltATMAGeE.jar
Jun 14, 2020 6:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT-kGkvOr7RLsJMJmh0_5i-oR100ASONi4diedbck3XW-c.jar
Jun 14, 2020 6:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT-r9ggtWQckJ0FMXjJuJofFdO8XIoMAMsndoy2Rv7B4Zg.jar
Jun 14, 2020 6:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-tests-0qnj25zL4arij0Zjq2UrGYu0FZLdP2-0Xt_y-31NbtI.jar
Jun 14, 2020 6:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.23.0-SNAPSHOT-9tTCHb0sF-nqrUJ-9aLM5bAZpamiZcG7p9HKHtqZJIg.jar
Jun 14, 2020 6:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.23.0-SNAPSHOT-tests-u05BJUU1e7UY1qN2me5qN9vdjUkTsCuhSYeEGbyqA60.jar
Jun 14, 2020 6:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-AiqeJcxlpGtvcmQArmD3IUAg1A38xakjzi7fqsqqxig.jar
Jun 14, 2020 6:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /tmp/test5008553736369711753.zip to gs://temp-storage-for-perf-tests/loadtests/staging/test-SIZKkaxQ1GTeMM8PEMRLwbLRm76NhRBU9BqPZrIoOSo.jar
Jun 14, 2020 6:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-tests-VjtnpWoW3zKhxtynZUZ5EQqflOQbU0R5O81Z4FS--mU.jar
Jun 14, 2020 6:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-tests-FSukNdcF8crUVz_qtc4cUo8_KQj8152Us9Rv5ALkW9w.jar
Jun 14, 2020 6:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-tests-Wo92T-Sw9eKWZUrRABonOuSMMT4FMiRP7aVce5wbbc0.jar
Jun 14, 2020 6:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/java-fn-execution/build/libs/beam-runners-java-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-java-fn-execution-2.23.0-SNAPSHOT-6ZljiEnn2cPGBg5gvxC9Yp-P1jIVivpkGlCSVlFL2Gc.jar
Jun 14, 2020 6:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-mTL8hnwtm3LfaN7y1esYGfs86cJfJ8s0BM6f0whQhNI.jar
Jun 14, 2020 6:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-FI7gaoW51dw_slbDs9vbHJuSz1T9xGu4hcbT7fMj_Fs.jar
Jun 14, 2020 6:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-fn-execution-2.23.0-SNAPSHOT-bsocZgSzBilTo_xsY1Em1SP2YpshbqLvxBCtgLerK4M.jar
Jun 14, 2020 6:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.23.0-SNAPSHOT-fLdSh_dsVRhyeQ2dUeQGkmV6zk2CSRGo9CaBkfkltos.jar
Jun 14, 2020 6:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-fFRXaghKM2qc8Mvjr9LzYdFbko6Vkt6VGG2BtqSiLyQ.jar
Jun 14, 2020 6:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.23.0-SNAPSHOT-g01y-lGVag1jZbJpijsF9ZGxanU7WJSjrxAaNtpoHyU.jar
Jun 14, 2020 6:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-tests-ayfR4vTGsi1zfr1MbTbIROeHFh6RNXQy2N4O-ZHGoy4.jar
Jun 14, 2020 6:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tests-p4cPquUlTE34sA3rlTL0FHVgA1VaLBj5-3gNqjzHwYU.jar
Jun 14, 2020 6:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-9jj-9pM2jJA3Zc4WxN1y76U3wlwLAf7VpJaXA_2uwhs.jar
Jun 14, 2020 6:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-L6pHj1nB24y4Tr8D_h2d76vlHlskLc30xVkwaXt3b2w.jar
Jun 14, 2020 6:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.23.0-SNAPSHOT-RkFmLGX8B4UkZQ_GwbZYbeHIdq906YdKsBma1Hbo69E.jar
Jun 14, 2020 6:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-unshaded-VTC1V31uaJPatsGtI5gqiUoGX_t677YdiVtjHBetnKQ.jar
Jun 14, 2020 6:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-Rtrp_5pudGo-siMid3eSct_9DFA8irLx8NlE-CI8xSs.jar
Jun 14, 2020 6:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-L6pHj1nB24y4Tr8D_h2d76vlHlskLc30xVkwaXt3b2w.jar
Jun 14, 2020 6:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.23.0-SNAPSHOT-nrYmojcHuG88evJ-SLBRqJl8Bf0hsi82z31s2odkFlo.jar
Jun 14, 2020 6:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.23.0-SNAPSHOT-QDW-MPcimxguWkq6Q4bStyXOdwCtrQv46WGr2kNFSz4.jar
Jun 14, 2020 6:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.23.0-SNAPSHOT-0oquF83QImii7AbjRHKJR-OdgftFyQJh18E3vG6p4NE.jar
Jun 14, 2020 6:45:26 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-L6pHj1nB24y4Tr8D_h2d76vlHlskLc30xVkwaXt3b2w.jar&uploadType=resumable&upload_id=AAANsUmXEbIQknVM385AvaAyu0BlAemzA_c614gNaAPu4eEtuxyVNlSHjp1ULJBc5hD_2q_qMTVdmLt8W5EDp1ym9kQ.
Jun 14, 2020 6:45:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar>
java.io.IOException: Upload failed for 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-L6pHj1nB24y4Tr8D_h2d76vlHlskLc30xVkwaXt3b2w.jar'
at com.google.cloud.hadoop.util.BaseAbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(BaseAbstractGoogleAsyncWriteChannel.java:297)
at com.google.cloud.hadoop.util.BaseAbstractGoogleAsyncWriteChannel.close(BaseAbstractGoogleAsyncWriteChannel.java:214)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:221)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:168)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:152)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:136)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Jun 14, 2020 6:45:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-L6pHj1nB24y4Tr8D_h2d76vlHlskLc30xVkwaXt3b2w.jar
Jun 14, 2020 6:45:29 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 31 files newly uploaded in 4 seconds
Jun 14, 2020 6:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Jun 14, 2020 6:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Jun 14, 2020 6:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Jun 14, 2020 6:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Jun 14, 2020 6:45:30 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Jun 14, 2020 6:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <91368 bytes, hash bfdb2aed83488ce1338a8bff49f2b9fb3707a8cddad84affd5f8ddcd754040f8> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-v9sq7YNIjOEziov_SfK5-zcHqM3a2Er_1fjdzXVAQPg.pb
Jun 14, 2020 6:45:30 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.23.0-SNAPSHOT
Jun 14, 2020 6:45:31 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-06-14_11_45_30-10980767790616573864?project=apache-beam-testing
Jun 14, 2020 6:45:31 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-06-14_11_45_30-10980767790616573864
Jun 14, 2020 6:45:31 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-06-14_11_45_30-10980767790616573864
Jun 14, 2020 6:45:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-06-14T18:45:30.585Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Jun 14, 2020 6:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-14T18:45:37.208Z: Worker configuration: n1-standard-1 in us-central1-a.
Jun 14, 2020 6:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-14T18:45:37.872Z: Expanding CoGroupByKey operations into optimizable parts.
Jun 14, 2020 6:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-14T18:45:37.919Z: Expanding GroupByKey operations into optimizable parts.
Jun 14, 2020 6:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-14T18:45:37.981Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jun 14, 2020 6:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-14T18:45:38.070Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jun 14, 2020 6:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-14T18:45:38.112Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Jun 14, 2020 6:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-14T18:45:38.145Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Jun 14, 2020 6:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-14T18:45:38.181Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Jun 14, 2020 6:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-14T18:45:38.546Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Jun 14, 2020 6:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-14T18:45:38.627Z: Starting 5 workers in us-central1-a...
Jun 14, 2020 6:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-06-14T18:45:53.117Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jun 14, 2020 6:46:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-14T18:46:05.080Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running stage(s).
Jun 14, 2020 6:46:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-14T18:46:05.116Z: Resized worker pool to 4, though goal was 5. This could be a quota issue.
Jun 14, 2020 6:46:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-14T18:46:10.452Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Jun 14, 2020 6:46:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-14T18:46:22.589Z: Workers have started successfully.
Jun 14, 2020 6:46:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-14T18:46:22.623Z: Workers have started successfully.
Jun 14, 2020 6:46:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-14T18:46:51.722Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Jun 14, 2020 6:46:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-14T18:46:51.911Z: Cleaning up.
Jun 14, 2020 6:46:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-14T18:46:51.989Z: Stopping worker pool...
Jun 14, 2020 6:48:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-14T18:48:33.731Z: Autoscaling: Resized worker pool from 5 to 0.
Jun 14, 2020 6:48:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-14T18:48:33.776Z: Worker pool stopped.
Jun 14, 2020 6:48:39 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-06-14_11_45_30-10980767790616573864 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): a34d7a75-9dfb-471b-8ba0-a8fabaefefbb and timestamp: 2020-06-14T18:48:40.039000000Z:
Metric: Value:
fields_read 4375276.0
read_time 13.744
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Jun 14, 2020 6:48:40 PM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.024 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.031 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 5,5,main]) completed. Took 3 mins 26.249 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 16s
104 actionable tasks: 68 executed, 36 from cache
Publishing build scan...
https://gradle.com/s/ppjweqnmmaeks
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #626
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/626/display/redirect>
Changes:
------------------------------------------
[...truncated 293.37 KB...]
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Jun 14, 2020 12:45:19 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Jun 14, 2020 12:45:19 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery writeDisposition is set to: WRITE_EMPTY
Jun 14, 2020 12:45:19 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Jun 14, 2020 12:45:19 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Jun 14, 2020 12:45:19 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery writeDisposition is set to: WRITE_EMPTY
Jun 14, 2020 12:45:19 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Jun 14, 2020 12:45:20 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Jun 14, 2020 12:45:20 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Jun 14, 2020 12:45:20 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Jun 14, 2020 12:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 209 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Jun 14, 2020 12:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil$PackageAttributes forFileToStage
INFO: Staging custom dataflow-worker.jar as beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-GEg8r38lCpnLtuW_oCbGCgZn_y_gMSrRMdkIy_OofOA.jar
Jun 14, 2020 12:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.23.0-SNAPSHOT-xWFnC_hvGAccSej8xvLQiZ-0hsrq2Dvpl5CeqNerAS0.jar
Jun 14, 2020 12:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/java-fn-execution/build/libs/beam-runners-java-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-java-fn-execution-2.23.0-SNAPSHOT-JNoFhHCpxOwn7qZVIi-yj_G9ruhHaQAx8-qHPegTOGc.jar
Jun 14, 2020 12:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-2kZqu_uXwyQSGTfjXBlt2TwI5GMDkKjTXtmrBX0SofA.jar
Jun 14, 2020 12:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT-tVcO5U_vRBNG6BfR1kAocbgtcgDaIRsTPsNembtORw8.jar
Jun 14, 2020 12:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-fn-execution-2.23.0-SNAPSHOT-vAD5DU7fS29O53JJTa4J5DZLz7dIJOBsVivu0Qb6fQk.jar
Jun 14, 2020 12:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-tests-Yjz4aVJu1ysXnigsUw4_MAge5yaevZOXKQUuvQVZI0A.jar
Jun 14, 2020 12:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.23.0-SNAPSHOT-eXlbQmJ9H2pjjiTWyYCxU3Za9CCm51BdmKEC_HeZ_hg.jar
Jun 14, 2020 12:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tests-1kcO-fq7m5YYkxvAeBxu0B15Aj7t_eVAMl9rsoiiEZ8.jar
Jun 14, 2020 12:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-tests-w-7yGYNQ9_9yZkT7K7M_ko-6iNTMmMXtfjpdDWQwPCU.jar
Jun 14, 2020 12:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-GEg8r38lCpnLtuW_oCbGCgZn_y_gMSrRMdkIy_OofOA.jar
Jun 14, 2020 12:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.23.0-SNAPSHOT-tests-Pi-WIYZ758BxgGSY6RRQUlSsZK_lwKAfyJ8B9OKZxEY.jar
Jun 14, 2020 12:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.23.0-SNAPSHOT-rbnbPrleF5opXnsrb4RY78dHUya3vsTJ8RITPtFrBmo.jar
Jun 14, 2020 12:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.23.0-SNAPSHOT-WnAqYjWX3gcqO4C25DKeGN43ttfO6SEg2TDKm_hA5cQ.jar
Jun 14, 2020 12:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-tests-QpAiwpmS7b_U4bPPcPrtp9Gx1Z-AQw-w9ZRY6vkddeQ.jar
Jun 14, 2020 12:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-p8tB967qddQCmhCh9bqBhLPnQjhIw3VTeMqbMNEvKhY.jar
Jun 14, 2020 12:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-tests-GDNgo6pltbljB2ID0eVqBIvrOZKBoZThIZrYVwWs9jU.jar
Jun 14, 2020 12:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.23.0-SNAPSHOT-MhnTkUFw0izdWX71lCZ91mNDw8PX_5iHehWxQbzpGTw.jar
Jun 14, 2020 12:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-LOxtC3yZRK14gZK_dEjdFQLfz5Hju3BpvgLSHJu9alc.jar
Jun 14, 2020 12:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-e-ke3-Gze2lmNJBh8-siArfpL6I8Be9YR-Ysk2pXfuQ.jar
Jun 14, 2020 12:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-YXGcraw4AcvihEgIB1Nq76MDjDe8Xaqu-kJD6Bw7Zd8.jar
Jun 14, 2020 12:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-I0EnpYTHtl7sv2U4KhHoSt09OipWDEfVF4B1adRvU6A.jar
Jun 14, 2020 12:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-tests-x7tjQ1CzWpFO8scQj_6kmTxQAFygDuNiLFeaJAGIyuY.jar
Jun 14, 2020 12:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-yAjcKRW24K-clcx7l-5p0aheMHgGaQ2-LN89vfOCqFM.jar
Jun 14, 2020 12:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT-OxbB3Fw_ih51mSs9C3SUkViJUbT0C1uwlcEEKvnzKIU.jar
Jun 14, 2020 12:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-unshaded-dbpZZzSVZDiktzrwwYXb8K5VLiREAjaLzpv8NNDMU8E.jar
Jun 14, 2020 12:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /tmp/test1242228312193204146.zip to gs://temp-storage-for-perf-tests/loadtests/staging/test-ngzOJ7s6vzYxjCUQxzt2wwJY2cryA5YL6mqCc_1vxCw.jar
Jun 14, 2020 12:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-tests-KwBGle449m0F0F1debL5qrU2Bf008b97G7wSvD1SJf4.jar
Jun 14, 2020 12:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-GEg8r38lCpnLtuW_oCbGCgZn_y_gMSrRMdkIy_OofOA.jar
Jun 14, 2020 12:45:23 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.23.0-SNAPSHOT-sVTm6igbp1882Vluqid0KjHNgT9TbHJcrlQc1VTEJBo.jar
Jun 14, 2020 12:45:23 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.23.0-SNAPSHOT-8UlwLW4HO40_6_fyH7xsxKp0yvXNLlwqxyO8etAo1aQ.jar
Jun 14, 2020 12:45:23 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.23.0-SNAPSHOT-lUZ_dwGnXVSDLD7V9T0JM9tDgRdnk37lpuu14edMkNA.jar
Jun 14, 2020 12:45:23 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-GEg8r38lCpnLtuW_oCbGCgZn_y_gMSrRMdkIy_OofOA.jar&uploadType=resumable&upload_id=AAANsUki_NwjPIeWFzMk-BegPPogUR7J1rL89OXuJ9axsllfFRYecGnO48uSb-sCgPQvYK2VOHFXvTgDpqd4nctrGes.
Jun 14, 2020 12:45:23 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar>
java.io.IOException: Upload failed for 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-GEg8r38lCpnLtuW_oCbGCgZn_y_gMSrRMdkIy_OofOA.jar'
at com.google.cloud.hadoop.util.BaseAbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(BaseAbstractGoogleAsyncWriteChannel.java:297)
at com.google.cloud.hadoop.util.BaseAbstractGoogleAsyncWriteChannel.close(BaseAbstractGoogleAsyncWriteChannel.java:214)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:221)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:168)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:152)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:136)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Jun 14, 2020 12:45:31 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-GEg8r38lCpnLtuW_oCbGCgZn_y_gMSrRMdkIy_OofOA.jar
Jun 14, 2020 12:45:32 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 31 files newly uploaded in 9 seconds
Jun 14, 2020 12:45:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Jun 14, 2020 12:45:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Jun 14, 2020 12:45:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Jun 14, 2020 12:45:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Jun 14, 2020 12:45:32 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Jun 14, 2020 12:45:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <91368 bytes, hash 7530f588d7ce2e04dbd8a93042dc08b383cd276452aed53e6570dc95d082fdc6> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-dTD1iNfOLgTb2KkwQtwIs4PNJ2RSrtU-ZXDcldCC_cY.pb
Jun 14, 2020 12:45:32 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.23.0-SNAPSHOT
Jun 14, 2020 12:45:33 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-06-14_05_45_32-6291091626775836526?project=apache-beam-testing
Jun 14, 2020 12:45:33 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-06-14_05_45_32-6291091626775836526
Jun 14, 2020 12:45:33 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-06-14_05_45_32-6291091626775836526
Jun 14, 2020 12:45:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-06-14T12:45:32.837Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Jun 14, 2020 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-14T12:45:39.672Z: Worker configuration: n1-standard-1 in us-central1-a.
Jun 14, 2020 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-14T12:45:40.467Z: Expanding CoGroupByKey operations into optimizable parts.
Jun 14, 2020 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-14T12:45:40.503Z: Expanding GroupByKey operations into optimizable parts.
Jun 14, 2020 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-14T12:45:40.538Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jun 14, 2020 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-14T12:45:40.630Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jun 14, 2020 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-14T12:45:40.669Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Jun 14, 2020 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-14T12:45:40.704Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Jun 14, 2020 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-14T12:45:40.738Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Jun 14, 2020 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-14T12:45:41.291Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Jun 14, 2020 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-14T12:45:41.381Z: Starting 5 workers in us-central1-a...
Jun 14, 2020 12:46:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-06-14T12:45:58.239Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jun 14, 2020 12:46:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-14T12:46:06.932Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running stage(s).
Jun 14, 2020 12:46:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-14T12:46:06.970Z: Resized worker pool to 4, though goal was 5. This could be a quota issue.
Jun 14, 2020 12:46:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-14T12:46:12.329Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Jun 14, 2020 12:46:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-14T12:46:27.113Z: Workers have started successfully.
Jun 14, 2020 12:46:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-14T12:46:27.148Z: Workers have started successfully.
Jun 14, 2020 12:46:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-14T12:46:57.587Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Jun 14, 2020 12:46:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-14T12:46:57.958Z: Cleaning up.
Jun 14, 2020 12:46:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-14T12:46:58.043Z: Stopping worker pool...
Jun 14, 2020 12:48:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-14T12:48:48.950Z: Autoscaling: Resized worker pool from 5 to 0.
Jun 14, 2020 12:48:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-14T12:48:48.998Z: Worker pool stopped.
Jun 14, 2020 12:48:55 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-06-14_05_45_32-6291091626775836526 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 821cc11c-8cef-44a3-b8d1-1acbfa254438 and timestamp: 2020-06-14T12:48:55.392000000Z:
Metric: Value:
fields_read 4375276.0
read_time 14.199
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Jun 14, 2020 12:48:55 PM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.022 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.031 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 7,5,main]) completed. Took 3 mins 44.42 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 35s
104 actionable tasks: 68 executed, 36 from cache
Publishing build scan...
https://gradle.com/s/5xnpydaoskpye
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #625
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/625/display/redirect?page=changes>
Changes:
[github] [BEAM-9679] Add Partition task to Core Transform katas (#11979)
------------------------------------------
[...truncated 292.79 KB...]
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:164)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Jun 14, 2020 6:45:26 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Jun 14, 2020 6:45:26 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery writeDisposition is set to: WRITE_EMPTY
Jun 14, 2020 6:45:26 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Jun 14, 2020 6:45:26 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Jun 14, 2020 6:45:26 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery writeDisposition is set to: WRITE_EMPTY
Jun 14, 2020 6:45:26 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Jun 14, 2020 6:45:26 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Jun 14, 2020 6:45:27 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Jun 14, 2020 6:45:27 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Jun 14, 2020 6:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 209 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Jun 14, 2020 6:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil$PackageAttributes forFileToStage
INFO: Staging custom dataflow-worker.jar as beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-Fwexi8aMjBuaNkWr3o4cDW_8B08gOayr8Nv6Dc1A7V0.jar
Jun 14, 2020 6:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-tests-Aqauwy_XC9AO4WBSiRPEdz8Ag6qJoJtgY8zR4Ea7zgA.jar
Jun 14, 2020 6:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-GXABxOB0u5VHHAwdV7tpxDTyuK_x-dIEZGzcuiYSRwk.jar
Jun 14, 2020 6:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-u4y6pbNsJOq7kIMiXe2iBlmsGzbF5K3WwPCV96Tv_Yc.jar
Jun 14, 2020 6:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.23.0-SNAPSHOT-dyQR24_a4R-RKYHOpjV-DhROCjwuh-mRO66GPUPlwz4.jar
Jun 14, 2020 6:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-RVBN9FQg4ZV4oSFfW6XK08TUjHR_i-6vj0TKUdgMRrM.jar
Jun 14, 2020 6:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-Fwexi8aMjBuaNkWr3o4cDW_8B08gOayr8Nv6Dc1A7V0.jar
Jun 14, 2020 6:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-tests-FyOss-jvDF0H2qkIlLK7W10BoQIa5eEl11Tis6WaZHw.jar
Jun 14, 2020 6:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.23.0-SNAPSHOT-79UQKFSk8ttMAyv_ZbOVKPVcCrETImFb3flTgjOTPVA.jar
Jun 14, 2020 6:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-tests-AU0q-TMF6k4Fxd3ac-PcG3UCk-jQrw8Xi_mBOBraca0.jar
Jun 14, 2020 6:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tests-mCXT6_kmT7tdDd2SKWiKahFsu7Hgke4-aFUBrEGp0X0.jar
Jun 14, 2020 6:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-yCUJN7Rd0a9vCoIgZqucIMj5dqvhOiVjPB9447XKsnI.jar
Jun 14, 2020 6:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /tmp/test1049772256794695843.zip to gs://temp-storage-for-perf-tests/loadtests/staging/test-KgfauwfMvYKWkBOFTFs54xak5rEIyQhpzwOuW5PF16Y.jar
Jun 14, 2020 6:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-rllFr2_prRG5-6XsGwAZfK_LNXGf2A5P5CFQGK-Z7ws.jar
Jun 14, 2020 6:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-unshaded--m3tuu9sxVdthwH72XBsYL2ond06deQEiLB1K1M2YaY.jar
Jun 14, 2020 6:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.23.0-SNAPSHOT-hNTLK5jVX3oQOuBN2gCaCCYUPDH0XNqrorRCj8krVys.jar
Jun 14, 2020 6:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT-OkiThNKRTj2r8sx0x2PP-VoL2e8o0-Ca2lDQ4FgZYvE.jar
Jun 14, 2020 6:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.23.0-SNAPSHOT-bC1r93wmXTeN40ofiQBV-bcwUk-xI3lxk0Xo8Tys2nU.jar
Jun 14, 2020 6:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-tests-oHbXOWVJAQ5hUTL-d33wZ8ZyfY5QG4hBmG-vgT9pkdM.jar
Jun 14, 2020 6:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-fn-execution-2.23.0-SNAPSHOT-VNIzD--2cpYSkSiYI6JMKAm-Xzl3VhrvHpekGNm5TaE.jar
Jun 14, 2020 6:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT-00BUkI-Fz5MUfafAbvezlPA_cc2FBaxDdz0nFAcd4GA.jar
Jun 14, 2020 6:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-tests-JdazevX2Y9QsIOECAz_4qUzAIwGky9Q8QfMn52HC6-0.jar
Jun 14, 2020 6:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/java-fn-execution/build/libs/beam-runners-java-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-java-fn-execution-2.23.0-SNAPSHOT-RMbH_fEk06N1I7qwWqjUE9pbBuaqWT1J5RJkU345QHM.jar
Jun 14, 2020 6:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.23.0-SNAPSHOT-uzsDOKoeeubO_DQD2S6KRZU98OLFzxPJWsuNots6Ii0.jar
Jun 14, 2020 6:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-tests-51iUFLCK3wKaPOVvZP7X0AY4zkgxUTldbb-cnaaTd-o.jar
Jun 14, 2020 6:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.23.0-SNAPSHOT-tests-HxW1RWYbU6VRbongM5_OnC-GF-qrvl540PbUrEqW3Gw.jar
Jun 14, 2020 6:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-aC6YhI8bZ0qYkKyxRTDpyHt9GLUxZ5RnbV6eSJq8zTY.jar
Jun 14, 2020 6:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-ypNHsYJi2R1i-G_vtMnNZStSHBw-peszLab39mZSZ88.jar
Jun 14, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-Fwexi8aMjBuaNkWr3o4cDW_8B08gOayr8Nv6Dc1A7V0.jar
Jun 14, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.23.0-SNAPSHOT-6s2BCdyhDCLf_Efjar34ghtZbU_in2CC_fpbNT-MHWI.jar
Jun 14, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.23.0-SNAPSHOT-bWBO_qYDwcBdFjPb1ctYed0sz8lKS4FElnmpGatR-RI.jar
Jun 14, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.23.0-SNAPSHOT-y2uIOr6kEhwmwvi9xyd6dDEFWef7gafbWydVF5ie-B0.jar
Jun 14, 2020 6:45:31 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-Fwexi8aMjBuaNkWr3o4cDW_8B08gOayr8Nv6Dc1A7V0.jar&uploadType=resumable&upload_id=AAANsUlvVLtY98X7-4FqlLVJIsmVhiQLIpqZapxk25tkA_wngyK1NZM8mhrf-BhkYgHbhrUiMh_PUAbO5MiGXvtSow8.
Jun 14, 2020 6:45:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar>
java.io.IOException: Upload failed for 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-Fwexi8aMjBuaNkWr3o4cDW_8B08gOayr8Nv6Dc1A7V0.jar'
at com.google.cloud.hadoop.util.BaseAbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(BaseAbstractGoogleAsyncWriteChannel.java:297)
at com.google.cloud.hadoop.util.BaseAbstractGoogleAsyncWriteChannel.close(BaseAbstractGoogleAsyncWriteChannel.java:214)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:221)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:168)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:152)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:136)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Jun 14, 2020 6:45:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-Fwexi8aMjBuaNkWr3o4cDW_8B08gOayr8Nv6Dc1A7V0.jar
Jun 14, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 31 files newly uploaded in 7 seconds
Jun 14, 2020 6:45:37 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Jun 14, 2020 6:45:37 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Jun 14, 2020 6:45:37 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Jun 14, 2020 6:45:37 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Jun 14, 2020 6:45:37 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Jun 14, 2020 6:45:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <91368 bytes, hash d1ab41e2091e1f5286f5cb7ed70af82c9132eb293ccf7050e39190d06c823e58> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-0atB4gkeH1KG9ct-1wr4LJEy6yk8z3BQ45GQ0GyCPlg.pb
Jun 14, 2020 6:45:38 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.23.0-SNAPSHOT
Jun 14, 2020 6:45:39 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-06-13_23_45_38-8162264493969991375?project=apache-beam-testing
Jun 14, 2020 6:45:39 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-06-13_23_45_38-8162264493969991375
Jun 14, 2020 6:45:39 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-06-13_23_45_38-8162264493969991375
Jun 14, 2020 6:45:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-06-14T06:45:38.105Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Jun 14, 2020 6:45:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-14T06:45:45.320Z: Worker configuration: n1-standard-1 in us-central1-a.
Jun 14, 2020 6:45:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-14T06:45:45.939Z: Expanding CoGroupByKey operations into optimizable parts.
Jun 14, 2020 6:45:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-14T06:45:45.977Z: Expanding GroupByKey operations into optimizable parts.
Jun 14, 2020 6:45:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-14T06:45:46.007Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jun 14, 2020 6:45:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-14T06:45:46.086Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jun 14, 2020 6:45:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-14T06:45:46.114Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Jun 14, 2020 6:45:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-14T06:45:46.151Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Jun 14, 2020 6:45:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-14T06:45:46.186Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Jun 14, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-14T06:45:46.608Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Jun 14, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-14T06:45:46.693Z: Starting 5 workers in us-central1-a...
Jun 14, 2020 6:45:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-06-14T06:45:53.680Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jun 14, 2020 6:46:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-14T06:46:19.053Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Jun 14, 2020 6:46:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-14T06:46:33.709Z: Workers have started successfully.
Jun 14, 2020 6:46:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-14T06:46:33.739Z: Workers have started successfully.
Jun 14, 2020 6:47:04 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-14T06:47:04.218Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Jun 14, 2020 6:47:04 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-14T06:47:04.409Z: Cleaning up.
Jun 14, 2020 6:47:04 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-14T06:47:04.501Z: Stopping worker pool...
Jun 14, 2020 6:49:06 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-14T06:49:06.521Z: Autoscaling: Resized worker pool from 5 to 0.
Jun 14, 2020 6:49:06 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-14T06:49:06.566Z: Worker pool stopped.
Jun 14, 2020 6:49:13 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-06-13_23_45_38-8162264493969991375 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 25ba6109-b2a2-4637-ab23-3fb45440caf1 and timestamp: 2020-06-14T06:49:13.166000000Z:
Metric: Value:
fields_read 4375276.0
read_time 14.708
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Jun 14, 2020 6:49:13 AM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.026 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.033 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Daemon worker,5,main]) completed. Took 3 mins 55.629 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 49s
104 actionable tasks: 68 executed, 36 from cache
Publishing build scan...
https://gradle.com/s/jv4ytypamqmxm
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #624
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/624/display/redirect>
Changes:
------------------------------------------
[...truncated 291.76 KB...]
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:164)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Jun 14, 2020 12:45:24 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Jun 14, 2020 12:45:24 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery writeDisposition is set to: WRITE_EMPTY
Jun 14, 2020 12:45:24 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Jun 14, 2020 12:45:24 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Jun 14, 2020 12:45:24 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery writeDisposition is set to: WRITE_EMPTY
Jun 14, 2020 12:45:24 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Jun 14, 2020 12:45:24 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Jun 14, 2020 12:45:24 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Jun 14, 2020 12:45:25 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Jun 14, 2020 12:45:27 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 209 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Jun 14, 2020 12:45:27 AM org.apache.beam.runners.dataflow.util.PackageUtil$PackageAttributes forFileToStage
INFO: Staging custom dataflow-worker.jar as beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-VoLQ4WYNAZiyyiW8prkQDvi3W6iA_Hd68epf06d0_xQ.jar
Jun 14, 2020 12:45:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT-ffJurHPYAd9u09BxGvhvgtYGjSXHX0Q7gKh6J_o-VKg.jar
Jun 14, 2020 12:45:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.23.0-SNAPSHOT-ebr4wvS14cA0mDM5Q7pXWKqe-fPzbPwSsV65Vlz3ahg.jar
Jun 14, 2020 12:45:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-iFK1lsUZqLi2XkUs3lteFFqTyF1qUdRukGdRTE5igpw.jar
Jun 14, 2020 12:45:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-l1kJIbNcCPrGxhkXmnj6dRIy6v6feIkkdJj5Q457jHQ.jar
Jun 14, 2020 12:45:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /tmp/test3299111542074381156.zip to gs://temp-storage-for-perf-tests/loadtests/staging/test-N_RKgyLOCEni8z-l_ubQFTcnEjVB0IUNx8O-8ZydVUw.jar
Jun 14, 2020 12:45:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-ipBA2pUVCISVVd7hKSOxXSd0vcLhuJGrwFGl-sw_0iw.jar
Jun 14, 2020 12:45:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-T65UKsRUJQcWiLFI61ErBtPYsyYvh3aVjNgLucHBSAU.jar
Jun 14, 2020 12:45:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-jG3TTR7mJcxIBEjrVXxpIV_eIHWWOUcoNfOQcKKj5pQ.jar
Jun 14, 2020 12:45:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-DYZcf_rAdKpfI8BXaw3yQaDF4TXmtkiPCKUFUu2C8ys.jar
Jun 14, 2020 12:45:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-0kNMyjycLiEnt9bCZ-Yvr6amEZPuIJO_zsYPQcZcnIE.jar
Jun 14, 2020 12:45:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-fn-execution-2.23.0-SNAPSHOT-z1DM0QocapGV6fjHYKQadRVoZCVDbkQTXiErwsrYk1o.jar
Jun 14, 2020 12:45:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-tests-4tGM9y1F_gx_u-raFmAsczCGI1KPe1P4QTHpUFoxA30.jar
Jun 14, 2020 12:45:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-tests-PWhJzdFsnVI03eGnW0C3iX9EmhtleCbDeogC0caiPaw.jar
Jun 14, 2020 12:45:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-tests-Boi5zttNpswucz-vwXdLs3c-XofvvYHDPB8VoXYHD4w.jar
Jun 14, 2020 12:45:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/java-fn-execution/build/libs/beam-runners-java-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-java-fn-execution-2.23.0-SNAPSHOT-ENrxknJJBKMeN3D-kEB0xnH1qDrl1o4lwv2oLFpzg1k.jar
Jun 14, 2020 12:45:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.23.0-SNAPSHOT-NqPZbslsnPNizSfWOFro7SlleMjoC7MqNRE5r2r1y5c.jar
Jun 14, 2020 12:45:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT-mtdVTseN1NVfQF3oyj1OmX7eztdEHrq7NePWZiTxiIY.jar
Jun 14, 2020 12:45:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.23.0-SNAPSHOT-tests-jde-2ON0EFDgv6CM1lBXj6HNYGLy3vkdzrCi1DGOzVs.jar
Jun 14, 2020 12:45:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-tests-pUogku8nRGZ5wteYd6dLURpQaztgj3XSK6pAsCWBfHU.jar
Jun 14, 2020 12:45:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tests-8gf1-Ksj87mSYDSb2krxs77rcc7g5Hm7D6dHOzGoD3g.jar
Jun 14, 2020 12:45:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-VoLQ4WYNAZiyyiW8prkQDvi3W6iA_Hd68epf06d0_xQ.jar
Jun 14, 2020 12:45:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.23.0-SNAPSHOT-cN4tu-tPkqEA_5Dkn5aH3n-kV51txSvvRxgcGkNr4nI.jar
Jun 14, 2020 12:45:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.23.0-SNAPSHOT-VPhzIOnyqU2aU_w2kV7ZhV2TD0dhyc9SufVNTpRoDe4.jar
Jun 14, 2020 12:45:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.23.0-SNAPSHOT-7hvFCW_mJT64_Qf_MPfGxAkW40fpcHV6CBD_SCJVzB8.jar
Jun 14, 2020 12:45:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-tests-OUdUr8byhbkY4GEyFjtIsbS3bHEH2Eu6ccRJ2jOZ_MQ.jar
Jun 14, 2020 12:45:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-unshaded-6BoNcVcxB18xYz4cqTfsb-bm92Am3Zw34I5R4G-R-yM.jar
Jun 14, 2020 12:45:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-tests-NWw9mtqjbjjz5Q8PNeyO5FUf_mzBCV6v6Yo8xN9wM-8.jar
Jun 14, 2020 12:45:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-VoLQ4WYNAZiyyiW8prkQDvi3W6iA_Hd68epf06d0_xQ.jar
Jun 14, 2020 12:45:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.23.0-SNAPSHOT-zUdg_HVscupIdiuUuDDvIcyKqwvLYvLw6YGL_r5g2QA.jar
Jun 14, 2020 12:45:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.23.0-SNAPSHOT-dkCyE_tleKULrnZDCRb8L7_A8eZW9eEbWahsrnmQyvk.jar
Jun 14, 2020 12:45:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.23.0-SNAPSHOT-HpSFhsPYNufT-iX9TDmxvLcr8zUgepaFe4766l-lrxw.jar
Jun 14, 2020 12:45:28 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-VoLQ4WYNAZiyyiW8prkQDvi3W6iA_Hd68epf06d0_xQ.jar&uploadType=resumable&upload_id=AAANsUklqgCpDFl9qRUdM5uAEvxUeB2ClNl4v7YidGWUUIK2AAobPUQmPPkLoyfCpP4OIilMvNPaEPNHw3m29NZfQGc.
Jun 14, 2020 12:45:28 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar>
java.io.IOException: Upload failed for 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-VoLQ4WYNAZiyyiW8prkQDvi3W6iA_Hd68epf06d0_xQ.jar'
at com.google.cloud.hadoop.util.BaseAbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(BaseAbstractGoogleAsyncWriteChannel.java:297)
at com.google.cloud.hadoop.util.BaseAbstractGoogleAsyncWriteChannel.close(BaseAbstractGoogleAsyncWriteChannel.java:214)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:221)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:168)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:152)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:136)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Jun 14, 2020 12:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-VoLQ4WYNAZiyyiW8prkQDvi3W6iA_Hd68epf06d0_xQ.jar
Jun 14, 2020 12:45:32 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 31 files newly uploaded in 5 seconds
Jun 14, 2020 12:45:33 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Jun 14, 2020 12:45:33 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Jun 14, 2020 12:45:33 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Jun 14, 2020 12:45:33 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Jun 14, 2020 12:45:33 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Jun 14, 2020 12:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <91368 bytes, hash e684795794ef584d4129981cac8099dde107c4913d3cae4ddeb101f70bfc858a> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-5oR5V5TvWE1BKZgcrICZ3eEHxJE9PK5N3rEB9wv8hYo.pb
Jun 14, 2020 12:45:33 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.23.0-SNAPSHOT
Jun 14, 2020 12:45:35 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-06-13_17_45_33-16992201563020243372?project=apache-beam-testing
Jun 14, 2020 12:45:35 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-06-13_17_45_33-16992201563020243372
Jun 14, 2020 12:45:35 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-06-13_17_45_33-16992201563020243372
Jun 14, 2020 12:45:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-06-14T00:45:33.731Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Jun 14, 2020 12:45:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-14T00:45:41.858Z: Worker configuration: n1-standard-1 in us-central1-a.
Jun 14, 2020 12:45:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-14T00:45:42.861Z: Expanding CoGroupByKey operations into optimizable parts.
Jun 14, 2020 12:45:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-14T00:45:42.901Z: Expanding GroupByKey operations into optimizable parts.
Jun 14, 2020 12:45:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-14T00:45:42.939Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jun 14, 2020 12:45:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-14T00:45:43.031Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jun 14, 2020 12:45:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-14T00:45:43.067Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Jun 14, 2020 12:45:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-14T00:45:43.105Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Jun 14, 2020 12:45:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-14T00:45:43.142Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Jun 14, 2020 12:45:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-14T00:45:43.593Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Jun 14, 2020 12:45:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-14T00:45:43.694Z: Starting 5 workers in us-central1-a...
Jun 14, 2020 12:46:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-06-14T00:46:06.554Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jun 14, 2020 12:46:10 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-14T00:46:10.340Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Jun 14, 2020 12:46:30 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-14T00:46:28.044Z: Workers have started successfully.
Jun 14, 2020 12:46:30 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-14T00:46:28.082Z: Workers have started successfully.
Jun 14, 2020 12:47:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-14T00:46:59.814Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Jun 14, 2020 12:47:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-14T00:47:00.040Z: Cleaning up.
Jun 14, 2020 12:47:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-14T00:47:00.118Z: Stopping worker pool...
Jun 14, 2020 12:49:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-14T00:49:05.457Z: Autoscaling: Resized worker pool from 5 to 0.
Jun 14, 2020 12:49:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-14T00:49:05.509Z: Worker pool stopped.
Jun 14, 2020 12:49:15 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-06-13_17_45_33-16992201563020243372 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): bc0d9735-41e9-4909-adee-86e7b019d28d and timestamp: 2020-06-14T00:49:15.162000000Z:
Metric: Value:
fields_read 4375276.0
read_time 15.658
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Jun 14, 2020 12:49:15 AM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.025 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.033 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':',5,main]) completed. Took 4 mins 0.34 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 52s
104 actionable tasks: 68 executed, 36 from cache
Publishing build scan...
https://gradle.com/s/zybmyluhkrojg
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #623
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/623/display/redirect>
Changes:
------------------------------------------
[...truncated 294.47 KB...]
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:164)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Jun 13, 2020 6:47:22 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Jun 13, 2020 6:47:22 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery writeDisposition is set to: WRITE_EMPTY
Jun 13, 2020 6:47:22 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Jun 13, 2020 6:47:22 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Jun 13, 2020 6:47:22 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery writeDisposition is set to: WRITE_EMPTY
Jun 13, 2020 6:47:22 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Jun 13, 2020 6:47:22 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Jun 13, 2020 6:47:22 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Jun 13, 2020 6:47:23 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Jun 13, 2020 6:47:25 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 209 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Jun 13, 2020 6:47:25 PM org.apache.beam.runners.dataflow.util.PackageUtil$PackageAttributes forFileToStage
INFO: Staging custom dataflow-worker.jar as beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-_SiuHci_NnenekI2u5-D6cWHYyg1OV8yxrEbrq5317M.jar
Jun 13, 2020 6:47:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/java-fn-execution/build/libs/beam-runners-java-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-java-fn-execution-2.23.0-SNAPSHOT-r66Y6CqbftBu40HppI5PsG59za0BZBjVHYnFAer_CiM.jar
Jun 13, 2020 6:47:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-unshaded-YzZe6Zgi40PqU9SdtOeCBX2LaF9yjF8W_-8lL6lK7ZI.jar
Jun 13, 2020 6:47:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-l8G4bQzUAjkd--a5_0M0TxV8ME39j6bvyCNfSIoNb2w.jar
Jun 13, 2020 6:47:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-2mTscyJERliM4lHoHfo2HaL4LWL1RQ20iKMtrfFYTMI.jar
Jun 13, 2020 6:47:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-tests-6mUNdE3EIuiTM7tH-lfAHj18kDexv6fqmLaWzz-1g8g.jar
Jun 13, 2020 6:47:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.23.0-SNAPSHOT-N0eZB3ERq39oYjFqAMSxZdKX3Pqf7xHf9w2x3vLn3-g.jar
Jun 13, 2020 6:47:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /tmp/test3886874396143450321.zip to gs://temp-storage-for-perf-tests/loadtests/staging/test-oiuFWPW0tMRWBrMKzX3WzxaU7eLRzuo-ecy51mpAqWM.jar
Jun 13, 2020 6:47:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.23.0-SNAPSHOT-Hwd9BUwWEgOUUvklbV8X7n3fjorTC-1zUN1Cksi2UIo.jar
Jun 13, 2020 6:47:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-tests-WB8lpSoYtd5Rs5evJntMVAoCG4ohEnUHgo4atS3ok-k.jar
Jun 13, 2020 6:47:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.23.0-SNAPSHOT-YHoPc3-8OiXwalQbURSgfq336vrsQpmjMP6T_lgbA5c.jar
Jun 13, 2020 6:47:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-tests-KaIeoWZ8JVsstLubn6cgwLMSoNieoVnmwBcgDloyz3Y.jar
Jun 13, 2020 6:47:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-iwkLghsra-V5nI8HBLpwdSn0pCwSbfqNNMhX-qmRCec.jar
Jun 13, 2020 6:47:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.23.0-SNAPSHOT-YrQEgVkIYnCWZ_IK7puyeIpTiPqTpKp_Vzot3BP3_aU.jar
Jun 13, 2020 6:47:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-wobrJUNAASn24jiNkdFkk1IVrjMjgPG1XlMwaLmvg7A.jar
Jun 13, 2020 6:47:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-_SiuHci_NnenekI2u5-D6cWHYyg1OV8yxrEbrq5317M.jar
Jun 13, 2020 6:47:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-tests-h0qNRdkBi5sJKWhNvMElKO4eQNraQiISYxDomJI3-Z0.jar
Jun 13, 2020 6:47:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-tests-OQqKNkvqBWsNnhdPvYY9mF0jTY0_PNVN5fslMvLFZ-o.jar
Jun 13, 2020 6:47:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-rg306kWA0FNCxeQGbJcsrAT-xfTFwTMUc_-KJLkWAHM.jar
Jun 13, 2020 6:47:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.23.0-SNAPSHOT-bZT8o8FKU5qo-9af7pVdddYF-A-P6HvcoTSpcTxU_bU.jar
Jun 13, 2020 6:47:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.23.0-SNAPSHOT-tests-rWtzqvZTEvoSTMWifd1atInmWjEYgx9pEEGOzZlMxuc.jar
Jun 13, 2020 6:47:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tests-62fO7w9Ok_mFRZ5o8RfuM0vQ0ECKopsoXy61XuzD4BE.jar
Jun 13, 2020 6:47:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-ReeULjXWdfc5Lr7ylepkrLvvkDhr_d3xqTmwef2VMxQ.jar
Jun 13, 2020 6:47:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-tests-C9UE9EZ7bzxhggvQo3hJ4-xmptu8QGZsWoost2szBBI.jar
Jun 13, 2020 6:47:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT-VXpIqCFZc33Zu9Z14LWIAOSsBgBUhncwBHgDy5gbAhA.jar
Jun 13, 2020 6:47:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-3q7F17onxvB8VQJTdSMJAy4E2JyNVCoDXP_5gwNAInM.jar
Jun 13, 2020 6:47:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT-1k-i8wbFmrnc-ufC3_jhPyU2h3wkdp0Tqt4_nPL6WBk.jar
Jun 13, 2020 6:47:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-fn-execution-2.23.0-SNAPSHOT-rYErGXxHfaBdUyyxN-wAr5kHSMQKkD4x0_1HKDh5Ts0.jar
Jun 13, 2020 6:47:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-_SiuHci_NnenekI2u5-D6cWHYyg1OV8yxrEbrq5317M.jar
Jun 13, 2020 6:47:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.23.0-SNAPSHOT-AEkBipsx0NsXu8A0s6-6xHBJIv4uvYqQ8RMXaej9BuA.jar
Jun 13, 2020 6:47:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.23.0-SNAPSHOT-7AV-B7m-7hHgDQUqsMT7YYPcafGGDDFtQp5aE3q3OQI.jar
Jun 13, 2020 6:47:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.23.0-SNAPSHOT-AA8edUfIt05oO66_zYFuaBVREQyoekRcCJhEW1vf8S8.jar
Jun 13, 2020 6:47:26 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-_SiuHci_NnenekI2u5-D6cWHYyg1OV8yxrEbrq5317M.jar&uploadType=resumable&upload_id=AAANsUnfZQuOuyU47ujtVZDkhp_jYSgPYrwerCA-H4_YKaRxShDSUpsSC3rwJTWTXk3g1C8vz0gTOkklyZcZdhJVDBFLzWOGlw.
Jun 13, 2020 6:47:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar>
java.io.IOException: Upload failed for 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-_SiuHci_NnenekI2u5-D6cWHYyg1OV8yxrEbrq5317M.jar'
at com.google.cloud.hadoop.util.BaseAbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(BaseAbstractGoogleAsyncWriteChannel.java:297)
at com.google.cloud.hadoop.util.BaseAbstractGoogleAsyncWriteChannel.close(BaseAbstractGoogleAsyncWriteChannel.java:214)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:221)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:168)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:152)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:136)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Jun 13, 2020 6:47:33 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-_SiuHci_NnenekI2u5-D6cWHYyg1OV8yxrEbrq5317M.jar
Jun 13, 2020 6:47:33 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 31 files newly uploaded in 8 seconds
Jun 13, 2020 6:47:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Jun 13, 2020 6:47:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Jun 13, 2020 6:47:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Jun 13, 2020 6:47:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Jun 13, 2020 6:47:34 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Jun 13, 2020 6:47:34 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <91368 bytes, hash 4eb92fd18a15679fd44ccd31673117c1e4f2214739923cf7246fecfbf21719e2> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-Trkv0YoVZ5_UTM0xZzEXweTyIUc5kjz3JG_s-_IXGeI.pb
Jun 13, 2020 6:47:34 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.23.0-SNAPSHOT
Jun 13, 2020 6:47:35 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-06-13_11_47_34-15575361032439528658?project=apache-beam-testing
Jun 13, 2020 6:47:35 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-06-13_11_47_34-15575361032439528658
Jun 13, 2020 6:47:35 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-06-13_11_47_34-15575361032439528658
Jun 13, 2020 6:47:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-06-13T18:47:34.725Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Jun 13, 2020 6:47:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-13T18:47:42.110Z: Worker configuration: n1-standard-1 in us-central1-a.
Jun 13, 2020 6:47:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-13T18:47:42.838Z: Expanding CoGroupByKey operations into optimizable parts.
Jun 13, 2020 6:47:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-13T18:47:42.884Z: Expanding GroupByKey operations into optimizable parts.
Jun 13, 2020 6:47:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-13T18:47:42.924Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jun 13, 2020 6:47:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-13T18:47:43.065Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jun 13, 2020 6:47:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-13T18:47:43.094Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Jun 13, 2020 6:47:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-13T18:47:43.130Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Jun 13, 2020 6:47:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-13T18:47:43.163Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Jun 13, 2020 6:47:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-13T18:47:43.591Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Jun 13, 2020 6:47:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-13T18:47:43.675Z: Starting 5 workers in us-central1-a...
Jun 13, 2020 6:47:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-06-13T18:47:55.706Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jun 13, 2020 6:48:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-13T18:48:19.127Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Jun 13, 2020 6:48:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-13T18:48:36.444Z: Workers have started successfully.
Jun 13, 2020 6:48:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-13T18:48:36.471Z: Workers have started successfully.
Jun 13, 2020 6:49:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-13T18:49:09.602Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Jun 13, 2020 6:49:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-13T18:49:09.863Z: Cleaning up.
Jun 13, 2020 6:49:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-13T18:49:09.956Z: Stopping worker pool...
Jun 13, 2020 6:51:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-13T18:51:08.728Z: Autoscaling: Resized worker pool from 5 to 0.
Jun 13, 2020 6:51:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-13T18:51:08.773Z: Worker pool stopped.
Jun 13, 2020 6:51:15 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-06-13_11_47_34-15575361032439528658 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): ff6102d2-813a-43af-b91e-b5115c6ee911 and timestamp: 2020-06-13T18:51:15.889000000Z:
Metric: Value:
read_time 15.035
fields_read 4375276.0
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Jun 13, 2020 6:51:16 PM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 2 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.024 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.039 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 6,5,main]) completed. Took 4 mins 2.426 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 17s
104 actionable tasks: 71 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/ussj32ahwdunm
Stopped 1 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #622
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/622/display/redirect?page=changes>
Changes:
[daniel.o.programmer] [BEAM-10235] Adding a CountElms transform to the Go SDK.
------------------------------------------
[...truncated 294.42 KB...]
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:164)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Jun 13, 2020 12:45:21 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Jun 13, 2020 12:45:21 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery writeDisposition is set to: WRITE_EMPTY
Jun 13, 2020 12:45:21 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Jun 13, 2020 12:45:21 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Jun 13, 2020 12:45:21 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery writeDisposition is set to: WRITE_EMPTY
Jun 13, 2020 12:45:21 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Jun 13, 2020 12:45:21 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Jun 13, 2020 12:45:21 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Jun 13, 2020 12:45:22 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Jun 13, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 209 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Jun 13, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil$PackageAttributes forFileToStage
INFO: Staging custom dataflow-worker.jar as beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-3r6Cyj44dD0-2H0O4-lbqdULInO8U_XBDPv9fE3_8aI.jar
Jun 13, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-tests-FftD7wkhc_lhLBgaqRbT7clfBFRQD67tIM9mj9-mZSU.jar
Jun 13, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-Lm2evKPOx3tGCmrLc1LKCz_eg2-FiyvbkMXR235Y9vE.jar
Jun 13, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-hGpvBsIzA4JhlTAm1rU-ypQac1zmYGtP4HnaVCtxu68.jar
Jun 13, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-7RZtHU7NwKavh03Mt9mUKeXVSW18LjzNpxxcdLDy0Po.jar
Jun 13, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /tmp/test211733552788212626.zip to gs://temp-storage-for-perf-tests/loadtests/staging/test-7011ce-cjKJE6W7zmUa0vBo6D5JagzJWVuI2Z-1TOw8.jar
Jun 13, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-tests-CkN31tICHDUTTEGVi7C-XJRqjVRBEsZNE47whuD7koI.jar
Jun 13, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.23.0-SNAPSHOT-tests-6kJvw1B3IbJeVSaf0fszY3ALCBH1wcumIkx-_xuVOUg.jar
Jun 13, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-3r6Cyj44dD0-2H0O4-lbqdULInO8U_XBDPv9fE3_8aI.jar
Jun 13, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-unOP42KQlkMlg1m0UmrtCciS5zj19x_ClksZnYj_pU4.jar
Jun 13, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT-r5dPOb78HqBusYTNJWKziwh4uR-5pG6R2-KdhHfV-tY.jar
Jun 13, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-tests-tag1qpp35qxRD2NbhvrIgpA9vuBEt_NzlMJ3XTLsDn0.jar
Jun 13, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/java-fn-execution/build/libs/beam-runners-java-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-java-fn-execution-2.23.0-SNAPSHOT-X5msu-gBfUMlqSpCNb1pTQ6n2A2aq5rOLKaOObPxdlM.jar
Jun 13, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.23.0-SNAPSHOT-vVW9obitYx24gvRY3hgooZzojVr4xGtXO3wGGvZpoC4.jar
Jun 13, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-tests-Z3fna8dDDKUEJzY0GuGyldK4b00V6d24FFyUsybEUQ0.jar
Jun 13, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.23.0-SNAPSHOT-oPjx58X6a9JTpNf5PnlBFniLnTL6yta1Ghs1yh777OI.jar
Jun 13, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-mvs9oy6MyLYYNZd5zlsRJdgGwTMoDbOrwDn5jM6Y1g0.jar
Jun 13, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-R9rdqgeGrXPejCY8TXeuenun8xV9syxoR2F3vVbiYMM.jar
Jun 13, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.23.0-SNAPSHOT-sb2ox7_9Uq1qhfDbRjdpbpeZX97loMe9PhxgnTJ8bmE.jar
Jun 13, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-bpNhJIi1Ew12z5GpOdFImaTnJXZigFxVee8VMuPqHMY.jar
Jun 13, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tests-87pRKrg51aB8F1uKNWBPjN80TMQfSiT-H77dYt4WPU4.jar
Jun 13, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.23.0-SNAPSHOT-7N04MxFA0IJfX0PTkb3TCcGVrScay8XzBCsaQIDO0mI.jar
Jun 13, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-tests-eWCpABJ-Xj6LRL8RvBD7Dbx70PIK6KJCBjOchJHIbXA.jar
Jun 13, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.23.0-SNAPSHOT-eRD2KU9bhHi-Q939_nutvIEdokwGQ4BHssmOcHUygBc.jar
Jun 13, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-fn-execution-2.23.0-SNAPSHOT-myQUPID6qFpbKws7HpV-V9Sgab9mjTEXlX_9YjMEHIU.jar
Jun 13, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT-fHa5MXCO9VNgC5dE1kVZi5KjhaqdvX0YTzhjWrys5zw.jar
Jun 13, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-unshaded-IDn0NyXycUrvyfly2lICLKAXwr70Cs67n_47PNul0dg.jar
Jun 13, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-tests-IdnSCjHpkPtQnTXNYa9UoGcVanvrZnKMhIGgGSUHWoE.jar
Jun 13, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-3r6Cyj44dD0-2H0O4-lbqdULInO8U_XBDPv9fE3_8aI.jar
Jun 13, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.23.0-SNAPSHOT-VvsfsFN_YG6wmU5EYBUSx0g6bKNffO0UnkAT1HKmwoo.jar
Jun 13, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.23.0-SNAPSHOT-FMAIg7P89BbpZ1vDaGLhG091WS5aKnh-_ptrzTf615M.jar
Jun 13, 2020 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.23.0-SNAPSHOT-s3N38wp4iNpZ6zJK2FspV805PE_YTH5JwrHgtnyZpXQ.jar
Jun 13, 2020 12:45:25 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-3r6Cyj44dD0-2H0O4-lbqdULInO8U_XBDPv9fE3_8aI.jar&uploadType=resumable&upload_id=AAANsUnw9nlJHHo6fNWumhWtDN0gFiTAAXA9zb_iBQ_JfjvmRfitL7DryC-MJT5mSJhkAyvjho7Kfa9IAnjt2YBukQD4eRiv1Q.
Jun 13, 2020 12:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar>
java.io.IOException: Upload failed for 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-3r6Cyj44dD0-2H0O4-lbqdULInO8U_XBDPv9fE3_8aI.jar'
at com.google.cloud.hadoop.util.BaseAbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(BaseAbstractGoogleAsyncWriteChannel.java:297)
at com.google.cloud.hadoop.util.BaseAbstractGoogleAsyncWriteChannel.close(BaseAbstractGoogleAsyncWriteChannel.java:214)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:221)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:168)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:152)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:136)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Jun 13, 2020 12:45:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-3r6Cyj44dD0-2H0O4-lbqdULInO8U_XBDPv9fE3_8aI.jar
Jun 13, 2020 12:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 31 files newly uploaded in 6 seconds
Jun 13, 2020 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Jun 13, 2020 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Jun 13, 2020 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Jun 13, 2020 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Jun 13, 2020 12:45:30 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Jun 13, 2020 12:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <91367 bytes, hash dffd711b701a14bc0fd14444f6ae09ddc05f7c45166d91dac01d43786bc91c3e> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-3_1xG3AaFLwP0URE9q4J3cBffEUWbZHawB1DeGvJHD4.pb
Jun 13, 2020 12:45:31 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.23.0-SNAPSHOT
Jun 13, 2020 12:45:32 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-06-13_05_45_31-1205351863372966564?project=apache-beam-testing
Jun 13, 2020 12:45:32 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-06-13_05_45_31-1205351863372966564
Jun 13, 2020 12:45:32 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-06-13_05_45_31-1205351863372966564
Jun 13, 2020 12:45:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-06-13T12:45:31.274Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Jun 13, 2020 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-13T12:45:38.496Z: Worker configuration: n1-standard-1 in us-central1-a.
Jun 13, 2020 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-13T12:45:39.301Z: Expanding CoGroupByKey operations into optimizable parts.
Jun 13, 2020 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-13T12:45:39.345Z: Expanding GroupByKey operations into optimizable parts.
Jun 13, 2020 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-13T12:45:39.364Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jun 13, 2020 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-13T12:45:39.440Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jun 13, 2020 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-13T12:45:39.471Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Jun 13, 2020 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-13T12:45:39.503Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Jun 13, 2020 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-13T12:45:39.533Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Jun 13, 2020 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-13T12:45:39.885Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Jun 13, 2020 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-13T12:45:39.949Z: Starting 5 workers in us-central1-a...
Jun 13, 2020 12:46:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-06-13T12:46:02.656Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jun 13, 2020 12:46:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-13T12:46:11.514Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Jun 13, 2020 12:46:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-13T12:46:28.058Z: Workers have started successfully.
Jun 13, 2020 12:46:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-13T12:46:28.093Z: Workers have started successfully.
Jun 13, 2020 12:46:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-13T12:46:58.811Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Jun 13, 2020 12:46:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-13T12:46:59.002Z: Cleaning up.
Jun 13, 2020 12:46:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-13T12:46:59.072Z: Stopping worker pool...
Jun 13, 2020 12:48:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-13T12:48:27.742Z: Autoscaling: Resized worker pool from 5 to 0.
Jun 13, 2020 12:48:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-13T12:48:27.781Z: Worker pool stopped.
Jun 13, 2020 12:48:33 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-06-13_05_45_31-1205351863372966564 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): ad45c0c2-b616-4a32-9a7e-d01725430d1a and timestamp: 2020-06-13T12:48:33.994000000Z:
Metric: Value:
fields_read 4375276.0
read_time 13.003
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Jun 13, 2020 12:48:34 PM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.024 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.032 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 2,5,main]) completed. Took 3 mins 24.135 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 13s
104 actionable tasks: 68 executed, 36 from cache
Publishing build scan...
https://gradle.com/s/3w5wzdjxakore
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #621
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/621/display/redirect>
Changes:
------------------------------------------
[...truncated 294.81 KB...]
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Jun 13, 2020 6:45:36 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Jun 13, 2020 6:45:36 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery writeDisposition is set to: WRITE_EMPTY
Jun 13, 2020 6:45:36 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Jun 13, 2020 6:45:36 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Jun 13, 2020 6:45:36 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery writeDisposition is set to: WRITE_EMPTY
Jun 13, 2020 6:45:36 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Jun 13, 2020 6:45:37 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Jun 13, 2020 6:45:37 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Jun 13, 2020 6:45:37 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Jun 13, 2020 6:45:39 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 209 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Jun 13, 2020 6:45:39 AM org.apache.beam.runners.dataflow.util.PackageUtil$PackageAttributes forFileToStage
INFO: Staging custom dataflow-worker.jar as beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-B6wYOsdhk_PMYp8-MVjgU7CmsxXCsW0cBjRvkSowapM.jar
Jun 13, 2020 6:45:39 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.23.0-SNAPSHOT-fm8mEMjM61qvwGkhkGJpT0eNifcCIMiYzbGBurWNhHg.jar
Jun 13, 2020 6:45:39 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-tests-8SXGpjajHfBvAcCzKm-WbCCyNKbeGSFIgKn7FcdhRuI.jar
Jun 13, 2020 6:45:39 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.23.0-SNAPSHOT-aoA0tO3Nx5GlNGf0Y6xVY8DZFXNef6OcfBQuzeVaRjc.jar
Jun 13, 2020 6:45:39 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-tests-hNJlwpJv5mMupOxRJLqULaWtH1DmQT59HCtOxfqqLDc.jar
Jun 13, 2020 6:45:39 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tests-TlSwCr2o5n7Qjr9ruLYLqZdrm4vhyL3BAySPQ0QQ_YM.jar
Jun 13, 2020 6:45:39 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT-D-MYg3XJGCatc5KNmdqVUogPoma6wlg5TB6mmL5V6RY.jar
Jun 13, 2020 6:45:39 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-YIUDJJ1Ua6cXbsx197_pan_QxwYozTiSL2RXo2ZFDA0.jar
Jun 13, 2020 6:45:39 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /tmp/test2807025952301607855.zip to gs://temp-storage-for-perf-tests/loadtests/staging/test-Hco_POoNVlwvvNUMP7x-gbzyVTCrVpqizT9RZ9z0msI.jar
Jun 13, 2020 6:45:39 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-gfoAV6n2h7_WLchZ24tE6g-Yqx1vg43x4KlaJ8Y6sPI.jar
Jun 13, 2020 6:45:39 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-tests-N4gr1GgEDoI7LZGVI7HAmRTkJqB-UKfHOIrrxGnOIbM.jar
Jun 13, 2020 6:45:39 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-JGHtW-f0epyWIUjd7n_9eMOFasPqISynHsoYjOz69b0.jar
Jun 13, 2020 6:45:39 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/java-fn-execution/build/libs/beam-runners-java-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-java-fn-execution-2.23.0-SNAPSHOT-q_d0g73ScUMvCpa9iPj-cgEZhsth7Qvzsk2bcCiNCLY.jar
Jun 13, 2020 6:45:39 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-Bcro_tIaLr8RHphEJyBdNoyX4W9F09tzwYTggAETFGw.jar
Jun 13, 2020 6:45:39 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-As-SUBcnUBzTCFgmuUM3pTZQirOdFaxQNj7CFgejUHY.jar
Jun 13, 2020 6:45:39 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-tests-BZefKdl8wX516WmSyqJVvoEZuQIUrVnGnUVvfEfGHo4.jar
Jun 13, 2020 6:45:39 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-yzs7g6i6YASMGTDYX8zaQ15fa70s-v_3JIHuOn85rek.jar
Jun 13, 2020 6:45:39 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.23.0-SNAPSHOT-JnpgX3m8V1rNdEedcnXQVzDlUmwrYIFiuiwNj7oNfIA.jar
Jun 13, 2020 6:45:39 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-B6wYOsdhk_PMYp8-MVjgU7CmsxXCsW0cBjRvkSowapM.jar
Jun 13, 2020 6:45:39 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.23.0-SNAPSHOT-G_9p0WJnSJuuYMOE9sUtlC46GF4Lp_US-VHAlJSX0LU.jar
Jun 13, 2020 6:45:39 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-tests-aEOr29XFON9KIEpjeKRPDloutKxZD5tfhFNfSoL7AJs.jar
Jun 13, 2020 6:45:39 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.23.0-SNAPSHOT-XQmPWjPqr9w1hfyfsHhpYpmnXJGBrvUikubRyvrKfJ8.jar
Jun 13, 2020 6:45:39 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-fn-execution-2.23.0-SNAPSHOT-25W2atZdybpXmfnd_83Asxge4jKaC5mvh9Mzp6cOSJI.jar
Jun 13, 2020 6:45:39 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.23.0-SNAPSHOT-tests-iekJjYX4cyIFIrjFE6SlwyomOVxilD-3hQ6wmwi8rEQ.jar
Jun 13, 2020 6:45:39 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-5ydGTc9r5adNZ1xIkyj9p9Ug_82G8AHz4ztuwrvEqCo.jar
Jun 13, 2020 6:45:39 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT-R4O286TwaMQLMPNnybXU0gM1-eeuu-zobn2t3qUjgMA.jar
Jun 13, 2020 6:45:39 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-unshaded-kn-8PQDX76jPrhnTTQptz8lrXg_qV38YH4w4GHKZyMU.jar
Jun 13, 2020 6:45:39 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-tests-OAEr6R9aC9S5iQoxggwlf3isYHJ4gX94dY3WjfyRZiY.jar
Jun 13, 2020 6:45:40 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-B6wYOsdhk_PMYp8-MVjgU7CmsxXCsW0cBjRvkSowapM.jar
Jun 13, 2020 6:45:40 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.23.0-SNAPSHOT-p74IBbDIw2dluFMC3krPbnZamly7CkreO3Dy-LMewl4.jar
Jun 13, 2020 6:45:40 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.23.0-SNAPSHOT-rElAI12SaswZZSVgaorJV1__vpbz_kIomQODTBxe9YI.jar
Jun 13, 2020 6:45:40 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.23.0-SNAPSHOT-Aows1F9mBcY7UfhVlYIhFlSDQ1PzeppXiy6EXTQRr1M.jar
Jun 13, 2020 6:45:41 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-B6wYOsdhk_PMYp8-MVjgU7CmsxXCsW0cBjRvkSowapM.jar&uploadType=resumable&upload_id=AAANsUl4kX_Gq4lkdkkJ2Of58nUDcjCDQZi2_2-KR3EJrK-C330XV9r96Aj3V-WRaDAHeQUMSpXPQqUBcTYPBuClyK0.
Jun 13, 2020 6:45:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar>
java.io.IOException: Upload failed for 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-B6wYOsdhk_PMYp8-MVjgU7CmsxXCsW0cBjRvkSowapM.jar'
at com.google.cloud.hadoop.util.BaseAbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(BaseAbstractGoogleAsyncWriteChannel.java:297)
at com.google.cloud.hadoop.util.BaseAbstractGoogleAsyncWriteChannel.close(BaseAbstractGoogleAsyncWriteChannel.java:214)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:221)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:168)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:152)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:136)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Jun 13, 2020 6:45:45 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-B6wYOsdhk_PMYp8-MVjgU7CmsxXCsW0cBjRvkSowapM.jar
Jun 13, 2020 6:45:46 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 31 files newly uploaded in 6 seconds
Jun 13, 2020 6:45:46 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Jun 13, 2020 6:45:46 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Jun 13, 2020 6:45:46 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Jun 13, 2020 6:45:46 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Jun 13, 2020 6:45:46 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Jun 13, 2020 6:45:46 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <91368 bytes, hash f514856daec965d14f38c7cb7808f3dbd7ce7eb3075544e56965e06bd2a27766> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-9RSFba7JZdFPOMfLeAjz29fOfrMHVUTlaWXga9Kid2Y.pb
Jun 13, 2020 6:45:47 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.23.0-SNAPSHOT
Jun 13, 2020 6:45:48 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-06-12_23_45_47-12707786348407345333?project=apache-beam-testing
Jun 13, 2020 6:45:48 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-06-12_23_45_47-12707786348407345333
Jun 13, 2020 6:45:48 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-06-12_23_45_47-12707786348407345333
Jun 13, 2020 6:45:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-06-13T06:45:47.297Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Jun 13, 2020 6:45:57 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-13T06:45:56.047Z: Worker configuration: n1-standard-1 in us-central1-a.
Jun 13, 2020 6:45:57 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-13T06:45:56.991Z: Expanding CoGroupByKey operations into optimizable parts.
Jun 13, 2020 6:45:57 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-13T06:45:57.044Z: Expanding GroupByKey operations into optimizable parts.
Jun 13, 2020 6:45:57 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-13T06:45:57.085Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jun 13, 2020 6:45:57 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-13T06:45:57.181Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jun 13, 2020 6:45:57 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-13T06:45:57.207Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Jun 13, 2020 6:45:57 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-13T06:45:57.244Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Jun 13, 2020 6:45:57 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-13T06:45:57.285Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Jun 13, 2020 6:46:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-13T06:45:57.753Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Jun 13, 2020 6:46:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-13T06:45:57.849Z: Starting 5 workers in us-central1-a...
Jun 13, 2020 6:46:06 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-06-13T06:46:05.395Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jun 13, 2020 6:46:24 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-13T06:46:24.563Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running stage(s).
Jun 13, 2020 6:46:24 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-13T06:46:24.608Z: Resized worker pool to 4, though goal was 5. This could be a quota issue.
Jun 13, 2020 6:46:30 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-13T06:46:29.961Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Jun 13, 2020 6:46:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-13T06:46:49.534Z: Workers have started successfully.
Jun 13, 2020 6:46:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-13T06:46:49.574Z: Workers have started successfully.
Jun 13, 2020 6:47:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-13T06:47:23.233Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Jun 13, 2020 6:47:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-13T06:47:23.447Z: Cleaning up.
Jun 13, 2020 6:47:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-13T06:47:23.548Z: Stopping worker pool...
Jun 13, 2020 6:49:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-13T06:49:49.674Z: Autoscaling: Resized worker pool from 5 to 0.
Jun 13, 2020 6:49:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-13T06:49:49.734Z: Worker pool stopped.
Jun 13, 2020 6:49:54 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-06-12_23_45_47-12707786348407345333 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 45fcc321-5ed5-434b-949b-8898c3875b47 and timestamp: 2020-06-13T06:49:54.721000000Z:
Metric: Value:
read_time 16.265
fields_read 4375276.0
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Jun 13, 2020 6:49:55 AM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 2 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.022 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.122 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 2,5,main]) completed. Took 4 mins 26.439 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 32s
104 actionable tasks: 70 executed, 34 from cache
Publishing build scan...
https://gradle.com/s/i2pymm6ncoq7y
Stopped 1 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #620
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/620/display/redirect?page=changes>
Changes:
[github] [BEAM-10250]: Update io_matrix with new SplunkIO (#11998)
[github] Remove a hack used to retrieve schematized data from HL7v2 messages, …
------------------------------------------
[...truncated 294.18 KB...]
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Jun 13, 2020 1:38:40 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Jun 13, 2020 1:38:40 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery writeDisposition is set to: WRITE_EMPTY
Jun 13, 2020 1:38:40 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Jun 13, 2020 1:38:40 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Jun 13, 2020 1:38:40 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery writeDisposition is set to: WRITE_EMPTY
Jun 13, 2020 1:38:40 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Jun 13, 2020 1:38:41 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Jun 13, 2020 1:38:41 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Jun 13, 2020 1:38:41 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Jun 13, 2020 1:38:44 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 209 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Jun 13, 2020 1:38:44 AM org.apache.beam.runners.dataflow.util.PackageUtil$PackageAttributes forFileToStage
INFO: Staging custom dataflow-worker.jar as beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-v-T_by8a5klG9As5k33Ydb5VgxqEwzkS0HaAcg10tIQ.jar
Jun 13, 2020 1:38:44 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-tests-IrrwVT0KhwwqLML8HHhaYh-zPNYv2a-s5GBdb-Xerpg.jar
Jun 13, 2020 1:38:44 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tests-vWQwZ6be58U6wnM-wZdZwJODQDCTQx_Btwrkg7rmTsk.jar
Jun 13, 2020 1:38:44 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-zKNBIViBtc3xUXKojPMk9Q-xEJLBhEAs0gY_U7pRMMg.jar
Jun 13, 2020 1:38:44 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT-RipijtvdyRBHWPKfsae-BGer5nx1Zca-xLprvb53wB8.jar
Jun 13, 2020 1:38:44 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-v-T_by8a5klG9As5k33Ydb5VgxqEwzkS0HaAcg10tIQ.jar
Jun 13, 2020 1:38:44 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /tmp/test3929748029450737287.zip to gs://temp-storage-for-perf-tests/loadtests/staging/test-VUhy5wsZzqXQAb9RcMpFrpdU5q8wHfV7I5VhHKbDN2I.jar
Jun 13, 2020 1:38:44 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-AceCKz5jyC1wNjr5LQgLCR8W3NJEJjeZ1hqIHyZ4DXs.jar
Jun 13, 2020 1:38:44 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-tests-EtGkCu1j7d4MF5Y6zVXAjuYSYUW09qNAutSJB_2zLx4.jar
Jun 13, 2020 1:38:44 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.23.0-SNAPSHOT-tests-bhJuA2joVjRvxAJA2X-x7s099v7b3Ir2M_biYpfZXCE.jar
Jun 13, 2020 1:38:44 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-B83GHKOTBkbQcvC4Lp9B1TN0jzHzgeJpKY1oYOXISBQ.jar
Jun 13, 2020 1:38:44 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-v3lxeisRjIpkBtQoKH6RLK64DI90FT2GSLjRBWYAgho.jar
Jun 13, 2020 1:38:44 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.23.0-SNAPSHOT-KKv-ZdlYnZuu89hJgbPISWxZXSlB9qA3BZ1MIT1SVS8.jar
Jun 13, 2020 1:38:44 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-E0MMTnGGV4_YXpdrRNKpAOo2r7rweV9xH6M35czxyOw.jar
Jun 13, 2020 1:38:44 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-Pj6yZSZRjoY-B0iHJqgAiPubqU40omyU47xcc8WUe_8.jar
Jun 13, 2020 1:38:44 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-tests-HuqLxm76M3v140GnmJI9DPjjDhOkGnJxmoAiY49gG8w.jar
Jun 13, 2020 1:38:44 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-q8ZOg4Qc6p309O4ZQphfNNhupe-OUlBJFGjdPA3fp80.jar
Jun 13, 2020 1:38:44 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.23.0-SNAPSHOT-qVJQ4hkOJGqaixF6bVhm2YASNFuE7w7kgxBE1-E2uLc.jar
Jun 13, 2020 1:38:44 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-fn-execution-2.23.0-SNAPSHOT-ByBhkA9DJg88UHwS-G6ibHL6jtpaKXcdnljRccXultk.jar
Jun 13, 2020 1:38:44 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-tests-oVgxOCd7et-kJD2JX4R63r6C5MOpZCGEFA9FRUbC4fc.jar
Jun 13, 2020 1:38:44 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.23.0-SNAPSHOT-uRLyb-1IqWczFMaDGpWJEVvHq9cxMALaXv-Lwk6AArY.jar
Jun 13, 2020 1:38:44 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT-9icJJupJ8QODQFm9XlKSIlpWfaHkOuCNGcyKmV5QJJ8.jar
Jun 13, 2020 1:38:44 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-tests-KcYOXOEcXDRygu4fMPZ1CCN1LgCqe1zMq7keEbWwSho.jar
Jun 13, 2020 1:38:44 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-unshaded-apbCufwL8BdkaXgmeboacy0bacvzIbyx0ZcMsNFBSFo.jar
Jun 13, 2020 1:38:44 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.23.0-SNAPSHOT-SrEOYeFoCdi6ir1OgwVellTpfpg21lrrH5IuPl2Sn98.jar
Jun 13, 2020 1:38:44 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-tests-bfi_oiRQNlLSlJ7ZwQwKFo777GDs8d6gXUzLW50nAUw.jar
Jun 13, 2020 1:38:44 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.23.0-SNAPSHOT-wgiBCTRR4_B63_xCFQ1T2PmWkhTxc7AYdGNoqTy6LJU.jar
Jun 13, 2020 1:38:44 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/java-fn-execution/build/libs/beam-runners-java-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-java-fn-execution-2.23.0-SNAPSHOT-JgwKVWl8zGkZn1MbrUhK67TylJsGCnrKzzFpMcwI_B8.jar
Jun 13, 2020 1:38:44 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-v-T_by8a5klG9As5k33Ydb5VgxqEwzkS0HaAcg10tIQ.jar
Jun 13, 2020 1:38:44 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.23.0-SNAPSHOT-IUGKqVm-kGZRzFphDydxOWrKeJHfv_s3U-_ZK2fGvgQ.jar
Jun 13, 2020 1:38:44 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.23.0-SNAPSHOT-_t5jo3IjgKLDoJOaEgxMVTbw-l8TIKWtkCg9tutFYxI.jar
Jun 13, 2020 1:38:44 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.23.0-SNAPSHOT-die9yxHZyVD1Tn4ZeZ0ikqq58GfilCMCnK_G_wQJYoU.jar
Jun 13, 2020 1:38:45 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-v-T_by8a5klG9As5k33Ydb5VgxqEwzkS0HaAcg10tIQ.jar&uploadType=resumable&upload_id=AAANsUkagqRPSwqJ4aAlXarOdJpf1x_XWL9RYDfEa_FU4qz5qay5LT4gIZNu0jLjIv3qJINm3bU2x9NEfIreVxWhaYw.
Jun 13, 2020 1:38:45 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar>
java.io.IOException: Upload failed for 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-v-T_by8a5klG9As5k33Ydb5VgxqEwzkS0HaAcg10tIQ.jar'
at com.google.cloud.hadoop.util.BaseAbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(BaseAbstractGoogleAsyncWriteChannel.java:297)
at com.google.cloud.hadoop.util.BaseAbstractGoogleAsyncWriteChannel.close(BaseAbstractGoogleAsyncWriteChannel.java:214)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:221)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:168)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:152)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:136)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Jun 13, 2020 1:38:49 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-v-T_by8a5klG9As5k33Ydb5VgxqEwzkS0HaAcg10tIQ.jar
Jun 13, 2020 1:38:50 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 31 files newly uploaded in 6 seconds
Jun 13, 2020 1:38:50 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Jun 13, 2020 1:38:50 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Jun 13, 2020 1:38:50 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Jun 13, 2020 1:38:50 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Jun 13, 2020 1:38:50 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Jun 13, 2020 1:38:50 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <91368 bytes, hash 1c6e79b982d40276e9a4f37887f255de0e7ad7e0577991909263db283b64a7b8> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-HG55uYLUAnbppPN4h_JV3g561-BXeZGQkmPbKDtkp7g.pb
Jun 13, 2020 1:38:51 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.23.0-SNAPSHOT
Jun 13, 2020 1:38:52 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-06-12_18_38_51-15031012651935367696?project=apache-beam-testing
Jun 13, 2020 1:38:52 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-06-12_18_38_51-15031012651935367696
Jun 13, 2020 1:38:52 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-06-12_18_38_51-15031012651935367696
Jun 13, 2020 1:38:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-06-13T01:38:51.272Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Jun 13, 2020 1:38:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-13T01:38:58.784Z: Worker configuration: n1-standard-1 in us-central1-a.
Jun 13, 2020 1:39:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-13T01:38:59.726Z: Expanding CoGroupByKey operations into optimizable parts.
Jun 13, 2020 1:39:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-13T01:38:59.769Z: Expanding GroupByKey operations into optimizable parts.
Jun 13, 2020 1:39:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-13T01:38:59.798Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jun 13, 2020 1:39:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-13T01:38:59.879Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jun 13, 2020 1:39:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-13T01:38:59.904Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Jun 13, 2020 1:39:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-13T01:38:59.928Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Jun 13, 2020 1:39:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-13T01:38:59.951Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Jun 13, 2020 1:39:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-13T01:39:00.347Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Jun 13, 2020 1:39:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-13T01:39:00.415Z: Starting 5 workers in us-central1-a...
Jun 13, 2020 1:39:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-06-13T01:39:11.382Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jun 13, 2020 1:39:29 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-13T01:39:28.215Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running stage(s).
Jun 13, 2020 1:39:29 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-13T01:39:28.249Z: Resized worker pool to 4, though goal was 5. This could be a quota issue.
Jun 13, 2020 1:39:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-13T01:39:33.617Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Jun 13, 2020 1:39:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-13T01:39:51.858Z: Workers have started successfully.
Jun 13, 2020 1:39:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-13T01:39:51.881Z: Workers have started successfully.
Jun 13, 2020 1:40:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-13T01:40:23.006Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Jun 13, 2020 1:40:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-13T01:40:23.285Z: Cleaning up.
Jun 13, 2020 1:40:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-13T01:40:23.380Z: Stopping worker pool...
Jun 13, 2020 1:42:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-13T01:42:15.270Z: Autoscaling: Resized worker pool from 5 to 0.
Jun 13, 2020 1:42:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-13T01:42:15.321Z: Worker pool stopped.
Jun 13, 2020 1:42:21 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-06-12_18_38_51-15031012651935367696 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 35943560-acb6-47cb-a194-50e420b54a73 and timestamp: 2020-06-13T01:42:21.678000000Z:
Metric: Value:
read_time 13.203
fields_read 4375276.0
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Jun 13, 2020 1:42:22 AM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 2 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.026 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.035 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 4,5,main]) completed. Took 3 mins 50.701 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 6s
104 actionable tasks: 70 executed, 34 from cache
Publishing build scan...
https://gradle.com/s/lcd3ixfiixqx6
Stopped 1 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #619
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/619/display/redirect?page=changes>
Changes:
[github] [BEAM-9615] Add String UTF8 coder. (#11989)
[github] [BEAM-7163] Correcting godoc for passert.Sum (#11999)
------------------------------------------
[...truncated 294.61 KB...]
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:164)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Jun 12, 2020 6:45:34 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Jun 12, 2020 6:45:34 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery writeDisposition is set to: WRITE_EMPTY
Jun 12, 2020 6:45:34 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Jun 12, 2020 6:45:34 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Jun 12, 2020 6:45:34 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery writeDisposition is set to: WRITE_EMPTY
Jun 12, 2020 6:45:34 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Jun 12, 2020 6:45:34 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Jun 12, 2020 6:45:34 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Jun 12, 2020 6:45:35 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Jun 12, 2020 6:45:37 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 209 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Jun 12, 2020 6:45:37 PM org.apache.beam.runners.dataflow.util.PackageUtil$PackageAttributes forFileToStage
INFO: Staging custom dataflow-worker.jar as beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-xj8PseItlq8Hid-jRExQY6QsRD6_9lW7iT3J7_zvzcg.jar
Jun 12, 2020 6:45:37 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.23.0-SNAPSHOT-JJwXtGwfNr1mT0zih4oU2j8FrawqatrX2aekqmKLl8A.jar
Jun 12, 2020 6:45:37 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-EX7O01x7GfNQGUGKlZsBd7mw62SkhGNKv58i7bS6Wb0.jar
Jun 12, 2020 6:45:37 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-E8m2aDVDeYikTDJS4i83rm7iNAsKjLFsXKJsJ9x0uu8.jar
Jun 12, 2020 6:45:37 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-xj8PseItlq8Hid-jRExQY6QsRD6_9lW7iT3J7_zvzcg.jar
Jun 12, 2020 6:45:37 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/java-fn-execution/build/libs/beam-runners-java-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-java-fn-execution-2.23.0-SNAPSHOT-SlBqGjhJJ5wEfB-Eu7OcjcLKd90zxaTJ5-FjtqnCG2w.jar
Jun 12, 2020 6:45:37 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.23.0-SNAPSHOT-s8UE6VxTNdXcU8xfnRWpelrBhdfRRXgGsbrNCPZiRhE.jar
Jun 12, 2020 6:45:37 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-tests-dHL2sCn8GoWWfJx24hErl_3TyIID6a7J-5xDjYG-Huc.jar
Jun 12, 2020 6:45:37 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-5sjC50MkIyDZYhaCeC0wb8CIUZ5F5DnxQqF7StC3roI.jar
Jun 12, 2020 6:45:37 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.23.0-SNAPSHOT-cj3Pv1DGtvLz3c7F1zEdB92ojIOJ-AHa1gmrrMm4bwc.jar
Jun 12, 2020 6:45:37 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-wIqj3w23VTrIIRYaiRHbFIKZ9ye3vZP8Nif-EIHM92Y.jar
Jun 12, 2020 6:45:37 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.23.0-SNAPSHOT-KgIos4LpDW0cw2WNtSR1SohWorvvHSN-dKo7f6x9MLI.jar
Jun 12, 2020 6:45:37 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-4bYYMbutGOCt-Abm-xazOar4vz5ELtY4SyhnQd-Na60.jar
Jun 12, 2020 6:45:37 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.23.0-SNAPSHOT-OFNRDiyAnehpjFOl0q1nycMJOKXzC0wkd9VxjTvGsdI.jar
Jun 12, 2020 6:45:37 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-w22Tbq_PE6rqZLOHSEbU380Uq3ey5GJDwGBmK6bf8Ww.jar
Jun 12, 2020 6:45:37 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.23.0-SNAPSHOT-tests-c5txzFSAfw4Jkot3UZ1eAUApz3wesmwEIEIRyEYMRso.jar
Jun 12, 2020 6:45:37 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT-ie0Jab7MWKCcqo-ek0zcBREVWFj1HumnoUCq0sud2Hs.jar
Jun 12, 2020 6:45:37 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-tests-dkmCy-dKg6r4ivomMwamsXFesXuWgUJBvhTk1i9tdrk.jar
Jun 12, 2020 6:45:37 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-tests-Uo4k5S-0ZI0oG-cifBD4lR15XVSWFDLLDTkYHT1BpyA.jar
Jun 12, 2020 6:45:37 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT-wEiXCZQZ9KqK6Hko1ASydI0V7Ez4mj84Aq6Po56HmLI.jar
Jun 12, 2020 6:45:37 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /tmp/test6546454324727212326.zip to gs://temp-storage-for-perf-tests/loadtests/staging/test-otQR9-Mn_WPe6QHcnzcCSTsyxHPB8YK8pA-ckJVje3A.jar
Jun 12, 2020 6:45:37 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-tests-KNVTNy7ghH0rU-a_Tguwp3fmWyRml6_Iox1wkKAGeIM.jar
Jun 12, 2020 6:45:37 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-tests-pwlHymdXAD8YHHJu673_vefsKOCh4jCt5p46mbEZDrM.jar
Jun 12, 2020 6:45:37 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-tests-E3Qpm89RzFA4RqDyiHaZaab-dWNVVC_RieqXxdTNd5Q.jar
Jun 12, 2020 6:45:37 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-fn-execution-2.23.0-SNAPSHOT-EoGlfPH7vYgzFoYVxK1SfTB4o0PFsQv62RnB7LXE7GM.jar
Jun 12, 2020 6:45:37 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-s090Vf51rJxhbMAHltDbpdNdMvB3RJUSFiDMCK9_SNc.jar
Jun 12, 2020 6:45:37 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tests-16jHBN7JazawZMtxTGHqfppis8CvBRhfZQyHt7xCw5U.jar
Jun 12, 2020 6:45:37 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-unshaded-xy6292CZc_HmsTtSwnYWianQoXQx6NsftCDaxtYlgAg.jar
Jun 12, 2020 6:45:37 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-xj8PseItlq8Hid-jRExQY6QsRD6_9lW7iT3J7_zvzcg.jar
Jun 12, 2020 6:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.23.0-SNAPSHOT-hrqETvU0LJlNbO6M21nfFpUzhANRIVbazUgZQJzBJMY.jar
Jun 12, 2020 6:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.23.0-SNAPSHOT-FoHhjfIsEsYrFFLOKl4IwBdjpnS6daXRVNu6OCk-3Zc.jar
Jun 12, 2020 6:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.23.0-SNAPSHOT-s_DJIVGm1cF82ksvc_LhpVGxpUVOwssFrEJ7ybB4WjY.jar
Jun 12, 2020 6:45:38 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-xj8PseItlq8Hid-jRExQY6QsRD6_9lW7iT3J7_zvzcg.jar&uploadType=resumable&upload_id=AAANsUkS-C9qVTO2c83hZ8lJPnF_LSrtdHzsR8Ep7DtoXbK9vX6z5ufn6UmLGiZLZXxAphfgffOPB4jWLCreNDWhKxE.
Jun 12, 2020 6:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar>
java.io.IOException: Upload failed for 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-xj8PseItlq8Hid-jRExQY6QsRD6_9lW7iT3J7_zvzcg.jar'
at com.google.cloud.hadoop.util.BaseAbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(BaseAbstractGoogleAsyncWriteChannel.java:297)
at com.google.cloud.hadoop.util.BaseAbstractGoogleAsyncWriteChannel.close(BaseAbstractGoogleAsyncWriteChannel.java:214)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:221)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:168)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:152)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:136)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Jun 12, 2020 6:45:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-xj8PseItlq8Hid-jRExQY6QsRD6_9lW7iT3J7_zvzcg.jar
Jun 12, 2020 6:45:45 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 31 files newly uploaded in 8 seconds
Jun 12, 2020 6:45:45 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Jun 12, 2020 6:45:45 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Jun 12, 2020 6:45:45 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Jun 12, 2020 6:45:45 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Jun 12, 2020 6:45:45 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Jun 12, 2020 6:45:45 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <91368 bytes, hash 27ef18bb7f75be3958ea57864c6ec2fd66c83a98c893dfdae75b8e52eeb18b6c> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-J-8Yu391vjlY6leGTG7C_WbIOpjIk9_a51uOUu6xi2w.pb
Jun 12, 2020 6:45:46 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.23.0-SNAPSHOT
Jun 12, 2020 6:45:47 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-06-12_11_45_46-14993940962187651354?project=apache-beam-testing
Jun 12, 2020 6:45:47 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-06-12_11_45_46-14993940962187651354
Jun 12, 2020 6:45:47 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-06-12_11_45_46-14993940962187651354
Jun 12, 2020 6:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-06-12T18:45:46.222Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Jun 12, 2020 6:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-12T18:45:53.895Z: Worker configuration: n1-standard-1 in us-central1-a.
Jun 12, 2020 6:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-12T18:45:54.953Z: Expanding CoGroupByKey operations into optimizable parts.
Jun 12, 2020 6:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-12T18:45:55.001Z: Expanding GroupByKey operations into optimizable parts.
Jun 12, 2020 6:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-12T18:45:55.030Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jun 12, 2020 6:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-12T18:45:55.115Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jun 12, 2020 6:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-12T18:45:55.147Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Jun 12, 2020 6:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-12T18:45:55.180Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Jun 12, 2020 6:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-12T18:45:55.212Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Jun 12, 2020 6:45:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-12T18:45:55.737Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Jun 12, 2020 6:45:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-12T18:45:55.816Z: Starting 5 workers in us-central1-a...
Jun 12, 2020 6:46:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-06-12T18:46:29.238Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jun 12, 2020 6:46:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-12T18:46:32.710Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Jun 12, 2020 6:46:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-12T18:46:53.250Z: Workers have started successfully.
Jun 12, 2020 6:46:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-12T18:46:53.277Z: Workers have started successfully.
Jun 12, 2020 6:47:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-12T18:47:24.249Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Jun 12, 2020 6:47:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-12T18:47:24.469Z: Cleaning up.
Jun 12, 2020 6:47:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-12T18:47:24.560Z: Stopping worker pool...
Jun 12, 2020 6:49:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-12T18:49:40.682Z: Autoscaling: Resized worker pool from 5 to 0.
Jun 12, 2020 6:49:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-12T18:49:40.738Z: Worker pool stopped.
Jun 12, 2020 6:49:46 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-06-12_11_45_46-14993940962187651354 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 549685b1-a3e1-4a05-8df1-7ded62cdd6b8 and timestamp: 2020-06-12T18:49:46.389000000Z:
Metric: Value:
fields_read 4375276.0
read_time 14.265
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Jun 12, 2020 6:49:46 PM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 2 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.024 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.03 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 7,5,main]) completed. Took 4 mins 20.716 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 27s
104 actionable tasks: 70 executed, 34 from cache
Publishing build scan...
https://gradle.com/s/wshumhh2ejrbg
Stopped 1 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #618
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/618/display/redirect>
Changes:
------------------------------------------
[...truncated 294.38 KB...]
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:164)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Jun 12, 2020 12:45:25 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Jun 12, 2020 12:45:25 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery writeDisposition is set to: WRITE_EMPTY
Jun 12, 2020 12:45:25 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Jun 12, 2020 12:45:25 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Jun 12, 2020 12:45:25 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery writeDisposition is set to: WRITE_EMPTY
Jun 12, 2020 12:45:25 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Jun 12, 2020 12:45:25 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Jun 12, 2020 12:45:25 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Jun 12, 2020 12:45:26 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Jun 12, 2020 12:45:28 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 209 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Jun 12, 2020 12:45:28 PM org.apache.beam.runners.dataflow.util.PackageUtil$PackageAttributes forFileToStage
INFO: Staging custom dataflow-worker.jar as beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-rV__AWsWnHb3fcK_K-Q7jLIMiB_Oq4_01ZjKta3Vckk.jar
Jun 12, 2020 12:45:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.23.0-SNAPSHOT-7lO-fVEmK7hoVruUSNQKZkPp7CPRby4LfwUYUsnfG_E.jar
Jun 12, 2020 12:45:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-tests-OGN8r9ieMGOBWnHCoWLydAYa_MWd-Y4G-DjPkygpsAM.jar
Jun 12, 2020 12:45:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-m-VXmhI0Zmfz2Y4Q6sWTiohwinGSsHUagM0oJgdnbp4.jar
Jun 12, 2020 12:45:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tests-kWMtfoIu3zMumKwtqW8tvq6IX3UqEQAxbEfcZBULgU8.jar
Jun 12, 2020 12:45:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.23.0-SNAPSHOT-9vXpukIVT1SBeQgyOlq9_La7Zgr0ZgwQVSFXeGvIttQ.jar
Jun 12, 2020 12:45:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /tmp/test5010241258163377440.zip to gs://temp-storage-for-perf-tests/loadtests/staging/test-C_sWBfVAoQ469cUwuaC3VxHkMWMnJ3QUok8S6RWsVfk.jar
Jun 12, 2020 12:45:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT-kKuooHt_EHAoVzYROrI87MD9qnjxxfXgJZ9zKRGomJc.jar
Jun 12, 2020 12:45:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-tests-MoI0CZksUfCE0NPWseijaEFIHIMDxcsFNEUanNDsJMk.jar
Jun 12, 2020 12:45:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.23.0-SNAPSHOT-yFitZoMT9QVz65ty0VYBsgT9F0JznwcFE-4JCBaa1bQ.jar
Jun 12, 2020 12:45:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-BHVS-mxxcl9llxTXw8gGtrkyhCP4Zdp_IjM9Rp870UM.jar
Jun 12, 2020 12:45:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.23.0-SNAPSHOT-2QkNrwno3-bkooX0YNRfa_Jhfljc_kucNz78jse99W0.jar
Jun 12, 2020 12:45:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/java-fn-execution/build/libs/beam-runners-java-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-java-fn-execution-2.23.0-SNAPSHOT-ozc8eEingc-U947IjLPZ5duQ_rpUEVWaeHAaIFlpWhI.jar
Jun 12, 2020 12:45:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-tests-Skt_tyH9AhfQM6ZDwm5i7w8HXUcUPCgtM2fMO0D11qs.jar
Jun 12, 2020 12:45:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-0Q6FQnS0AohtXc7_VcPIZMHWv54zb1y1HYsRl-6LPXo.jar
Jun 12, 2020 12:45:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-tests-GrLpnL4M9kmjiRkYEBAh2pjWPQ7Bwgvba35XkT-zFFw.jar
Jun 12, 2020 12:45:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-gCrOLAP341FYnxCsxWtskcAJ3GAMUIWUTg94Yjdh9qE.jar
Jun 12, 2020 12:45:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-fn-execution-2.23.0-SNAPSHOT-YWz8Sj-SH8vkaSy03K1grD3lGITnI6YDJbBxozax9bA.jar
Jun 12, 2020 12:45:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.23.0-SNAPSHOT-tests-Mp1QN3I--snZV8zI6UjlTDnzuayVbnMoG_iIdtBFcSs.jar
Jun 12, 2020 12:45:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tgRHZ2xmDJMDlrPO0_zdScmHHoEyMkTGh8XDO-H7PHo.jar
Jun 12, 2020 12:45:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-vAydijOOTXpeYgebOJg-S_-PZgC_vsKYUv9m_DgIjDU.jar
Jun 12, 2020 12:45:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.23.0-SNAPSHOT-qCAX6thbXd9qNlFXQ0d5RppE-utdXHusXD8mxVhaVkc.jar
Jun 12, 2020 12:45:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-rV__AWsWnHb3fcK_K-Q7jLIMiB_Oq4_01ZjKta3Vckk.jar
Jun 12, 2020 12:45:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-tests-eIRx1yYuPexZo8TiZqPDWV28cbjggxL295BZ2Mjlvds.jar
Jun 12, 2020 12:45:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-3moWgFIqRa9K1nI3QMc9-E_SzGzG9zTRxzsnkgZTqX8.jar
Jun 12, 2020 12:45:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-tests-dCevMexBmxLJJz1tBRwzqHK-kvPDBTeTHLaNy7vZEZo.jar
Jun 12, 2020 12:45:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT-wrISbtkBOl14CTLIIwxfgAHCOcJhn5DUk3Oif8oSEE4.jar
Jun 12, 2020 12:45:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-unshaded-PHnkuhFW10-8OCzObY3I9b62u-eCuFV-ZU7H8WZY0iQ.jar
Jun 12, 2020 12:45:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-rV__AWsWnHb3fcK_K-Q7jLIMiB_Oq4_01ZjKta3Vckk.jar
Jun 12, 2020 12:45:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.23.0-SNAPSHOT-zmUsXWGdEMxApTQP2rmmDwfcACJZYgZSxTuqSg7U9pw.jar
Jun 12, 2020 12:45:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.23.0-SNAPSHOT-uJbBi-wZf5i5CmPGuLmY7YuJSQZMU3uWxBtaFGz9dRk.jar
Jun 12, 2020 12:45:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.23.0-SNAPSHOT-18Gen5cBB41m6hJIPSrjMxXP3apIHferwLVT86yVMTs.jar
Jun 12, 2020 12:45:29 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-rV__AWsWnHb3fcK_K-Q7jLIMiB_Oq4_01ZjKta3Vckk.jar&uploadType=resumable&upload_id=AAANsUmW7iAVCfzCpY2SZn6VoMmPuowRrKKCIrPXvn8KY3ng1N3rBFjswzWivSqp-2BAwq_UoaGkYnKC6eEJPaTJOzx3lOoyiw.
Jun 12, 2020 12:45:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar>
java.io.IOException: Upload failed for 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-rV__AWsWnHb3fcK_K-Q7jLIMiB_Oq4_01ZjKta3Vckk.jar'
at com.google.cloud.hadoop.util.BaseAbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(BaseAbstractGoogleAsyncWriteChannel.java:297)
at com.google.cloud.hadoop.util.BaseAbstractGoogleAsyncWriteChannel.close(BaseAbstractGoogleAsyncWriteChannel.java:214)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:221)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:168)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:152)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:136)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Jun 12, 2020 12:45:33 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-rV__AWsWnHb3fcK_K-Q7jLIMiB_Oq4_01ZjKta3Vckk.jar
Jun 12, 2020 12:45:34 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 31 files newly uploaded in 6 seconds
Jun 12, 2020 12:45:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Jun 12, 2020 12:45:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Jun 12, 2020 12:45:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Jun 12, 2020 12:45:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Jun 12, 2020 12:45:34 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Jun 12, 2020 12:45:34 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <91368 bytes, hash 8b136c497d5ee8a1955d90c59c75cab1bcf76b0fd3febdd36bf8f7ded9fabe74> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-ixNsSX1e6KGVXZDFnHXKsbz3aw_T_r3Ta_j33tn6vnQ.pb
Jun 12, 2020 12:45:35 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.23.0-SNAPSHOT
Jun 12, 2020 12:45:36 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-06-12_05_45_35-2883435218027491533?project=apache-beam-testing
Jun 12, 2020 12:45:36 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-06-12_05_45_35-2883435218027491533
Jun 12, 2020 12:45:36 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-06-12_05_45_35-2883435218027491533
Jun 12, 2020 12:45:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-06-12T12:45:35.408Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Jun 12, 2020 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-12T12:45:43.919Z: Worker configuration: n1-standard-1 in us-central1-a.
Jun 12, 2020 12:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-12T12:45:44.760Z: Expanding CoGroupByKey operations into optimizable parts.
Jun 12, 2020 12:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-12T12:45:44.904Z: Expanding GroupByKey operations into optimizable parts.
Jun 12, 2020 12:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-12T12:45:44.937Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jun 12, 2020 12:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-12T12:45:45.019Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jun 12, 2020 12:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-12T12:45:45.050Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Jun 12, 2020 12:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-12T12:45:45.087Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Jun 12, 2020 12:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-12T12:45:45.119Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Jun 12, 2020 12:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-12T12:45:45.454Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Jun 12, 2020 12:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-12T12:45:45.531Z: Starting 5 workers in us-central1-a...
Jun 12, 2020 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-06-12T12:45:51.088Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jun 12, 2020 12:46:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-12T12:46:12.416Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Jun 12, 2020 12:46:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-12T12:46:31.302Z: Workers have started successfully.
Jun 12, 2020 12:46:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-12T12:46:31.334Z: Workers have started successfully.
Jun 12, 2020 12:47:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-12T12:47:04.224Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Jun 12, 2020 12:47:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-12T12:47:04.431Z: Cleaning up.
Jun 12, 2020 12:47:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-12T12:47:04.517Z: Stopping worker pool...
Jun 12, 2020 12:48:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-12T12:48:52.882Z: Autoscaling: Resized worker pool from 5 to 0.
Jun 12, 2020 12:48:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-12T12:48:52.928Z: Worker pool stopped.
Jun 12, 2020 12:48:59 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-06-12_05_45_35-2883435218027491533 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 043234b0-4626-49ee-a49b-68354cd132e4 and timestamp: 2020-06-12T12:48:59.332000000Z:
Metric: Value:
read_time 15.384
fields_read 4375276.0
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Jun 12, 2020 12:48:59 PM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.038 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.049 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 11,5,main]) completed. Took 3 mins 43.563 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 40s
104 actionable tasks: 68 executed, 36 from cache
Publishing build scan...
https://gradle.com/s/szv4ppg4wzjdy
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #617
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/617/display/redirect?page=changes>
Changes:
[github] Support HOP and SESSION as TVF (#11868)
------------------------------------------
[...truncated 298.09 KB...]
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Jun 12, 2020 6:45:40 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Jun 12, 2020 6:45:40 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery writeDisposition is set to: WRITE_EMPTY
Jun 12, 2020 6:45:40 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Jun 12, 2020 6:45:40 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Jun 12, 2020 6:45:40 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery writeDisposition is set to: WRITE_EMPTY
Jun 12, 2020 6:45:40 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Jun 12, 2020 6:45:40 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Jun 12, 2020 6:45:40 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Jun 12, 2020 6:45:41 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Jun 12, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 209 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Jun 12, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.PackageUtil$PackageAttributes forFileToStage
INFO: Staging custom dataflow-worker.jar as beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-vbYCHnZwFhdxb4RttM_JOHqvF4AyYh7PFMNoQfdI228.jar
Jun 12, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.23.0-SNAPSHOT-RlgyXW5WDgGU9Jy5Sd5UNW1W3-vA6uTh1q8-iZX62tI.jar
Jun 12, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-tests-NmpNplWuNfoUwxSNSxnZqQ9f-WZkOm7Y4Z9cVa3pnZw.jar
Jun 12, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-tests-wUijjjeP7uIhcuzAxJJSa7J6Sx6tpqtj63YcQHUk_AY.jar
Jun 12, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /tmp/test2882757056700669998.zip to gs://temp-storage-for-perf-tests/loadtests/staging/test-m6gjqo4K-yLGLqKGzUulOX_QwUDjwYZk11_JTGK4nEQ.jar
Jun 12, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.23.0-SNAPSHOT-HrGILlizQVaJmm7N9Lhf3OPtgMyOhvW0FNimhx-KkqQ.jar
Jun 12, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-DVE2JDIik2ep004PkBkmeFOuwmTZqByjUeZTF2YjoCc.jar
Jun 12, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-ZBU0-z-baTG_KaRB97h6Mvg1wIGGDVMm7vN2SzB1Jz8.jar
Jun 12, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-tests-0b-cakLbXetH89o8NRqg1cNbywrML7jo4PgNjxJfMPc.jar
Jun 12, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-tests-P8bIGxwteeVNXPzuLFlB07hBFaYSAn7PSA_mXnKpYhI.jar
Jun 12, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT-VIcBurLhQJxEYt5NoaEqMlikrpzVvhU8xQg44lZspLw.jar
Jun 12, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-lq7IfbTtX65a_jRpyfQ0V6SIOGpJhgwilVTVJ9qgjQ4.jar
Jun 12, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-tests-hWANT8WQhEqVjc-UDpn_chJdGYw5m3l1O4aSY5YhvAw.jar
Jun 12, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.23.0-SNAPSHOT-HG64iBs8Z5BTC0FFFbdYSp57-efBn63hsyRdTAQmdXk.jar
Jun 12, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/java-fn-execution/build/libs/beam-runners-java-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-java-fn-execution-2.23.0-SNAPSHOT-Qgbeintc0g789bDsTwZe4l-Z52O5avdHVevJI0C_F8g.jar
Jun 12, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.23.0-SNAPSHOT-tests-EodlMGq_LgW7AkC9PQm78GvteM6ClBd_2dT3zP4YVCA.jar
Jun 12, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-Dgbmm9RWAH-w1w5TmJnwc61plfpAn4ipsF-tAG_bTns.jar
Jun 12, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-vbYCHnZwFhdxb4RttM_JOHqvF4AyYh7PFMNoQfdI228.jar
Jun 12, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT-Nn98-CfZ95nvFkn_3xZkXQhmm_moYJnBBEqtoRgvhGs.jar
Jun 12, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-VmRJ_Yw4AyB658wnmaovpK_emPFCx9HRDkRldPBw71E.jar
Jun 12, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.23.0-SNAPSHOT-YpnjiR-fXPeHjVF0KKnVwWes2usvNrfYYIjUKDjPhjw.jar
Jun 12, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.23.0-SNAPSHOT-qlMPdJgEWuKFD4B9NVW0GC_-GJJUeDcjlesHNLValcw.jar
Jun 12, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tests-ewdaiRffAc4SAxYIkVxABAb41Krs8yJZPt0r-MvIAmk.jar
Jun 12, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-6s6n-Ze6hVcoOARuWJKCVEunrCzBcodrv9M9uw7Hgcc.jar
Jun 12, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-fn-execution-2.23.0-SNAPSHOT-K_nRlJ5kmvjmu9i9e-c-6YzmHpA3uzpwfz-yRjTOg3M.jar
Jun 12, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-Bsb9uHt8QjpkJkpvp4fQV2GrkngkBPUgzW1x_EASCys.jar
Jun 12, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-unshaded-snBqlXVjnYex_xfbRsFa9A_w0Ca9waaFNAhGAThbpWg.jar
Jun 12, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-tests-Rf9wGDafmOk7_9awI6CuoAAd3ND8MWv0RWphDAwp7Xo.jar
Jun 12, 2020 6:45:44 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-vbYCHnZwFhdxb4RttM_JOHqvF4AyYh7PFMNoQfdI228.jar
Jun 12, 2020 6:45:44 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.23.0-SNAPSHOT-trWNibgPeew4lCU_GTHQc2lQYV9Nm8y1c7jvSePhv3Q.jar
Jun 12, 2020 6:45:44 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.23.0-SNAPSHOT-76JNVQ3VmLkY3auaL0rzQO1vJ8y2FwgtFjT90knKMdc.jar
Jun 12, 2020 6:45:44 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.23.0-SNAPSHOT-9YYnoHlyaVS1WgCxgtuRrfK6SLbuMnKpOGhAALB06Yc.jar
Jun 12, 2020 6:45:45 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-vbYCHnZwFhdxb4RttM_JOHqvF4AyYh7PFMNoQfdI228.jar&uploadType=resumable&upload_id=AAANsUlRhrZrXofJ6jGNBuhiZ2E6AqtgBEZf2jyE-Tq6HaOv_PpkCUGPAHo9WApfgGT6j1xXPJoRgqTYTsQsXsGYZ1Y.
Jun 12, 2020 6:45:45 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar>
java.io.IOException: Upload failed for 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-vbYCHnZwFhdxb4RttM_JOHqvF4AyYh7PFMNoQfdI228.jar'
at com.google.cloud.hadoop.util.BaseAbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(BaseAbstractGoogleAsyncWriteChannel.java:297)
at com.google.cloud.hadoop.util.BaseAbstractGoogleAsyncWriteChannel.close(BaseAbstractGoogleAsyncWriteChannel.java:214)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:221)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:168)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:152)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:136)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Jun 12, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-vbYCHnZwFhdxb4RttM_JOHqvF4AyYh7PFMNoQfdI228.jar
Jun 12, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 31 files newly uploaded in 5 seconds
Jun 12, 2020 6:45:49 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Jun 12, 2020 6:45:49 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Jun 12, 2020 6:45:49 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Jun 12, 2020 6:45:49 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Jun 12, 2020 6:45:49 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Jun 12, 2020 6:45:49 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <91368 bytes, hash 10b6d432cb7ef60fb04ff42721fd93ee7f03e526ee95e876d564fdfa7b453745> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-ELbUMst-9g-wT_QnIf2T7n8D5Sbuleh21WT9-ntFN0U.pb
Jun 12, 2020 6:45:49 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.23.0-SNAPSHOT
Jun 12, 2020 6:45:50 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-06-11_23_45_49-5496915197890954822?project=apache-beam-testing
Jun 12, 2020 6:45:50 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-06-11_23_45_49-5496915197890954822
Jun 12, 2020 6:45:50 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-06-11_23_45_49-5496915197890954822
Jun 12, 2020 6:45:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-06-12T06:45:49.585Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Jun 12, 2020 6:45:57 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-12T06:45:56.632Z: Worker configuration: n1-standard-1 in us-central1-a.
Jun 12, 2020 6:45:57 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-12T06:45:57.368Z: Expanding CoGroupByKey operations into optimizable parts.
Jun 12, 2020 6:45:57 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-12T06:45:57.398Z: Expanding GroupByKey operations into optimizable parts.
Jun 12, 2020 6:45:57 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-12T06:45:57.428Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jun 12, 2020 6:45:57 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-12T06:45:57.513Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jun 12, 2020 6:45:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-12T06:45:57.540Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Jun 12, 2020 6:45:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-12T06:45:57.580Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Jun 12, 2020 6:45:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-12T06:45:57.605Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Jun 12, 2020 6:45:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-12T06:45:57.952Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Jun 12, 2020 6:45:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-12T06:45:58.028Z: Starting 5 workers in us-central1-a...
Jun 12, 2020 6:46:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-06-12T06:46:15.754Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jun 12, 2020 6:46:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-12T06:46:41.975Z: Autoscaling: Raised the number of workers to 2 based on the rate of progress in the currently running stage(s).
Jun 12, 2020 6:46:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-12T06:46:42.011Z: Resized worker pool to 2, though goal was 5. This could be a quota issue.
Jun 12, 2020 6:46:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-12T06:46:47.341Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Jun 12, 2020 6:47:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-12T06:47:02.104Z: Workers have started successfully.
Jun 12, 2020 6:47:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-12T06:47:02.141Z: Workers have started successfully.
Jun 12, 2020 6:47:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-12T06:47:39.710Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Jun 12, 2020 6:47:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-12T06:47:39.907Z: Cleaning up.
Jun 12, 2020 6:47:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-12T06:47:39.990Z: Stopping worker pool...
Jun 12, 2020 6:49:45 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-12T06:49:45.064Z: Autoscaling: Resized worker pool from 5 to 0.
Jun 12, 2020 6:49:45 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-12T06:49:45.108Z: Worker pool stopped.
Jun 12, 2020 6:49:53 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-06-11_23_45_49-5496915197890954822 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 07430cc1-2680-4620-906f-3d4142b3105c and timestamp: 2020-06-12T06:49:53.777000000Z:
Metric: Value:
read_time 18.807
fields_read 4375276.0
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Jun 12, 2020 6:49:54 AM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 3 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.024 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.03 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 3,5,main]) completed. Took 4 mins 22.098 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 34s
104 actionable tasks: 74 executed, 30 from cache
Publishing build scan...
https://gradle.com/s/ejwf7trmumwjq
Stopped 2 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #616
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/616/display/redirect?page=changes>
Changes:
[robinyqiu] Simple code cleanup for ZetaSqlUtils
[github] Bump default Pubsub timeout to 60 seconds (#11985)
[github] Merge pull request #11950 from [BEAM-8596]: Add SplunkIO transform to
------------------------------------------
[...truncated 294.91 KB...]
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Jun 12, 2020 1:05:24 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Jun 12, 2020 1:05:24 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery writeDisposition is set to: WRITE_EMPTY
Jun 12, 2020 1:05:24 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Jun 12, 2020 1:05:24 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Jun 12, 2020 1:05:24 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery writeDisposition is set to: WRITE_EMPTY
Jun 12, 2020 1:05:24 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Jun 12, 2020 1:05:24 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Jun 12, 2020 1:05:25 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Jun 12, 2020 1:05:25 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Jun 12, 2020 1:05:27 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 209 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Jun 12, 2020 1:05:27 AM org.apache.beam.runners.dataflow.util.PackageUtil$PackageAttributes forFileToStage
INFO: Staging custom dataflow-worker.jar as beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-maRXxaU5JGXQIeBaTKEfgYgbw5-y6hBtPhnJOybXUwI.jar
Jun 12, 2020 1:05:28 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-6j-tiES_mbXjIlZdETfIb_FvtUN-0bUSM7LajE8dr3M.jar
Jun 12, 2020 1:05:28 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /tmp/test739444940205141014.zip to gs://temp-storage-for-perf-tests/loadtests/staging/test-t8Bx_3cgiJx3APSd2vQaI1sYyjoc-mz_deHCCWlph6Q.jar
Jun 12, 2020 1:05:28 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-osFkGHYsrGRT1O3fN-LGbCWXPfkN7Jo0HCEkoODYWBU.jar
Jun 12, 2020 1:05:28 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-tests-AHl5xy1ApAoyLNlq44i9e8UCKgpR35jo6wPhG5K9adU.jar
Jun 12, 2020 1:05:28 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-HsXLTJ1tbphXdnciPjN7xBfTh990H4JkNah6SejmgTo.jar
Jun 12, 2020 1:05:28 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-fd0eaJyZi-9kTA1Uc_yOUTyo9QvZ8bsn0NvynFrFh_k.jar
Jun 12, 2020 1:05:28 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-maRXxaU5JGXQIeBaTKEfgYgbw5-y6hBtPhnJOybXUwI.jar
Jun 12, 2020 1:05:28 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.23.0-SNAPSHOT-cyZW8xNljy9ebMXvuCbMzNejywTA1Zad7N7UIX8RiZU.jar
Jun 12, 2020 1:05:28 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-UyKhmhQPdo5R2ZATI6qAQXdTNrIMk6JY-LrB94NCeis.jar
Jun 12, 2020 1:05:28 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-fn-execution-2.23.0-SNAPSHOT--MwaEbbp0SUDe7tUj1VyX16Jh55uagr1wUedEZcQpRE.jar
Jun 12, 2020 1:05:28 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-tests-9XETw534CWhrecyoEClVSf6vAOxLTclguBEIECYOcJ4.jar
Jun 12, 2020 1:05:28 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.23.0-SNAPSHOT-T_mwzcGWadj26fOdtfYN5IEqBFwAnbz0dbYCVs1w3Fs.jar
Jun 12, 2020 1:05:28 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.23.0-SNAPSHOT-9WoQzqP4noPwFfKhhHwSXIzrt39eAuEtCzXPWWFXTeg.jar
Jun 12, 2020 1:05:28 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-tests-_bg6hbacWbT-SqXPY8rW_3CZE5fV1kFpO00bhPzqX8Q.jar
Jun 12, 2020 1:05:28 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-tests-4UYxiMZVs3AY106gc72kDThaiW70-spUV1hgq6Y6r6c.jar
Jun 12, 2020 1:05:28 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-1LwLKHvXHLvhpoqT0YZx5h__LoivWrumg5sl78gqoDM.jar
Jun 12, 2020 1:05:28 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-oxyNQrDepJDXMVLWAisnUCOGfRierRs95XIKb3sLAW8.jar
Jun 12, 2020 1:05:28 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT-nYNHgiTN5iXqh6oXkVd2GzYN2W8eR0hx8wn0mFt0SL0.jar
Jun 12, 2020 1:05:28 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tests-PWTnrsjAFgCcHyBZMmn6EK77AZdmk5L-lxxWjeawaJs.jar
Jun 12, 2020 1:05:28 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.23.0-SNAPSHOT-r2B40SCD2K0RL8okmV0ukIQOJzPqMlX2s3WYCBRsf40.jar
Jun 12, 2020 1:05:28 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-tests-5tjvN1qJETF3ZKAXlIyKyZdGKEN_WHXE1WjwvPyLDFQ.jar
Jun 12, 2020 1:05:28 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.23.0-SNAPSHOT-tests-Q6C2gtSWAClJ2NYdGiFREb0iDUxbuq_eg0F3xqWiBrM.jar
Jun 12, 2020 1:05:28 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.23.0-SNAPSHOT-w9oGgHW96QV5YnFja_qD212snPBxQwNmoBb2mvvm1gw.jar
Jun 12, 2020 1:05:28 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/java-fn-execution/build/libs/beam-runners-java-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-java-fn-execution-2.23.0-SNAPSHOT-ltL5AvJfzWCNOTgZREw2s7keEcezRZyx9YKsL4xmSsg.jar
Jun 12, 2020 1:05:28 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-tests-Aw1pQKYNrg5qNtrAPIBT76E9oHVOrFC66Kf3I2rpXUA.jar
Jun 12, 2020 1:05:28 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-maRXxaU5JGXQIeBaTKEfgYgbw5-y6hBtPhnJOybXUwI.jar
Jun 12, 2020 1:05:28 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT-g92BV0t5srZruWCbb0abxM7njdgkYY1JvpLYprr1fB0.jar
Jun 12, 2020 1:05:28 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-unshaded-Ge-5fy9Zz06YmS9tNWG-quKKSoN70v2TlHuuZexwPMA.jar
Jun 12, 2020 1:05:28 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.23.0-SNAPSHOT-AIHCiSSFlByZO_6iuqMAy9SR3wVNznzthSfbByUFMeE.jar
Jun 12, 2020 1:05:28 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.23.0-SNAPSHOT-snFvFOcHsmrpU0dIl-ZBw7Pk3wYXjHpqXs54T6c4jqk.jar
Jun 12, 2020 1:05:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.23.0-SNAPSHOT-YUapIgfKyuRMv_qb-tkRwBljiWlRy1Q1kjZgw9JAS40.jar
Jun 12, 2020 1:05:29 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-maRXxaU5JGXQIeBaTKEfgYgbw5-y6hBtPhnJOybXUwI.jar&uploadType=resumable&upload_id=AAANsUnm6Xf79hlKOoBn6wWHxl40u0JdliLrnrFCsGGWyFgPxpZWDjgaMjyz96v3TCCssjp-MjiC5gpbnRdNNihuP90.
Jun 12, 2020 1:05:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar>
java.io.IOException: Upload failed for 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-maRXxaU5JGXQIeBaTKEfgYgbw5-y6hBtPhnJOybXUwI.jar'
at com.google.cloud.hadoop.util.BaseAbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(BaseAbstractGoogleAsyncWriteChannel.java:297)
at com.google.cloud.hadoop.util.BaseAbstractGoogleAsyncWriteChannel.close(BaseAbstractGoogleAsyncWriteChannel.java:214)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:221)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:168)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:152)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:136)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Jun 12, 2020 1:05:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-maRXxaU5JGXQIeBaTKEfgYgbw5-y6hBtPhnJOybXUwI.jar
Jun 12, 2020 1:05:34 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 31 files newly uploaded in 6 seconds
Jun 12, 2020 1:05:34 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Jun 12, 2020 1:05:34 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Jun 12, 2020 1:05:34 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Jun 12, 2020 1:05:35 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Jun 12, 2020 1:05:35 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Jun 12, 2020 1:05:35 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <91367 bytes, hash 42890628109faf25c1d288bf9b871d7878d5c8e82a89198e5f06cf723f961b94> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-QokGKBCfryXB0oi_m4cdeHjVyOgqiRmOXwbPcj-WG5Q.pb
Jun 12, 2020 1:05:35 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.23.0-SNAPSHOT
Jun 12, 2020 1:05:36 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-06-11_18_05_35-13191941373601429123?project=apache-beam-testing
Jun 12, 2020 1:05:36 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-06-11_18_05_35-13191941373601429123
Jun 12, 2020 1:05:36 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-06-11_18_05_35-13191941373601429123
Jun 12, 2020 1:05:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-06-12T01:05:35.617Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Jun 12, 2020 1:05:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-12T01:05:42.885Z: Worker configuration: n1-standard-1 in us-central1-a.
Jun 12, 2020 1:05:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-12T01:05:43.726Z: Expanding CoGroupByKey operations into optimizable parts.
Jun 12, 2020 1:05:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-12T01:05:43.765Z: Expanding GroupByKey operations into optimizable parts.
Jun 12, 2020 1:05:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-12T01:05:43.803Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jun 12, 2020 1:05:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-12T01:05:43.885Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jun 12, 2020 1:05:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-12T01:05:43.921Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Jun 12, 2020 1:05:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-12T01:05:43.953Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Jun 12, 2020 1:05:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-12T01:05:43.978Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Jun 12, 2020 1:05:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-12T01:05:44.390Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Jun 12, 2020 1:05:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-12T01:05:44.473Z: Starting 5 workers in us-central1-a...
Jun 12, 2020 1:06:06 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-06-12T01:06:05.254Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jun 12, 2020 1:06:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-12T01:06:21.369Z: Autoscaling: Raised the number of workers to 3 based on the rate of progress in the currently running stage(s).
Jun 12, 2020 1:06:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-12T01:06:21.399Z: Resized worker pool to 3, though goal was 5. This could be a quota issue.
Jun 12, 2020 1:06:28 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-12T01:06:26.742Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Jun 12, 2020 1:06:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-12T01:06:40.906Z: Workers have started successfully.
Jun 12, 2020 1:06:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-12T01:06:40.941Z: Workers have started successfully.
Jun 12, 2020 1:07:15 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-12T01:07:13.982Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Jun 12, 2020 1:07:15 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-12T01:07:14.211Z: Cleaning up.
Jun 12, 2020 1:07:15 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-12T01:07:14.290Z: Stopping worker pool...
Jun 12, 2020 1:09:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-12T01:09:11.089Z: Autoscaling: Resized worker pool from 5 to 0.
Jun 12, 2020 1:09:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-12T01:09:11.129Z: Worker pool stopped.
Jun 12, 2020 1:09:18 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-06-11_18_05_35-13191941373601429123 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 7fa1fbe1-46cf-49dc-ba50-aa2777dc9b32 and timestamp: 2020-06-12T01:09:18.530000000Z:
Metric: Value:
read_time 15.054
fields_read 4375276.0
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Jun 12, 2020 1:09:19 AM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 2 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.02 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.031 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 4,5,main]) completed. Took 4 mins 2.786 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 9s
104 actionable tasks: 70 executed, 34 from cache
Publishing build scan...
https://gradle.com/s/nfrq34njcuzao
Stopped 1 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #615
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/615/display/redirect?page=changes>
Changes:
[kcweaver] [BEAM-10234] Fix error message for missing licenses.
[github] [BEAM-9951] Adding integration tests for synthetic pipelines in Go
[github] [BEAM-9217] Update DoFn javadoc for schema type translation (#11984)
------------------------------------------
[...truncated 295.50 KB...]
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Jun 11, 2020 8:09:10 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Jun 11, 2020 8:09:10 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery writeDisposition is set to: WRITE_EMPTY
Jun 11, 2020 8:09:11 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Jun 11, 2020 8:09:11 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Jun 11, 2020 8:09:11 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery writeDisposition is set to: WRITE_EMPTY
Jun 11, 2020 8:09:11 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Jun 11, 2020 8:09:11 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Jun 11, 2020 8:09:11 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Jun 11, 2020 8:09:12 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Jun 11, 2020 8:09:14 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 209 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Jun 11, 2020 8:09:14 PM org.apache.beam.runners.dataflow.util.PackageUtil$PackageAttributes forFileToStage
INFO: Staging custom dataflow-worker.jar as beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-vDWptr0SvOLRhH0HRbFHg2a2EVJg4VKeqkhRIz6hBrU.jar
Jun 11, 2020 8:09:14 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-tests-Z9pOZPUUNHwkQoz9gujJKstXypSRN6XwLNdm9j4Uz60.jar
Jun 11, 2020 8:09:14 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-gzyyulLJDqDHnpyhpHKMa5CynP4q1IQE4F73SLwqsAU.jar
Jun 11, 2020 8:09:14 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-tests-_fuw4vj0keSCxERBtjjfagHfeSolZ2tGir6oyKJQ8OE.jar
Jun 11, 2020 8:09:14 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-tests-kJVbJuGC65CHK_klLXkPDxVzZRKo8hH58gRVd0GhWuQ.jar
Jun 11, 2020 8:09:14 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /tmp/test2068334843219372257.zip to gs://temp-storage-for-perf-tests/loadtests/staging/test-NeVP2D6FpN-z_m9cWDzUjiyc390OL31MtPPilyaakos.jar
Jun 11, 2020 8:09:14 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT-p4r0UNOkPrRsI8FQbHfpm1e7bQj3Ri0_v3wkfn-xU38.jar
Jun 11, 2020 8:09:14 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.23.0-SNAPSHOT-DeLlYfibeKmZj8SSLf2aWSQ0Y7xVIE772ON2dUkplIM.jar
Jun 11, 2020 8:09:14 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-tests-ZC2V110UwXELeCrA5VMmRfLJH0y4GgDNo6ER_kVaF9Q.jar
Jun 11, 2020 8:09:14 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-MMutg5ebg5q4lgNSbIREhRZ1fNcyvOHoDPGA_ZtsdgA.jar
Jun 11, 2020 8:09:14 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.23.0-SNAPSHOT-RtqxRZCg8eHInj-6e3rW5M7EWB0WfC3a-ivRThA25HI.jar
Jun 11, 2020 8:09:14 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-lYTxGKI5Z8bHYF3T-nt5iCWjc4yJ7JImoOvo2wdniX4.jar
Jun 11, 2020 8:09:14 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-TyMAGK-bmxM1AD_3AYDmc7Ypw-oYuq7A1qiM0oLM7Hc.jar
Jun 11, 2020 8:09:14 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-hDK75136F8nix6UjFORExYQv8O4WTQkZ9k0jb54pBOM.jar
Jun 11, 2020 8:09:14 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.23.0-SNAPSHOT-s8CfCNo1LPXbQYBnOHcuCSMtXQYBjUOExl9hxzbWJqc.jar
Jun 11, 2020 8:09:14 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.23.0-SNAPSHOT-u244h1wciESle-zTlDfaguEWNG2PTSFKMq4LOeKSGr0.jar
Jun 11, 2020 8:09:14 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT--bvD5q0zUNoG3DhYO5QZuoXpKtEIJXbISOtDEn42Xv4.jar
Jun 11, 2020 8:09:14 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tests-v54Bw0G_XaahAvr3fxayOvOZCyQgT7U4C5laLwJjL3I.jar
Jun 11, 2020 8:09:14 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-fn-execution-2.23.0-SNAPSHOT-yaTWdpsCTmfm2wm8vUtMk1U1E5wgXxm_3cvvMIjsIQg.jar
Jun 11, 2020 8:09:14 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-tests-du1QoMw_yMTdbkp6PlcP_aQETe9WFMMoWeg5xnPiUTc.jar
Jun 11, 2020 8:09:14 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-OXevNLlepZZzPOGhFGmfU5Fv9LyN_VY9zQT9jxDalGg.jar
Jun 11, 2020 8:09:14 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-tests-NAo0p5tN7EvHMMunlynUjcgyhniqgnShTJbLkzf5xkw.jar
Jun 11, 2020 8:09:14 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.23.0-SNAPSHOT-tests-RbUyszHOyyDVNljJNxOoifTBv1sIJ7o6-MUb0Xw602w.jar
Jun 11, 2020 8:09:14 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.23.0-SNAPSHOT-1-FkuYUFc1xKFS-Cl0rsH5wEQtk0GaW2OrO4lQDpB50.jar
Jun 11, 2020 8:09:14 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT-cCbmMQsl5PCc4UfQXbbLKFbJLaCyztNl26TF26vZ30k.jar
Jun 11, 2020 8:09:14 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/java-fn-execution/build/libs/beam-runners-java-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-java-fn-execution-2.23.0-SNAPSHOT-EkiQfgnrXGnjXHB-8ObFp0nJPVG7gBizDkd9kbrtMIc.jar
Jun 11, 2020 8:09:14 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-unshaded-M7Ev-prOstG3t7GcNdSI6z4HrELt_PSeicswFGwfxew.jar
Jun 11, 2020 8:09:14 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-vDWptr0SvOLRhH0HRbFHg2a2EVJg4VKeqkhRIz6hBrU.jar
Jun 11, 2020 8:09:14 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-vDWptr0SvOLRhH0HRbFHg2a2EVJg4VKeqkhRIz6hBrU.jar
Jun 11, 2020 8:09:15 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.23.0-SNAPSHOT-h9EMd4ficb7NBX9atRx-Yxuqt4kFVjusJKCH8vLH-wQ.jar
Jun 11, 2020 8:09:15 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.23.0-SNAPSHOT-PzOFlgdNkALCJMpeDaoOF9UjItGHehweEZ0rLXFbDG4.jar
Jun 11, 2020 8:09:15 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.23.0-SNAPSHOT-_Ai4Rj9MAu30MpFFFb6Yi-KBdrfXCV1_I_oIjOs_UdQ.jar
Jun 11, 2020 8:09:16 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-vDWptr0SvOLRhH0HRbFHg2a2EVJg4VKeqkhRIz6hBrU.jar&uploadType=resumable&upload_id=AAANsUmcoAe3VkJUhAgcL7FLfTsqdv5WiGYYChPtRHRy66Q7rXf-7b3e62Jat3BYv18nTPUtyaUtqfD8G_hXkkvGmW8.
Jun 11, 2020 8:09:16 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar>
java.io.IOException: Upload failed for 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-vDWptr0SvOLRhH0HRbFHg2a2EVJg4VKeqkhRIz6hBrU.jar'
at com.google.cloud.hadoop.util.BaseAbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(BaseAbstractGoogleAsyncWriteChannel.java:297)
at com.google.cloud.hadoop.util.BaseAbstractGoogleAsyncWriteChannel.close(BaseAbstractGoogleAsyncWriteChannel.java:214)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:221)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:168)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:152)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:136)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Jun 11, 2020 8:09:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-vDWptr0SvOLRhH0HRbFHg2a2EVJg4VKeqkhRIz6hBrU.jar
Jun 11, 2020 8:09:22 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 31 files newly uploaded in 8 seconds
Jun 11, 2020 8:09:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Jun 11, 2020 8:09:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Jun 11, 2020 8:09:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Jun 11, 2020 8:09:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Jun 11, 2020 8:09:22 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Jun 11, 2020 8:09:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <91368 bytes, hash e27ca950c187f387f0805696ae6ad68854c60f15169a4d8719523450b1e3a24b> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-4nypUMGH84fwgFaWrmrWiFTGDxUWmk2HGVI0ULHjoks.pb
Jun 11, 2020 8:09:23 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.23.0-SNAPSHOT
Jun 11, 2020 8:09:24 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-06-11_13_09_23-13663564372690341389?project=apache-beam-testing
Jun 11, 2020 8:09:24 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-06-11_13_09_23-13663564372690341389
Jun 11, 2020 8:09:24 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-06-11_13_09_23-13663564372690341389
Jun 11, 2020 8:09:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-06-11T20:09:23.476Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Jun 11, 2020 8:09:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-11T20:09:32.153Z: Worker configuration: n1-standard-1 in us-central1-a.
Jun 11, 2020 8:09:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-11T20:09:33.082Z: Expanding CoGroupByKey operations into optimizable parts.
Jun 11, 2020 8:09:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-11T20:09:33.140Z: Expanding GroupByKey operations into optimizable parts.
Jun 11, 2020 8:09:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-11T20:09:33.176Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jun 11, 2020 8:09:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-11T20:09:33.288Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jun 11, 2020 8:09:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-11T20:09:33.322Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Jun 11, 2020 8:09:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-11T20:09:33.353Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Jun 11, 2020 8:09:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-11T20:09:33.390Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Jun 11, 2020 8:09:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-11T20:09:33.775Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Jun 11, 2020 8:09:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-11T20:09:33.862Z: Starting 5 workers in us-central1-a...
Jun 11, 2020 8:10:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-06-11T20:10:02.147Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jun 11, 2020 8:10:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-11T20:10:05.215Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running stage(s).
Jun 11, 2020 8:10:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-11T20:10:05.250Z: Resized worker pool to 4, though goal was 5. This could be a quota issue.
Jun 11, 2020 8:10:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-11T20:10:10.549Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Jun 11, 2020 8:10:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-11T20:10:21.983Z: Workers have started successfully.
Jun 11, 2020 8:10:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-11T20:10:22.026Z: Workers have started successfully.
Jun 11, 2020 8:11:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-11T20:10:59.892Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Jun 11, 2020 8:11:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-11T20:11:00.163Z: Cleaning up.
Jun 11, 2020 8:11:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-11T20:11:00.262Z: Stopping worker pool...
Jun 11, 2020 8:13:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-11T20:13:20.693Z: Autoscaling: Resized worker pool from 5 to 0.
Jun 11, 2020 8:13:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-11T20:13:20.737Z: Worker pool stopped.
Jun 11, 2020 8:13:25 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-06-11_13_09_23-13663564372690341389 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): e016c76d-44bc-443e-88d6-4b65cd7fbda7 and timestamp: 2020-06-11T20:13:26.019000000Z:
Metric: Value:
fields_read 4375276.0
read_time 19.577
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Jun 11, 2020 8:13:26 PM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 2 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.024 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.032 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 10,5,main]) completed. Took 4 mins 24.373 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 39s
104 actionable tasks: 70 executed, 34 from cache
Publishing build scan...
https://gradle.com/s/4fifpiou5ohcq
Stopped 1 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #614
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/614/display/redirect>
Changes:
------------------------------------------
[...truncated 294.88 KB...]
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:164)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Jun 11, 2020 12:45:38 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Jun 11, 2020 12:45:38 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery writeDisposition is set to: WRITE_EMPTY
Jun 11, 2020 12:45:38 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Jun 11, 2020 12:45:38 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Jun 11, 2020 12:45:38 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery writeDisposition is set to: WRITE_EMPTY
Jun 11, 2020 12:45:38 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Jun 11, 2020 12:45:39 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Jun 11, 2020 12:45:39 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Jun 11, 2020 12:45:39 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Jun 11, 2020 12:45:41 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 209 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Jun 11, 2020 12:45:41 PM org.apache.beam.runners.dataflow.util.PackageUtil$PackageAttributes forFileToStage
INFO: Staging custom dataflow-worker.jar as beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-FvcB41h1kUOuQZASyntQAXV6q3wSVFM1FgBIXxYhni4.jar
Jun 11, 2020 12:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.23.0-SNAPSHOT-tests-BWPzjP_lVSpHjSJ8W-HYtPMA-Lg5mKBsEh6nMWQi8mg.jar
Jun 11, 2020 12:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-CK7JiNy3JswYh0cNrad3CrmhOsR5z6YoHrl9y53KOM4.jar
Jun 11, 2020 12:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-J2tdHBSRxg96qVj15AvZTB9Y7IFfinohZghbAnRDLEY.jar
Jun 11, 2020 12:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.23.0-SNAPSHOT-ZqZ9iDxy4FXaJzumI48X95MJtN2ZX_UavZg1Wu3b-ZA.jar
Jun 11, 2020 12:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tests-XY3WVWZ0oJWw5aSZ8YMs04mHeKtXGkV7h_SKVVfIzBE.jar
Jun 11, 2020 12:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-tests-ZPDFCgkLrcJ-O64dwsR_GBIO4ElKpKawBcSrses-DB0.jar
Jun 11, 2020 12:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-TbR6UmefH8xIYKU6UqhKYqrZ14KDQhsl796vj0KShts.jar
Jun 11, 2020 12:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT-F4G4ZCQZhMM6rHn3IoVGfOezP_VLbD-0ToYxnQ12CQA.jar
Jun 11, 2020 12:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/java-fn-execution/build/libs/beam-runners-java-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-java-fn-execution-2.23.0-SNAPSHOT-8NnPSj8lfG9BGZ5zfNWIKdRz8kihed3WKq0-h82RMJE.jar
Jun 11, 2020 12:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /tmp/test3044208984443173070.zip to gs://temp-storage-for-perf-tests/loadtests/staging/test-T580HHQynQjva-GPUgnnFdbJVGFjgLtB-RPPy_hkhC8.jar
Jun 11, 2020 12:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-5raFExULdQ_utGlIYLoY-0YC4kv3LxTZrfcsiqRR98Q.jar
Jun 11, 2020 12:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-tests-6tMPjs8gZAW7z07tXrKncQ_ghqr05k-I4SSSG9v7Qf0.jar
Jun 11, 2020 12:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-PFVZ3PH9ZF7uSziFXJQR-YTKyWGniLMPILsyrvXpmFk.jar
Jun 11, 2020 12:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-tests-JdtDSnEBkGkmoGQ-KbQvaVp9qt_ljtuxd8p28hHu-tk.jar
Jun 11, 2020 12:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-eVVRUZRoDmjwGZd3LvLmVn8MXfA41grQkONZ8vIzbWQ.jar
Jun 11, 2020 12:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.23.0-SNAPSHOT-KzHGWFOL_uidHliJgHsDDmBaFxglEb2nddth2v0OwzY.jar
Jun 11, 2020 12:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-tests-9IJhxMH6pUsl0RvUa9zixG-7Cdd4f2V84z2vnsZASbk.jar
Jun 11, 2020 12:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.23.0-SNAPSHOT-sXOZTuf1zqd0o0e9Ss2xtuumwdb32Ae2IYWveRyikhI.jar
Jun 11, 2020 12:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT-ARuvfym7mtYT8AAHvbX5BV5eaj1y_OLR_VOy2H_BhRw.jar
Jun 11, 2020 12:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-I7dcYGPrmau8v98nH5QwpDXNuNMG7wJTz_lVZpaVH0E.jar
Jun 11, 2020 12:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-tests-x6Mg53czy0n45okKCd1HskcibikP8Zy2FS2r8ux3GH0.jar
Jun 11, 2020 12:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.23.0-SNAPSHOT--09DfAiHwgNYdQpYgzSbInZHchfPRUPWAbE8PewXTSE.jar
Jun 11, 2020 12:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-FvcB41h1kUOuQZASyntQAXV6q3wSVFM1FgBIXxYhni4.jar
Jun 11, 2020 12:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-fn-execution-2.23.0-SNAPSHOT-MAJCMXaJKKpOCB2ccaK3fsaWeYE3_4ZacArJPWim6nw.jar
Jun 11, 2020 12:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-tests-6zBnExMXGcpH7sHhbSjEQKlYhOw4CfXsB8IaPWrHsfo.jar
Jun 11, 2020 12:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.23.0-SNAPSHOT-5DBX5sraS6pxVwad5a1DGjYPMDxR8vPY0plWDpk-CLE.jar
Jun 11, 2020 12:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-unshaded-pXaowVao4jNMNpT4OVlaVTqdR8HYHX1gC9hX6CMj7M4.jar
Jun 11, 2020 12:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-FvcB41h1kUOuQZASyntQAXV6q3wSVFM1FgBIXxYhni4.jar
Jun 11, 2020 12:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.23.0-SNAPSHOT-nsth-6K4hdvx2Xkb7tZd-uYBI0iUXs68v6HeLe4Nn-s.jar
Jun 11, 2020 12:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.23.0-SNAPSHOT-haV9-KA9D99D-M8aDcHx5ycqY4KzVMo7NvDqvTFv2Z4.jar
Jun 11, 2020 12:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.23.0-SNAPSHOT-tLs_9-i8sJeonSnGuPeCOGGFdPY_oMoSKQlCp2aPtrs.jar
Jun 11, 2020 12:45:43 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-FvcB41h1kUOuQZASyntQAXV6q3wSVFM1FgBIXxYhni4.jar&uploadType=resumable&upload_id=AAANsUmtHiD3fpWGGUtd5egYPeRiZsZ1c3GlMR-3E1py-53PCx7ZhD7_HnrIA6OS_WXdm0EdrxFxXGfo_hNBfjRzzcU.
Jun 11, 2020 12:45:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar>
java.io.IOException: Upload failed for 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-FvcB41h1kUOuQZASyntQAXV6q3wSVFM1FgBIXxYhni4.jar'
at com.google.cloud.hadoop.util.BaseAbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(BaseAbstractGoogleAsyncWriteChannel.java:297)
at com.google.cloud.hadoop.util.BaseAbstractGoogleAsyncWriteChannel.close(BaseAbstractGoogleAsyncWriteChannel.java:214)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:221)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:168)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:152)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:136)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Jun 11, 2020 12:45:48 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-FvcB41h1kUOuQZASyntQAXV6q3wSVFM1FgBIXxYhni4.jar
Jun 11, 2020 12:45:49 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 31 files newly uploaded in 7 seconds
Jun 11, 2020 12:45:49 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Jun 11, 2020 12:45:49 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Jun 11, 2020 12:45:49 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Jun 11, 2020 12:45:49 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Jun 11, 2020 12:45:49 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Jun 11, 2020 12:45:49 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <91368 bytes, hash bf93e8df3471f8a7a7d180a8d21a11d8ea366f81546a4bd0b5c5891dc00ca444> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-v5Po3zRx-Ken0YCo0hoR2Oo2b4FUakvQtcWJHcAMpEQ.pb
Jun 11, 2020 12:45:50 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.23.0-SNAPSHOT
Jun 11, 2020 12:45:51 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-06-11_05_45_50-13589886202289793912?project=apache-beam-testing
Jun 11, 2020 12:45:51 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-06-11_05_45_50-13589886202289793912
Jun 11, 2020 12:45:51 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-06-11_05_45_50-13589886202289793912
Jun 11, 2020 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-06-11T12:45:50.185Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Jun 11, 2020 12:46:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-11T12:45:58.558Z: Worker configuration: n1-standard-1 in us-central1-a.
Jun 11, 2020 12:46:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-11T12:45:59.423Z: Expanding CoGroupByKey operations into optimizable parts.
Jun 11, 2020 12:46:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-11T12:45:59.462Z: Expanding GroupByKey operations into optimizable parts.
Jun 11, 2020 12:46:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-11T12:45:59.494Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jun 11, 2020 12:46:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-11T12:45:59.567Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jun 11, 2020 12:46:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-11T12:45:59.597Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Jun 11, 2020 12:46:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-11T12:45:59.633Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Jun 11, 2020 12:46:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-11T12:45:59.671Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Jun 11, 2020 12:46:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-11T12:46:00.470Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Jun 11, 2020 12:46:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-11T12:46:00.544Z: Starting 5 workers in us-central1-a...
Jun 11, 2020 12:46:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-06-11T12:46:21.352Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jun 11, 2020 12:46:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-11T12:46:36.422Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Jun 11, 2020 12:46:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-11T12:46:55.280Z: Workers have started successfully.
Jun 11, 2020 12:46:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-11T12:46:55.344Z: Workers have started successfully.
Jun 11, 2020 12:47:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-11T12:47:26.339Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Jun 11, 2020 12:47:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-11T12:47:26.576Z: Cleaning up.
Jun 11, 2020 12:47:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-11T12:47:26.654Z: Stopping worker pool...
Jun 11, 2020 12:49:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-11T12:49:43.163Z: Autoscaling: Resized worker pool from 5 to 0.
Jun 11, 2020 12:49:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-11T12:49:43.208Z: Worker pool stopped.
Jun 11, 2020 12:49:49 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-06-11_05_45_50-13589886202289793912 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 1b655dea-7eaa-462e-a1a9-d3beb38d2378 and timestamp: 2020-06-11T12:49:49.303000000Z:
Metric: Value:
fields_read 4375276.0
read_time 14.013
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Jun 11, 2020 12:49:49 PM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 2 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.025 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.037 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 10,5,main]) completed. Took 4 mins 19.896 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 30s
104 actionable tasks: 70 executed, 34 from cache
Publishing build scan...
https://gradle.com/s/juouwo4nemnso
Stopped 1 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #613
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/613/display/redirect>
Changes:
------------------------------------------
[...truncated 293.65 KB...]
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Jun 11, 2020 6:45:22 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Jun 11, 2020 6:45:22 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery writeDisposition is set to: WRITE_EMPTY
Jun 11, 2020 6:45:22 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Jun 11, 2020 6:45:22 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Jun 11, 2020 6:45:22 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery writeDisposition is set to: WRITE_EMPTY
Jun 11, 2020 6:45:22 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Jun 11, 2020 6:45:22 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Jun 11, 2020 6:45:22 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Jun 11, 2020 6:45:23 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Jun 11, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 209 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Jun 11, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil$PackageAttributes forFileToStage
INFO: Staging custom dataflow-worker.jar as beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-iUogIkufuT3JNuXh8Y24_IGC3PJCmQqSvA7qKuCPE4k.jar
Jun 11, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-iUogIkufuT3JNuXh8Y24_IGC3PJCmQqSvA7qKuCPE4k.jar
Jun 11, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-unshaded-kvWW9tfw6gFGisaKMZxEpiSMkpo25rhOQMMymyj7FZo.jar
Jun 11, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-8LFmJPXIjzXdzVgPyGlLZQCcr9Qau8A59R6hAHHpVbo.jar
Jun 11, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.23.0-SNAPSHOT-tests-1FHm-DB_hTU3pwcDI-M5ZBHiU6umUueImti9mlQWezY.jar
Jun 11, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-4RZTiHHypkv3Czom1ZslsRdUBfHVcOD8OuTraGGNHLE.jar
Jun 11, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-lM04RYlmKmIQmXS2qGgVD6MxZqkrdbjUm-ONJpjQuMc.jar
Jun 11, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/java-fn-execution/build/libs/beam-runners-java-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-java-fn-execution-2.23.0-SNAPSHOT-R6jRerq4AqGkARLgtVQ2GDcXexOmAAM366t2qVestEo.jar
Jun 11, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tests-w1PnCAOWvuZH6EfoXflFfWIfyyOisp-zjdem8vh7bI8.jar
Jun 11, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-wb2QOwv-EFBlSlBIBWfrsGYzq9MR18QBONxxf4JeiG0.jar
Jun 11, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-tests-PcmozgyuumUkxzYYLnKpJQ2RRggActTR7UBbb7bXkKc.jar
Jun 11, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.23.0-SNAPSHOT-2SUklalkLZgfX2PQFFE-zcRumJcAHGjMPEYqiw94E9k.jar
Jun 11, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-tests--EtqrMnBqjcZtTr40Z6EeB3oKHClnyxk2C5wQLJa-gM.jar
Jun 11, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-tests-NZWgwMxzeLFCXMtCwiWcXE4YUyl6wD-0Qs_W8-1w7e4.jar
Jun 11, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT-DU_EHs703y_Z2JRU2-HIBBtTnNECre7DZ5k2HdJlYxs.jar
Jun 11, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.23.0-SNAPSHOT-QWNnhegU7xNu6ZCGUyeQNQsjHqAdDRE4Wch3IMmKoOQ.jar
Jun 11, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-fn-execution-2.23.0-SNAPSHOT-t8nsI6VCieIOAmmKP7fQ2Pg1lT-ENgK9eorcl6J1XVc.jar
Jun 11, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.23.0-SNAPSHOT-SPHLsfzPjvjtvjtJLJEuMnyBweWr1eQA9o02OIJIsV0.jar
Jun 11, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.23.0-SNAPSHOT-R3mIEJwZzOmc74mVUBZ_A-mdDwCiZgD6aqmpC4IctU4.jar
Jun 11, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /tmp/test9093528016843603458.zip to gs://temp-storage-for-perf-tests/loadtests/staging/test-KKEC6v9SiNDwb6oosQOiV3rtoT3-mA0R7jEsT_9_gB4.jar
Jun 11, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-tests-9K82qnJwwf-IE4viwvSVj4g1lxUSuoJonS7cSroTAZM.jar
Jun 11, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-rY7TsNSbVbgRSYGikd-QlebFpPVtvD2c0D7xo3yix7c.jar
Jun 11, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-tests-KL9iVsgUfAx5KtJdpZlIoycD-u9wQhM8RwgSlcIsmrk.jar
Jun 11, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-BDkRNLkQFk_UuMs-pz91rkSSBuQCpkIJzGAywT-wwFo.jar
Jun 11, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-tests-HC7mHa6NJAhLxi1JRmxur5hxcJRXkomEQtzfHx5Teac.jar
Jun 11, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.23.0-SNAPSHOT-eOnWWYjzVFI2a-KniQcQA3g166C9wI5rbGSFCoyEBcY.jar
Jun 11, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT-X_9Aem_W8Lnlrn5pr7xXsy_Wuh4sL0VSAQ8nynths0s.jar
Jun 11, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-hsKuzOLItKGS2IS59BWKh6Vyk0VhgO40j8Mkld3WDGE.jar
Jun 11, 2020 6:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-iUogIkufuT3JNuXh8Y24_IGC3PJCmQqSvA7qKuCPE4k.jar
Jun 11, 2020 6:45:26 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.23.0-SNAPSHOT-24fGfi9ZIxwMnKGI4U7kAyo6uv5J9a8O-bYuR897tuM.jar
Jun 11, 2020 6:45:26 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.23.0-SNAPSHOT-hsgmwEP9uSZmrOeLhIaoO8XEss79F2mUMVvo9JoFnOw.jar
Jun 11, 2020 6:45:26 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.23.0-SNAPSHOT-ArhIehqXAqLW0nWKCIlWTjC2ab5EJ2rgMEFpZtW4NnQ.jar
Jun 11, 2020 6:45:27 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-iUogIkufuT3JNuXh8Y24_IGC3PJCmQqSvA7qKuCPE4k.jar&uploadType=resumable&upload_id=AAANsUlBaHeAAoyfR3u9RQ6im_1qBxC6hZqRiHusWnvSej99k5aWRI3RT-IfSI3ay1q12rhmqeEInYxf6Y-Hg1UyuWvqi2v6oQ.
Jun 11, 2020 6:45:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar>
java.io.IOException: Upload failed for 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-iUogIkufuT3JNuXh8Y24_IGC3PJCmQqSvA7qKuCPE4k.jar'
at com.google.cloud.hadoop.util.BaseAbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(BaseAbstractGoogleAsyncWriteChannel.java:297)
at com.google.cloud.hadoop.util.BaseAbstractGoogleAsyncWriteChannel.close(BaseAbstractGoogleAsyncWriteChannel.java:214)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:221)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:168)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:152)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:136)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Jun 11, 2020 6:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-iUogIkufuT3JNuXh8Y24_IGC3PJCmQqSvA7qKuCPE4k.jar
Jun 11, 2020 6:45:34 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 31 files newly uploaded in 9 seconds
Jun 11, 2020 6:45:35 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Jun 11, 2020 6:45:35 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Jun 11, 2020 6:45:35 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Jun 11, 2020 6:45:35 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Jun 11, 2020 6:45:35 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Jun 11, 2020 6:45:35 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <91368 bytes, hash 1cacc17c7e89c06733952983be9a3078139b760c16a52c2d9748859b19f954bb> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-HKzBfH6JwGczlSmDvpoweBObdgwWpSwtl0iFmxn5VLs.pb
Jun 11, 2020 6:45:36 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.23.0-SNAPSHOT
Jun 11, 2020 6:45:37 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-06-10_23_45_36-14396139557788831066?project=apache-beam-testing
Jun 11, 2020 6:45:37 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-06-10_23_45_36-14396139557788831066
Jun 11, 2020 6:45:37 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-06-10_23_45_36-14396139557788831066
Jun 11, 2020 6:45:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-06-11T06:45:36.627Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Jun 11, 2020 6:45:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-11T06:45:45.571Z: Worker configuration: n1-standard-1 in us-central1-a.
Jun 11, 2020 6:45:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-11T06:45:46.357Z: Expanding CoGroupByKey operations into optimizable parts.
Jun 11, 2020 6:45:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-11T06:45:46.461Z: Expanding GroupByKey operations into optimizable parts.
Jun 11, 2020 6:45:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-11T06:45:46.502Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jun 11, 2020 6:45:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-11T06:45:46.612Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jun 11, 2020 6:45:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-11T06:45:46.645Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Jun 11, 2020 6:45:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-11T06:45:46.684Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Jun 11, 2020 6:45:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-11T06:45:46.719Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Jun 11, 2020 6:45:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-11T06:45:47.164Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Jun 11, 2020 6:45:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-11T06:45:47.257Z: Starting 5 workers in us-central1-a...
Jun 11, 2020 6:46:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-06-11T06:45:59.981Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jun 11, 2020 6:46:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-11T06:46:19.958Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running stage(s).
Jun 11, 2020 6:46:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-11T06:46:20.011Z: Resized worker pool to 4, though goal was 5. This could be a quota issue.
Jun 11, 2020 6:46:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-11T06:46:25.476Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Jun 11, 2020 6:46:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-11T06:46:40.475Z: Workers have started successfully.
Jun 11, 2020 6:46:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-11T06:46:40.512Z: Workers have started successfully.
Jun 11, 2020 6:47:15 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-11T06:47:14.096Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Jun 11, 2020 6:47:15 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-11T06:47:14.312Z: Cleaning up.
Jun 11, 2020 6:47:15 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-11T06:47:14.411Z: Stopping worker pool...
Jun 11, 2020 6:49:04 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-11T06:49:04.060Z: Autoscaling: Resized worker pool from 5 to 0.
Jun 11, 2020 6:49:04 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-11T06:49:04.120Z: Worker pool stopped.
Jun 11, 2020 6:49:10 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-06-10_23_45_36-14396139557788831066 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): f7526e1c-2d55-42cd-8830-9a7a12cd16ce and timestamp: 2020-06-11T06:49:10.663000000Z:
Metric: Value:
read_time 15.215
fields_read 4375276.0
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Jun 11, 2020 6:49:11 AM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.026 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.035 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 11,5,main]) completed. Took 3 mins 57.019 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 50s
104 actionable tasks: 68 executed, 36 from cache
Publishing build scan...
https://gradle.com/s/gzahmqahudlya
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #612
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/612/display/redirect?page=changes>
Changes:
[robertwb] [BEAM-9577] Remove use of legacy artifact service in Python.
[robertwb] Simplify Python on Flink runner instructions.
[robertwb] Fix stray paragraph, separate and rework python.
[tysonjh] [BEAM-9999] Remove Gearpump runner.
[robertwb] Expand note on runner selection.
[amaliujia] [BEAM-10230] @Ignore: BYTES works with LIKE.
[robertwb] Move Beam Compatibility table below instructions.
[github] Finalize CHANGES.md for 2.22.0 (#11973)
[github] [BEAM-9679] Add CombinePerKey to Core Transforms Go Katas (#11936)
------------------------------------------
[...truncated 295.02 KB...]
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Jun 11, 2020 12:53:18 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Jun 11, 2020 12:53:18 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery writeDisposition is set to: WRITE_EMPTY
Jun 11, 2020 12:53:18 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Jun 11, 2020 12:53:18 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Jun 11, 2020 12:53:18 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery writeDisposition is set to: WRITE_EMPTY
Jun 11, 2020 12:53:18 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Jun 11, 2020 12:53:18 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Jun 11, 2020 12:53:18 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Jun 11, 2020 12:53:19 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Jun 11, 2020 12:53:21 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 209 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Jun 11, 2020 12:53:21 AM org.apache.beam.runners.dataflow.util.PackageUtil$PackageAttributes forFileToStage
INFO: Staging custom dataflow-worker.jar as beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-4cukW3dLFLxds8UbdR6Eq0NKeVoe49e3MwsWk1g7-6A.jar
Jun 11, 2020 12:53:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /tmp/test8145588249311883655.zip to gs://temp-storage-for-perf-tests/loadtests/staging/test-zfiDqxM3zcEFdrnnRTF71C_r3kcqHTAgAO7umJbs0Gg.jar
Jun 11, 2020 12:53:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-tests-byLGGP9weTdggq-2uuJXvNkNT3Z0j_LyVUMXEhE7QuQ.jar
Jun 11, 2020 12:53:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-fn-execution-2.23.0-SNAPSHOT-WddSzP8ij9Da66w0MtEi4bOcpGKg5XEunptyWRMpQWc.jar
Jun 11, 2020 12:53:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT-hoTS1-3-sY8sMkTm1ziU7iTYBbyw7Cc9eGojgM3lRWQ.jar
Jun 11, 2020 12:53:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-tests-uj9F2V6wWpLnRHJfJz5laER_zj4rUqV7UaY_YDrNvBQ.jar
Jun 11, 2020 12:53:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/java-fn-execution/build/libs/beam-runners-java-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-java-fn-execution-2.23.0-SNAPSHOT-b1tanqPBSOMYEACJVpzz_9ucURt9dHfaxzq6pVoJ2ow.jar
Jun 11, 2020 12:53:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-tests-NCaa1Dq8pbeXJ5lM3sNtI0jMA3k8Y9n6m4JRej2NEGs.jar
Jun 11, 2020 12:53:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.23.0-SNAPSHOT-yggbo8H0ORWvO1K46mpyVM4jc7sp7QxwEr4gtYKkBGE.jar
Jun 11, 2020 12:53:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-ly4CxtYOok5UPR6Bkgkkm1ZWRUg9qjNU0li8EI2c6LM.jar
Jun 11, 2020 12:53:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-4cukW3dLFLxds8UbdR6Eq0NKeVoe49e3MwsWk1g7-6A.jar
Jun 11, 2020 12:53:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.23.0-SNAPSHOT-_twIe_jr8M1AtsX3zqP6odUTOSN3a6nr84v95WMMwN8.jar
Jun 11, 2020 12:53:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-gmY-8EqDOLUHcr83_2E_8UV6Y57m2bM1NDCRANAKCbM.jar
Jun 11, 2020 12:53:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.23.0-SNAPSHOT-Aa3lIC5vQUuARZsRF2xQton2o0gzADyUNieVb9MHpvI.jar
Jun 11, 2020 12:53:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-tests-JCyaT4qVMVQHh4D-J4pz1kLWYoV0o-MteU42LGxK8W0.jar
Jun 11, 2020 12:53:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tJA8gBnAEfJ2znoUH6Mk6KA41kYIVzMa7BK_wj2sXwM.jar
Jun 11, 2020 12:53:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.23.0-SNAPSHOT-tests-JIMBq4c7XnN2ajVfdjrsrxBt5NSn4Ewjh2jyaL-KZgU.jar
Jun 11, 2020 12:53:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-Sz2XSLQOMh1d0YdUyE0RSucBZoPAP81QLeLVdiVV7Tw.jar
Jun 11, 2020 12:53:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-tests-4Iq2LR_iLfakPyxpJXZ_yRj5yQK7gsxC_vXCSScpAds.jar
Jun 11, 2020 12:53:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-1fIlfDZkmxcGGoq5CdcaIVEXSNTcQIjD6zpDk5lkmZw.jar
Jun 11, 2020 12:53:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-ctpx7qhFzhvdukeJ0805O0gv5n1P_aV6bXrJ32JW7W0.jar
Jun 11, 2020 12:53:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-tests-lME_nlF5U_VwTBhcLvsqlLp7r1TrQZxWhD_CVixluAU.jar
Jun 11, 2020 12:53:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tests-pslvUBQ0mA0jn5AWQcB09rwvlf3vneLx47j6mT298l4.jar
Jun 11, 2020 12:53:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.23.0-SNAPSHOT-wxBRSrQ-nQ0glvv9G_JLeziSnH49PxInWMWGhOk2KH8.jar
Jun 11, 2020 12:53:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-unshaded-72eac9XqZa7msHhi0KA9wgenx6CeYJUgTgd_k5Rw7-8.jar
Jun 11, 2020 12:53:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.23.0-SNAPSHOT-pl7IW3ML3GOoX99ULuS8rUkKeVy6TqSv0O9hLzRsPs8.jar
Jun 11, 2020 12:53:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT-w0xBlCIZ5Ju0RARvSSsdDW8ETm753w5vnsjog0SSKFM.jar
Jun 11, 2020 12:53:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-b3TY4X8HqGV_YlnNkxuAoxWTRAlYheAn5o3gTr671AM.jar
Jun 11, 2020 12:53:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-4cukW3dLFLxds8UbdR6Eq0NKeVoe49e3MwsWk1g7-6A.jar
Jun 11, 2020 12:53:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.23.0-SNAPSHOT-d0hzoPMeMmYLpQ4Y_sXuYTkl7CzKbZPh8Ca23_JXfBU.jar
Jun 11, 2020 12:53:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.23.0-SNAPSHOT-hqAt03yUhM_2SXZ0h1AR1NDc7pJy3Y2dkDrsc-Xw6ZM.jar
Jun 11, 2020 12:53:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.23.0-SNAPSHOT-s5waaLD7a828iWkJ5kARTsNhShyMJyxqj9kVqt5menc.jar
Jun 11, 2020 12:53:23 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-4cukW3dLFLxds8UbdR6Eq0NKeVoe49e3MwsWk1g7-6A.jar&uploadType=resumable&upload_id=AAANsUnXQuXCv9OZMo8AilVkY1_PteYLu1xGBmya2gsT60sVM9fMUi_JlwOM5_wfvTC8MlF-gnMEiPwvNZqTiyLhVOfMq1gpSg.
Jun 11, 2020 12:53:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar>
java.io.IOException: Upload failed for 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-4cukW3dLFLxds8UbdR6Eq0NKeVoe49e3MwsWk1g7-6A.jar'
at com.google.cloud.hadoop.util.BaseAbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(BaseAbstractGoogleAsyncWriteChannel.java:297)
at com.google.cloud.hadoop.util.BaseAbstractGoogleAsyncWriteChannel.close(BaseAbstractGoogleAsyncWriteChannel.java:214)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:221)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:168)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:152)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:136)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Jun 11, 2020 12:53:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-4cukW3dLFLxds8UbdR6Eq0NKeVoe49e3MwsWk1g7-6A.jar
Jun 11, 2020 12:53:27 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 31 files newly uploaded in 6 seconds
Jun 11, 2020 12:53:27 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Jun 11, 2020 12:53:27 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Jun 11, 2020 12:53:27 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Jun 11, 2020 12:53:28 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Jun 11, 2020 12:53:28 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Jun 11, 2020 12:53:28 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <91368 bytes, hash 3fe7518d046f2e18b63b6cd364336a38531f46f6ddc925e6433f9904f1608f67> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-P-dRjQRvLhi2O2zTZDNqOFMfRvbdySXmQz-ZBPFgj2c.pb
Jun 11, 2020 12:53:28 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.23.0-SNAPSHOT
Jun 11, 2020 12:53:30 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-06-10_17_53_28-18152934898162669644?project=apache-beam-testing
Jun 11, 2020 12:53:30 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-06-10_17_53_28-18152934898162669644
Jun 11, 2020 12:53:30 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-06-10_17_53_28-18152934898162669644
Jun 11, 2020 12:53:30 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-06-11T00:53:28.734Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Jun 11, 2020 12:53:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-11T00:53:36.160Z: Worker configuration: n1-standard-1 in us-central1-a.
Jun 11, 2020 12:53:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-11T00:53:36.960Z: Expanding CoGroupByKey operations into optimizable parts.
Jun 11, 2020 12:53:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-11T00:53:37.058Z: Expanding GroupByKey operations into optimizable parts.
Jun 11, 2020 12:53:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-11T00:53:37.087Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jun 11, 2020 12:53:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-11T00:53:37.160Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jun 11, 2020 12:53:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-11T00:53:37.188Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Jun 11, 2020 12:53:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-11T00:53:37.216Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Jun 11, 2020 12:53:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-11T00:53:37.243Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Jun 11, 2020 12:53:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-11T00:53:37.539Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Jun 11, 2020 12:53:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-11T00:53:37.604Z: Starting 5 workers in us-central1-a...
Jun 11, 2020 12:53:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-06-11T00:53:48.116Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jun 11, 2020 12:54:06 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-11T00:54:05.108Z: Autoscaling: Raised the number of workers to 3 based on the rate of progress in the currently running stage(s).
Jun 11, 2020 12:54:06 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-11T00:54:05.142Z: Resized worker pool to 3, though goal was 5. This could be a quota issue.
Jun 11, 2020 12:54:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-11T00:54:10.572Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Jun 11, 2020 12:54:28 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-11T00:54:27.591Z: Workers have started successfully.
Jun 11, 2020 12:54:28 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-11T00:54:27.631Z: Workers have started successfully.
Jun 11, 2020 12:55:04 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-11T00:55:04.322Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Jun 11, 2020 12:55:05 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-11T00:55:04.650Z: Cleaning up.
Jun 11, 2020 12:55:05 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-11T00:55:04.728Z: Stopping worker pool...
Jun 11, 2020 12:57:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-11T00:57:09.083Z: Autoscaling: Resized worker pool from 5 to 0.
Jun 11, 2020 12:57:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-11T00:57:09.121Z: Worker pool stopped.
Jun 11, 2020 12:57:14 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-06-10_17_53_28-18152934898162669644 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 23203976-fb64-4ba2-a8c2-35d71f61c04c and timestamp: 2020-06-11T00:57:15.030000000Z:
Metric: Value:
read_time 17.736
fields_read 4375276.0
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Jun 11, 2020 12:57:15 AM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 2 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.024 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.031 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 6,5,main]) completed. Took 4 mins 5.192 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 13s
104 actionable tasks: 70 executed, 34 from cache
Publishing build scan...
https://gradle.com/s/vglc7eszxusjm
Stopped 1 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #611
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/611/display/redirect?page=changes>
Changes:
[klk] Split Nexmark QueryTest and SqlQueryTest for clarity
[klk] Add ZetaSQL Nexmark variant
[annaqin] [BEAM-10225] Add log message when starting job server
[valentyn] [BEAM-10227] Switches typing version modifier to python_full_version so
[kamil.wasilewski] [BEAM-8134] Grafana dashboards for Nexmark tests
[github] [BEAM-9742] Add Configurable FluentBackoff to JdbcIO Write (#11396)
------------------------------------------
[...truncated 291.59 KB...]
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDirectReadMethod(BigQueryIOPushDownIT.java:150)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod STANDARD_ERROR
Jun 10, 2020 6:54:49 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
Jun 10, 2020 6:54:49 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery writeDisposition is set to: WRITE_EMPTY
Jun 10, 2020 6:54:49 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Jun 10, 2020 6:54:49 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
Jun 10, 2020 6:54:49 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery writeDisposition is set to: WRITE_EMPTY
Jun 10, 2020 6:54:50 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Jun 10, 2020 6:54:50 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..13=[{inputs}], expr#14=['story':VARCHAR], expr#15=[=($t8, $t14)], expr#16=['job':VARCHAR], expr#17=[=($t8, $t16)], expr#18=[OR($t15, $t17)], expr#19=[2], expr#20=[>($t5, $t19)], expr#21=[AND($t18, $t20)], author=[$t4], type=[$t8], title=[$t0], score=[$t5], $condition=[$t21])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDefaultMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output [PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed: PTransform.getOutputCoder called.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:164)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Jun 10, 2020 6:54:50 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Jun 10, 2020 6:54:50 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery writeDisposition is set to: WRITE_EMPTY
Jun 10, 2020 6:54:50 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Jun 10, 2020 6:54:50 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Jun 10, 2020 6:54:50 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery writeDisposition is set to: WRITE_EMPTY
Jun 10, 2020 6:54:50 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Jun 10, 2020 6:54:50 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Jun 10, 2020 6:54:50 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Jun 10, 2020 6:54:51 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Jun 10, 2020 6:54:53 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 209 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Jun 10, 2020 6:54:53 PM org.apache.beam.runners.dataflow.util.PackageUtil$PackageAttributes forFileToStage
INFO: Staging custom dataflow-worker.jar as beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-DVnGEm0m0ADrKE9ybjTr67RFzQKibUgsOO_jDIMnO4k.jar
Jun 10, 2020 6:54:53 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.23.0-SNAPSHOT-tests-b9hTCVb9XT-CeU2E7gy_jxqLa00_ZzOejUJnDbkgeEs.jar
Jun 10, 2020 6:54:53 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-DVnGEm0m0ADrKE9ybjTr67RFzQKibUgsOO_jDIMnO4k.jar
Jun 10, 2020 6:54:53 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.23.0-SNAPSHOT-BDVWDjMkMLSbElr-sUU69-bXeF4pFYSjYwhFy1Q2bAw.jar
Jun 10, 2020 6:54:53 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-cYhIL6oOQscHE6dkLcUN1PHHpXjS6gh5S-ozMmheJ0c.jar
Jun 10, 2020 6:54:53 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-OF72tA4hH15RKfsaNb1yTMwMtUK1CcZjoH8K7CLTxPw.jar
Jun 10, 2020 6:54:53 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-tests-DECbczIPWcJgYlDEQhaPmr99v6dUXs8ED9_QF_nhdDs.jar
Jun 10, 2020 6:54:53 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /tmp/test6871412552102099310.zip to gs://temp-storage-for-perf-tests/loadtests/staging/test-s6jvBJDkQmMRc9wiz6yigIntgDN4f0S3txB_T9lFsk4.jar
Jun 10, 2020 6:54:53 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-tests-AT98MMVnDiGLZwI5y1Kpc_0ZlXihvP7mhOEwKbTn5ts.jar
Jun 10, 2020 6:54:53 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-OVlA4TVP3ghNdzlt8Yp1UztPHr-geKcf9hbGptyaDHY.jar
Jun 10, 2020 6:54:53 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-tests-R9jy4MNL8sHEjsFTGoa18v0dzKFfpf-fmqPGS-yr_g0.jar
Jun 10, 2020 6:54:53 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-xinzm-QBX38jwayfekDdVp4ktA6DVwF0WtF6Atf_2FU.jar
Jun 10, 2020 6:54:53 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-Eyiu3BVSUBKKIF5rhOkLJmx0cIKPzQuimxzDAbC4ehM.jar
Jun 10, 2020 6:54:54 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/java-fn-execution/build/libs/beam-runners-java-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-java-fn-execution-2.23.0-SNAPSHOT-hWyhxr1zP7YlDIZrXQf7unyBidkPN6pf3JAE-AdUEp8.jar
Jun 10, 2020 6:54:54 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tests-K2czePrSCXq1ysEcXAn9fvlCLDx6LWTUYml7JMJVRQ8.jar
Jun 10, 2020 6:54:54 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.23.0-SNAPSHOT-nD2jlrjFmH-JQSWJ0ZR9z2ugPRwrMblV7FEXF05DRdE.jar
Jun 10, 2020 6:54:54 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-tests-Nd2Gx611dUw7RvF0EmV-In38pdrz9alBxFdzNNYlJEE.jar
Jun 10, 2020 6:54:54 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT-09L8-djGz-EX63bjxh1TBQogRyNdukTR04WNvX6U_kI.jar
Jun 10, 2020 6:54:54 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.23.0-SNAPSHOT-MuJdNTTu7GxJhDASpRF-FAL1DUORSW5BjzxKCHK8VKY.jar
Jun 10, 2020 6:54:54 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT-8XTrd0LBiuX3G9gjUOs7KMyBr35CVvnqEemofWmnbF8.jar
Jun 10, 2020 6:54:54 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-tests-vjwbX0nQkRH4pRfYEi72GrDsochSTUIwxxGAQv7SRC0.jar
Jun 10, 2020 6:54:54 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-fn-execution-2.23.0-SNAPSHOT-h89Mua5M_052xPHTAQNEhsHmBYPM6jseNDtc6od1wnI.jar
Jun 10, 2020 6:54:54 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.23.0-SNAPSHOT-Z3JMMD8IzrERTBsI6xIqSXqSRsZDqHbogLwFALvxOqQ.jar
Jun 10, 2020 6:54:54 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-unshaded-wuVrI1Fe7P2Oqy-Ka8VcxmwFKBcmh3sYeuKS4HCHeQE.jar
Jun 10, 2020 6:54:54 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.23.0-SNAPSHOT--af7i6_22LxMAXB7lQOrd0FNbi942FTSreZiFpbUjhg.jar
Jun 10, 2020 6:54:54 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-tests-dvVAVmJbIR1Ip9GbeD-B5nAUS7DpxumUaSRNUK7Jlfs.jar
Jun 10, 2020 6:54:54 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-h0w4RnnF-kqqHGlcV7LeC362YotcDuciHH9jI1H8wkM.jar
Jun 10, 2020 6:54:54 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-bqYs-ZhE_Dwt3mpJZuph_HfM51IAdZLGU-cqM8TsOms.jar
Jun 10, 2020 6:54:54 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.23.0-SNAPSHOT-SEe0UZoG47NPqsc13XZ4B_HwV8RD1bPvrBs1Lc_UqEg.jar
Jun 10, 2020 6:54:54 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.23.0-SNAPSHOT-MXx4lWlwJCSmb4raL5CHs1wNsWDaKVFQTAYAjZqWQxs.jar
Jun 10, 2020 6:54:54 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.23.0-SNAPSHOT--32GwURW26KUNUw-73qbqPqsmZ5ACK226Pum9kfIV0I.jar
Jun 10, 2020 6:54:55 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 179 files cached, 30 files newly uploaded in 1 seconds
Jun 10, 2020 6:54:55 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Jun 10, 2020 6:54:55 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Jun 10, 2020 6:54:55 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Jun 10, 2020 6:54:55 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Jun 10, 2020 6:54:55 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Jun 10, 2020 6:54:55 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <91367 bytes, hash edfcce8a9b31a18d52816cb5b91bade9c7690f3aff5ac63ba9695d70171fcda3> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-7fzOipsxoY1SgWy1uRut6cdpDzr_WsY7qWldcBcfzaM.pb
Jun 10, 2020 6:54:56 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.23.0-SNAPSHOT
Jun 10, 2020 6:54:57 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-06-10_11_54_56-12015661819334594219?project=apache-beam-testing
Jun 10, 2020 6:54:57 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-06-10_11_54_56-12015661819334594219
Jun 10, 2020 6:54:57 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-06-10_11_54_56-12015661819334594219
Jun 10, 2020 6:54:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-06-10T18:54:56.265Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Jun 10, 2020 6:55:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-10T18:55:04.561Z: Worker configuration: n1-standard-1 in us-central1-a.
Jun 10, 2020 6:55:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-10T18:55:05.555Z: Expanding CoGroupByKey operations into optimizable parts.
Jun 10, 2020 6:55:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-10T18:55:05.595Z: Expanding GroupByKey operations into optimizable parts.
Jun 10, 2020 6:55:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-10T18:55:05.634Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jun 10, 2020 6:55:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-10T18:55:05.719Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jun 10, 2020 6:55:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-10T18:55:05.751Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Jun 10, 2020 6:55:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-10T18:55:05.784Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Jun 10, 2020 6:55:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-10T18:55:05.822Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Jun 10, 2020 6:55:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-10T18:55:06.400Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Jun 10, 2020 6:55:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-10T18:55:06.482Z: Starting 5 workers in us-central1-a...
Jun 10, 2020 6:55:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-06-10T18:55:25.198Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jun 10, 2020 6:55:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-10T18:55:35.741Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Jun 10, 2020 6:55:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-10T18:55:52.593Z: Workers have started successfully.
Jun 10, 2020 6:55:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-10T18:55:52.632Z: Workers have started successfully.
Jun 10, 2020 6:56:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-10T18:56:34.881Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Jun 10, 2020 6:56:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-10T18:56:35.096Z: Cleaning up.
Jun 10, 2020 6:56:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-10T18:56:35.194Z: Stopping worker pool...
Jun 10, 2020 6:58:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-10T18:58:42.893Z: Autoscaling: Resized worker pool from 5 to 0.
Jun 10, 2020 6:58:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-10T18:58:42.945Z: Worker pool stopped.
Jun 10, 2020 6:58:48 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-06-10_11_54_56-12015661819334594219 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 25d22109-252a-4456-8ec1-5caa61c25325 and timestamp: 2020-06-10T18:58:48.529000000Z:
Metric: Value:
fields_read 4375276.0
read_time 23.351
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Jun 10, 2020 6:58:48 PM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 2 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.022 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.031 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 5,5,main]) completed. Took 4 mins 8.794 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 38s
104 actionable tasks: 71 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/6zhkqzyxivd3m
Stopped 1 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #610
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/610/display/redirect>
Changes:
------------------------------------------
[...truncated 295.58 KB...]
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Jun 10, 2020 12:45:35 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Jun 10, 2020 12:45:35 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery writeDisposition is set to: WRITE_EMPTY
Jun 10, 2020 12:45:35 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Jun 10, 2020 12:45:35 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Jun 10, 2020 12:45:35 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery writeDisposition is set to: WRITE_EMPTY
Jun 10, 2020 12:45:35 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Jun 10, 2020 12:45:35 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Jun 10, 2020 12:45:35 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Jun 10, 2020 12:45:36 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Jun 10, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 209 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Jun 10, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil$PackageAttributes forFileToStage
INFO: Staging custom dataflow-worker.jar as beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-sFI-DNk0-qDN6vC5IAPQyCTTRZ9STQ89WRB82OnEx8Y.jar
Jun 10, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-tests-xURVMsttXfaL1wBLkUB2jJAs-9jyHRgNGRLDHHVqzh4.jar
Jun 10, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.23.0-SNAPSHOT-etS2wTyuOfZq_5oJMcN9IBPuNsaNc2BpXhkwufeqkLo.jar
Jun 10, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-tests-5njbyfpSbeEcVLpg9i8COYip11JJKjz2AF1Bf52Sht4.jar
Jun 10, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-tests-qNUuXJlKBsLwQTAVUBIDetsgbBfE6hTm9qX-0fk4eZ4.jar
Jun 10, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/java-fn-execution/build/libs/beam-runners-java-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-java-fn-execution-2.23.0-SNAPSHOT--cUg7rIql3yN3fBENBM2p03PdXdnKUQ7qTPYgcgamfE.jar
Jun 10, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.23.0-SNAPSHOT-jCa9hBcahQTKMm-Zco8fEpSyT4mp9QryWUR1lwmMVk4.jar
Jun 10, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-tests-QK_t-HmUL1W-Wy8fhvjkRNrdTul2tvOwp2mliFyT8A4.jar
Jun 10, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-5vaJCzRRIKByyqVlGUP_eJTQ41Iwy64YTrs3jA_ApE4.jar
Jun 10, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-fId0o_MuPcEFx4nlsYsG3rdVvW23zoDqfzC29r4HPuM.jar
Jun 10, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT-KyRZve1SOGPFBIm-hgBaU-IgWZsr4n1WDmaaSkW6DyI.jar
Jun 10, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.23.0-SNAPSHOT-RFBz22JdmKZb6n45-WDrCHP-4MPaqUs3ALy9ycy3qEE.jar
Jun 10, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.23.0-SNAPSHOT-tests-IKmbXfYv24W2NNnPFu5Eh8Z5YQF7itCmXRLkT7MrHn0.jar
Jun 10, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.23.0-SNAPSHOT-5v-T5qKqXEdjM0fc-s666WROYaQ02PcMJGudEkO8Pxw.jar
Jun 10, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT-9Hdk2k0_yQfO6AzC_uDKXkHK_7BeiX99dQFmvRinBiE.jar
Jun 10, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-EGm-Sh3Zrs36_FyO9mOJICSFY9obHYuJHXFhHvD5QcQ.jar
Jun 10, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-7jifrpAejbxDJJepVBBR1DigiJTaV3jZPUkxsdo_8OE.jar
Jun 10, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-vUQHQpkj-qLr2POhvTr-vEDnpFH5xiBCI9O9DBBLKZ4.jar
Jun 10, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-hasfodoO0q9fCo1DOMVBrhJZ-xboNNXMSb5hau9M5Iw.jar
Jun 10, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tests-b11Kj69pdAnCc1lUP7aAJ6SgzYGU8RpU1ljqKiWmfWo.jar
Jun 10, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-EXB9CPsLzcSZEZ7a1kaR1VrJf89HnFJZX2fPNRwhVcY.jar
Jun 10, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-sFI-DNk0-qDN6vC5IAPQyCTTRZ9STQ89WRB82OnEx8Y.jar
Jun 10, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-tests-WiH_pSEMUBuKaP_rUSMdJNUT7M8P7JMpyxN0jfDoJNs.jar
Jun 10, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /tmp/test7854155476105310059.zip to gs://temp-storage-for-perf-tests/loadtests/staging/test-ba6wpUN0wASH5LzhL0f3sxGRL2NHqwFN20zrzNg2GB8.jar
Jun 10, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.23.0-SNAPSHOT-37z1g_ibgAgsm23a355y6hJ375FxtXRjxyDSdwlxNp0.jar
Jun 10, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-fn-execution-2.23.0-SNAPSHOT-lLTzO3BO4Befc8qgKs3-JUsdkpqRtTq9PaizyispgOY.jar
Jun 10, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-tests-TfqetCDoZr_Dc7sByTPBVG38fNDE1QNwRBOs9NTtoU4.jar
Jun 10, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-unshaded-j_TXwzeLrf-jfpOCnq9Pknf39fQKloH_9ujWvw11YT0.jar
Jun 10, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-sFI-DNk0-qDN6vC5IAPQyCTTRZ9STQ89WRB82OnEx8Y.jar
Jun 10, 2020 12:45:39 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.23.0-SNAPSHOT-rSNc0f_D8jlIzaSxHvEarN4jUHHtRwNCbJCID9D-_cE.jar
Jun 10, 2020 12:45:39 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.23.0-SNAPSHOT-NSir8EJt_vrW1lPj-l28dIf8qsqNN55EaMVYkhCK4mE.jar
Jun 10, 2020 12:45:39 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.23.0-SNAPSHOT-E-l2psx3xufoBmV9MNiDaNwVHq-x-txfLtTptTdF-kc.jar
Jun 10, 2020 12:45:40 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-sFI-DNk0-qDN6vC5IAPQyCTTRZ9STQ89WRB82OnEx8Y.jar&uploadType=resumable&upload_id=AAANsUnZUVp8zyb0DHfdB_v1_t-xQVuBPvyBCdC0HoO3cislLRf8w9JwBfusn3mGzAlUEgVbXG7ruX3T3q178LUifyqtDJu4Ag.
Jun 10, 2020 12:45:40 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar>
java.io.IOException: Upload failed for 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-sFI-DNk0-qDN6vC5IAPQyCTTRZ9STQ89WRB82OnEx8Y.jar'
at com.google.cloud.hadoop.util.BaseAbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(BaseAbstractGoogleAsyncWriteChannel.java:297)
at com.google.cloud.hadoop.util.BaseAbstractGoogleAsyncWriteChannel.close(BaseAbstractGoogleAsyncWriteChannel.java:214)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:221)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:168)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:152)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:136)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Jun 10, 2020 12:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-sFI-DNk0-qDN6vC5IAPQyCTTRZ9STQ89WRB82OnEx8Y.jar
Jun 10, 2020 12:45:43 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 31 files newly uploaded in 5 seconds
Jun 10, 2020 12:45:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Jun 10, 2020 12:45:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Jun 10, 2020 12:45:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Jun 10, 2020 12:45:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Jun 10, 2020 12:45:43 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Jun 10, 2020 12:45:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <91368 bytes, hash 49000885e29601cc2bc731dfc80cf7521322f403ffd6c5ecf8d9ea52d65f7b5f> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-SQAIheKWAcwrxzHfyAz3UhMi9AP_1sXs-NnqUtZfe18.pb
Jun 10, 2020 12:45:44 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.23.0-SNAPSHOT
Jun 10, 2020 12:45:45 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-06-10_05_45_44-4510915318570474779?project=apache-beam-testing
Jun 10, 2020 12:45:45 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-06-10_05_45_44-4510915318570474779
Jun 10, 2020 12:45:45 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-06-10_05_45_44-4510915318570474779
Jun 10, 2020 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-06-10T12:45:44.500Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Jun 10, 2020 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-10T12:45:53.665Z: Worker configuration: n1-standard-1 in us-central1-a.
Jun 10, 2020 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-10T12:45:54.852Z: Expanding CoGroupByKey operations into optimizable parts.
Jun 10, 2020 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-10T12:45:54.901Z: Expanding GroupByKey operations into optimizable parts.
Jun 10, 2020 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-10T12:45:54.933Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jun 10, 2020 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-10T12:45:55.016Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jun 10, 2020 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-10T12:45:55.053Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Jun 10, 2020 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-10T12:45:55.092Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Jun 10, 2020 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-10T12:45:55.133Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Jun 10, 2020 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-10T12:45:55.595Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Jun 10, 2020 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-10T12:45:55.681Z: Starting 5 workers in us-central1-a...
Jun 10, 2020 12:46:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-06-10T12:46:15.856Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jun 10, 2020 12:46:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-10T12:46:23.307Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running stage(s).
Jun 10, 2020 12:46:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-10T12:46:23.359Z: Resized worker pool to 4, though goal was 5. This could be a quota issue.
Jun 10, 2020 12:46:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-10T12:46:28.804Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Jun 10, 2020 12:46:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-10T12:46:42.441Z: Workers have started successfully.
Jun 10, 2020 12:46:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-10T12:46:42.478Z: Workers have started successfully.
Jun 10, 2020 12:47:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-10T12:47:17.237Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Jun 10, 2020 12:47:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-10T12:47:17.919Z: Cleaning up.
Jun 10, 2020 12:47:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-10T12:47:18.504Z: Stopping worker pool...
Jun 10, 2020 12:49:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-10T12:49:14.509Z: Autoscaling: Resized worker pool from 5 to 0.
Jun 10, 2020 12:49:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-10T12:49:14.559Z: Worker pool stopped.
Jun 10, 2020 12:49:24 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-06-10_05_45_44-4510915318570474779 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): b31a2181-043b-4630-b6df-39b0e984bf45 and timestamp: 2020-06-10T12:49:24.472000000Z:
Metric: Value:
read_time 17.197
fields_read 4375276.0
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Jun 10, 2020 12:49:24 PM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 2 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.021 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.028 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 11,5,main]) completed. Took 3 mins 57.829 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 5s
104 actionable tasks: 70 executed, 34 from cache
Publishing build scan...
https://gradle.com/s/lahwgcyrthfoi
Stopped 1 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #609
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/609/display/redirect?page=changes>
Changes:
[robertwb] [BEAM-9577] Migrate PortablePipelineJarCreator to new artifact service.
[robertwb] spotless
[github] [BEAM-10144] Update PipelineOptions snippets for best practices (#11851)
------------------------------------------
[...truncated 295.04 KB...]
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:164)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Jun 10, 2020 6:45:39 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Jun 10, 2020 6:45:39 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery writeDisposition is set to: WRITE_EMPTY
Jun 10, 2020 6:45:39 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Jun 10, 2020 6:45:39 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Jun 10, 2020 6:45:39 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery writeDisposition is set to: WRITE_EMPTY
Jun 10, 2020 6:45:39 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Jun 10, 2020 6:45:39 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Jun 10, 2020 6:45:39 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Jun 10, 2020 6:45:40 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Jun 10, 2020 6:45:42 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 209 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Jun 10, 2020 6:45:42 AM org.apache.beam.runners.dataflow.util.PackageUtil$PackageAttributes forFileToStage
INFO: Staging custom dataflow-worker.jar as beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-d4cPD6dbFCQ231s02kkzxPYzjhNh1a2fA3Dzfn9yoJg.jar
Jun 10, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tests-cfAu3tQA35g5KY9h3UpvOv5BKyhFrqbluXF_hXyMitQ.jar
Jun 10, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-JcwETJThNjnTbBET4-phx94dYTJWvAvTvkpTh-C-aNI.jar
Jun 10, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-tests-cjFUu25moV0bov8GE7W2YbDs1Ghm7GPU7pXnrABbAOI.jar
Jun 10, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/java-fn-execution/build/libs/beam-runners-java-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-java-fn-execution-2.23.0-SNAPSHOT-Ju39JMc_Ei8Tg1CkJxJ2ZERwAHbmo7t10bpSP9rOqP0.jar
Jun 10, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-fn-execution-2.23.0-SNAPSHOT-cSiRS97siRj-Jhe5q8BMSoJSlYYU43aCPq8BZm5Bpoc.jar
Jun 10, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.23.0-SNAPSHOT-22U8CeK_T6bRdE2jqw8T7gmHUqpGWaxEUUDpzFF2ky8.jar
Jun 10, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.23.0-SNAPSHOT--d6LjUCTRd-iutoFIqJ1vU5xQTPBDSmTs-iTtnwD4sE.jar
Jun 10, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-tests--dwgWio3Sh8fnvNCbAXxX0bSw5oqM0mRiWcebps5i7s.jar
Jun 10, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /tmp/test950348666435161507.zip to gs://temp-storage-for-perf-tests/loadtests/staging/test-CcBgvC4mYZTe98ZXfqWAEMAL2mJilw1YBy2Re4G5ZNg.jar
Jun 10, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-nzXZYjZh2osa0GAmTLrZrVmcA5Mfguvck0zWqdrka3A.jar
Jun 10, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.23.0-SNAPSHOT-tests-CtraUzuN904gVAsgH_p0QFOvL6hdmkXmwqVIKCnkJSQ.jar
Jun 10, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-o8fiQ5fV4qLqwIN4bGRGRd5IuwED9xJko7FK8VOJWYU.jar
Jun 10, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-dS_F0DZNEjPJEPvAF3HASqXWIuPJpklyTnKPMVXQQrg.jar
Jun 10, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.23.0-SNAPSHOT-Ja-UU075MJEH8doAecTgs8OJREYVY2JsQ-L3px2dy3I.jar
Jun 10, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-d4cPD6dbFCQ231s02kkzxPYzjhNh1a2fA3Dzfn9yoJg.jar
Jun 10, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-WzYJFRHJm8llR1O0rgA8rWjHyN0ezI_F0Ej8n2bYwJc.jar
Jun 10, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-CjqqAyWrII71wGYaS_rqsxfKPSgYlO_x-EB2e9Qt08Y.jar
Jun 10, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.23.0-SNAPSHOT-orGLfkJvb0HRECY8uOtpeu2rCyjNEumZ7eLg7ZdE4qY.jar
Jun 10, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-tests-yQ1y5Qe1od9MoxdONo6Ndk-pDUkEMssmFvREZ87vNLA.jar
Jun 10, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-tests-xIm98IBgO-4zqgtgYz92f0UVuv8FDF1-VqRBtaR5Jhg.jar
Jun 10, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-g6fNIAj4HetNVkDSnJuNVmeMnRqTVbaKOgbUAuiBhbE.jar
Jun 10, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-tests-DEQKnBy8E2tiWZ62uj-BpODidJJEqBo9sK7MQLOV6UI.jar
Jun 10, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.23.0-SNAPSHOT-Wyc9pkyYQKV80kjvt-PYkaACpFKuljdAdfoU_w2ezzk.jar
Jun 10, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-tests-umy1Mh0kXcWM-xlEjwvmsTfsp2GBIyuxNISbdRkN0X0.jar
Jun 10, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-unshaded-IcG3F3d7zK5u2ckNskJRt5FC7dGlUk6dooWw6OL-S1w.jar
Jun 10, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT-BNBKJVAO9xUeq3DLeJ5Z0OcIHbqp1s30lWOzpMnL9eQ.jar
Jun 10, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-d4cPD6dbFCQ231s02kkzxPYzjhNh1a2fA3Dzfn9yoJg.jar
Jun 10, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT-hoqmvp-zaGBf7DoeRFSyA96YzIWR-Ji2acCheBuP3ug.jar
Jun 10, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.23.0-SNAPSHOT-pSIcnAgPlZ4pcDN2S26ScZmEtyxyMJGTZqK6StM6u7I.jar
Jun 10, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.23.0-SNAPSHOT-o71U7NOOSwmmJhmQ4UuEv733-lX9D1nFREIIldJBlCY.jar
Jun 10, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.23.0-SNAPSHOT-JrWPlxtrJGrpsQpx9IFUUVs_pg-MfCdkE5KQOjcGNQ0.jar
Jun 10, 2020 6:45:44 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-d4cPD6dbFCQ231s02kkzxPYzjhNh1a2fA3Dzfn9yoJg.jar&uploadType=resumable&upload_id=AAANsUlezbWqW5yv9C4OP-gUEbO2PMYajjYCTBCiYmwE9V2tKNRLZSf4xg94haA0qQaFqO4K33vHm8e6lLamKaMeqQ8.
Jun 10, 2020 6:45:44 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar>
java.io.IOException: Upload failed for 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-d4cPD6dbFCQ231s02kkzxPYzjhNh1a2fA3Dzfn9yoJg.jar'
at com.google.cloud.hadoop.util.BaseAbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(BaseAbstractGoogleAsyncWriteChannel.java:297)
at com.google.cloud.hadoop.util.BaseAbstractGoogleAsyncWriteChannel.close(BaseAbstractGoogleAsyncWriteChannel.java:214)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:221)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:168)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:152)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:136)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Jun 10, 2020 6:45:49 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-d4cPD6dbFCQ231s02kkzxPYzjhNh1a2fA3Dzfn9yoJg.jar
Jun 10, 2020 6:45:50 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 31 files newly uploaded in 7 seconds
Jun 10, 2020 6:45:50 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Jun 10, 2020 6:45:50 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Jun 10, 2020 6:45:50 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Jun 10, 2020 6:45:50 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Jun 10, 2020 6:45:50 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Jun 10, 2020 6:45:50 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <91367 bytes, hash e73437e6a9c451a0eaedab8cddeaa43db68765142d219cc67bd73984c0ab7d66> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-5zQ35qnEUaDq7auM3eqkPbaHZRQtIZzGe9c5hMCrfWY.pb
Jun 10, 2020 6:45:51 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.23.0-SNAPSHOT
Jun 10, 2020 6:45:52 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-06-09_23_45_51-3022411180043889511?project=apache-beam-testing
Jun 10, 2020 6:45:52 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-06-09_23_45_51-3022411180043889511
Jun 10, 2020 6:45:52 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-06-09_23_45_51-3022411180043889511
Jun 10, 2020 6:45:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-06-10T06:45:51.410Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Jun 10, 2020 6:45:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-10T06:45:58.747Z: Worker configuration: n1-standard-1 in us-central1-a.
Jun 10, 2020 6:46:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-10T06:45:59.534Z: Expanding CoGroupByKey operations into optimizable parts.
Jun 10, 2020 6:46:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-10T06:45:59.586Z: Expanding GroupByKey operations into optimizable parts.
Jun 10, 2020 6:46:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-10T06:45:59.620Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jun 10, 2020 6:46:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-10T06:45:59.715Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jun 10, 2020 6:46:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-10T06:45:59.758Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Jun 10, 2020 6:46:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-10T06:45:59.788Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Jun 10, 2020 6:46:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-10T06:45:59.825Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Jun 10, 2020 6:46:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-10T06:46:00.407Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Jun 10, 2020 6:46:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-10T06:46:00.511Z: Starting 5 workers in us-central1-a...
Jun 10, 2020 6:46:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-06-10T06:46:08.787Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jun 10, 2020 6:46:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-10T06:46:38.047Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Jun 10, 2020 6:46:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-10T06:46:57.987Z: Workers have started successfully.
Jun 10, 2020 6:46:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-10T06:46:58.019Z: Workers have started successfully.
Jun 10, 2020 6:47:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-10T06:47:30.091Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Jun 10, 2020 6:47:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-10T06:47:30.337Z: Cleaning up.
Jun 10, 2020 6:47:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-10T06:47:30.440Z: Stopping worker pool...
Jun 10, 2020 6:49:14 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-10T06:49:13.008Z: Autoscaling: Resized worker pool from 5 to 0.
Jun 10, 2020 6:49:14 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-10T06:49:13.054Z: Worker pool stopped.
Jun 10, 2020 6:49:19 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-06-09_23_45_51-3022411180043889511 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): c6597824-0472-48ed-a54b-cdc324ab2f9b and timestamp: 2020-06-10T06:49:19.382000000Z:
Metric: Value:
fields_read 4375276.0
read_time 15.117
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Jun 10, 2020 6:49:19 AM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 2 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.023 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.03 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 3,5,main]) completed. Took 3 mins 47.982 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 58s
104 actionable tasks: 71 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/46kesidx7cdqu
Stopped 1 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #608
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/608/display/redirect?page=changes>
Changes:
[chuck.yang] Support STRUCT, FLOAT64, INT64 BigQuery types
[github] fixup! roll back changes (#11958)
[github] [ BEAM-3788] Updates kafka.py pydocs (#11928)
[github] [BEAM-8828] Added BigQueryTableProvider WriteDisposition configuration
[github] Prototype schema-inferring Row constructor. (#11901)
------------------------------------------
[...truncated 294.60 KB...]
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:164)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Jun 10, 2020 12:48:30 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Jun 10, 2020 12:48:30 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery writeDisposition is set to: WRITE_EMPTY
Jun 10, 2020 12:48:30 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Jun 10, 2020 12:48:30 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Jun 10, 2020 12:48:30 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery writeDisposition is set to: WRITE_EMPTY
Jun 10, 2020 12:48:30 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Jun 10, 2020 12:48:31 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Jun 10, 2020 12:48:31 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Jun 10, 2020 12:48:31 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Jun 10, 2020 12:48:33 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 209 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Jun 10, 2020 12:48:33 AM org.apache.beam.runners.dataflow.util.PackageUtil$PackageAttributes forFileToStage
INFO: Staging custom dataflow-worker.jar as beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-kefYbFjelll-Wl-jWfbaTdi9AMqs5XYZao29uYQegLU.jar
Jun 10, 2020 12:48:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.23.0-SNAPSHOT-JN58M8288nbqZKzxVDCcGtCGZ9Y3j-u-TGd1Bqi1ELw.jar
Jun 10, 2020 12:48:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-tests-zYcv4dmosWmxK7tzK0ATPMQC30mH3MSTLFbUsgRqWe0.jar
Jun 10, 2020 12:48:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.23.0-SNAPSHOT-tests-nBb80RpPzQzSaixX1esCJnGNDh95-24bIO9uncg_3Js.jar
Jun 10, 2020 12:48:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.23.0-SNAPSHOT-t-tOIP9yY-0Q7SUY-1PodDoPu5KbFByP3h8VCOMiDYc.jar
Jun 10, 2020 12:48:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT-QZIoaHteU6xOfwRuebCNyd6oaNlOUQIl9zjZ8jwMd1A.jar
Jun 10, 2020 12:48:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-iuXOiIU84QYVxNuQjBEWyB02DwyTm2y-2Slfjw5xXp8.jar
Jun 10, 2020 12:48:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-3g5eOmtOzqJD0tgbkrt7waraGma2oq5ZdZSMe326Cxc.jar
Jun 10, 2020 12:48:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.23.0-SNAPSHOT-humKCbplNSTHljvDuCAVGiOtsTP9ndQgXwxhuGwwMQE.jar
Jun 10, 2020 12:48:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-zsdk2adJLV6CAG1By0lt90P-FuI7aEdIl2FN_dbGrC0.jar
Jun 10, 2020 12:48:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-tests-wOzXllbEX5GtWVRu58FL-RK77rbiuAJRSa6b0QBfM2w.jar
Jun 10, 2020 12:48:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/java-fn-execution/build/libs/beam-runners-java-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-java-fn-execution-2.23.0-SNAPSHOT-hItGfibrd5sgjHMQEaa7ATsrt_8Ww0fcgr5JleiZY7s.jar
Jun 10, 2020 12:48:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-tests-C6_62YR-5nvStvf4jkTb1hDesxC7k7uzcCitGqqd814.jar
Jun 10, 2020 12:48:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT-Yyn_rpzxeWTuzRDc4QOKHkViKLofXslgfqKppJMQx_U.jar
Jun 10, 2020 12:48:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.23.0-SNAPSHOT-0fDpWtW7qofs9NuMfh6SgHcjZ66ZJ2tISmIAfUbEheA.jar
Jun 10, 2020 12:48:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-unshaded-o_DBa5MYsVOCqcJv2V8QPWeshRj5oYmfcIUMyTEgyp0.jar
Jun 10, 2020 12:48:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-tests-QRN4NXnqZUbL-Y1PWfBvUlgLfkSb2s6Hx4MNVZ6HatI.jar
Jun 10, 2020 12:48:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-Z6xvUaylH-xxqtjE5gwQXgmWNSVERHRWW9LVhBcySOM.jar
Jun 10, 2020 12:48:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-tests-i3swTOfuDWKbV3K-q5YIXhFvza1OCaO5RpFsP5QP_rI.jar
Jun 10, 2020 12:48:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-kefYbFjelll-Wl-jWfbaTdi9AMqs5XYZao29uYQegLU.jar
Jun 10, 2020 12:48:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-JPukyofBSZI0mE6SB82k-wf-d5XF61TgNKIqcDDxm_U.jar
Jun 10, 2020 12:48:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.23.0-SNAPSHOT-3EYlGeJo8dWdbpcgbni1iBpDNGFPmtj1PtjXsfIqTB0.jar
Jun 10, 2020 12:48:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tests-4dMkzSznX-XmXqv5VsX6X5gk-XITUL95HfOeql4GeEU.jar
Jun 10, 2020 12:48:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-fn-execution-2.23.0-SNAPSHOT-q26cQHn8YZjBwOFQbEYlPnYsH08N6_dxlC7CmYClEws.jar
Jun 10, 2020 12:48:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-Wh-Lqz-4WRZLsSkR-3rJj31yJtoK_r7cve-Wg3kvlP4.jar
Jun 10, 2020 12:48:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-_cvrWEcHk9RMIGOTU2_t9rYkft-AOZ0OC-CfdrQYG7Q.jar
Jun 10, 2020 12:48:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-tests-gUOUQgneSUiQ345nDt0ViTFZCmM0bLJ12Oz0JHcZT80.jar
Jun 10, 2020 12:48:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /tmp/test6404816247618250522.zip to gs://temp-storage-for-perf-tests/loadtests/staging/test-GnKZ92a6NBz92GGVQA9-Hubgr4S6mKdliWqKNplhifc.jar
Jun 10, 2020 12:48:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-kefYbFjelll-Wl-jWfbaTdi9AMqs5XYZao29uYQegLU.jar
Jun 10, 2020 12:48:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.23.0-SNAPSHOT-MbdViOJVXQ668wGZwmZkE2C-doZCdobyL0rExGPi2ec.jar
Jun 10, 2020 12:48:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.23.0-SNAPSHOT-7b__XYQjYFncnSGFQs_tfAchneC8p8UU7bnLtAwJvGs.jar
Jun 10, 2020 12:48:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.23.0-SNAPSHOT-wh-hZj7fKaNJXP3DiGmvBBpuPzlOP32UGR1hx6brkEM.jar
Jun 10, 2020 12:48:35 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-kefYbFjelll-Wl-jWfbaTdi9AMqs5XYZao29uYQegLU.jar&uploadType=resumable&upload_id=AAANsUlNSpw7pQQr_GzS8Gh3KQmmXavACk0SiH3EcFc-iNt4QeWS3r3XKOw_q11PQH-FoTXSvJdVxre3tZFTeSrgUro.
Jun 10, 2020 12:48:35 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar>
java.io.IOException: Upload failed for 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-kefYbFjelll-Wl-jWfbaTdi9AMqs5XYZao29uYQegLU.jar'
at com.google.cloud.hadoop.util.BaseAbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(BaseAbstractGoogleAsyncWriteChannel.java:297)
at com.google.cloud.hadoop.util.BaseAbstractGoogleAsyncWriteChannel.close(BaseAbstractGoogleAsyncWriteChannel.java:214)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:221)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:168)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:152)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:136)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Jun 10, 2020 12:48:39 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-kefYbFjelll-Wl-jWfbaTdi9AMqs5XYZao29uYQegLU.jar
Jun 10, 2020 12:48:40 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 31 files newly uploaded in 6 seconds
Jun 10, 2020 12:48:40 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Jun 10, 2020 12:48:40 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Jun 10, 2020 12:48:40 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Jun 10, 2020 12:48:40 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Jun 10, 2020 12:48:40 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Jun 10, 2020 12:48:40 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <91368 bytes, hash 02712cfba02b95d6f234e827c54d02b2a6f8f5b954b95a0d800e39aafbeabc81> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-AnEs-6ArldbyNOgnxU0Csqb49blUuVoNgA45qvvqvIE.pb
Jun 10, 2020 12:48:41 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.23.0-SNAPSHOT
Jun 10, 2020 12:48:42 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-06-09_17_48_41-7707399744199220533?project=apache-beam-testing
Jun 10, 2020 12:48:42 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-06-09_17_48_41-7707399744199220533
Jun 10, 2020 12:48:42 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-06-09_17_48_41-7707399744199220533
Jun 10, 2020 12:48:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-06-10T00:48:41.170Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Jun 10, 2020 12:48:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-10T00:48:48.527Z: Worker configuration: n1-standard-1 in us-central1-a.
Jun 10, 2020 12:48:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-10T00:48:49.370Z: Expanding CoGroupByKey operations into optimizable parts.
Jun 10, 2020 12:48:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-10T00:48:49.420Z: Expanding GroupByKey operations into optimizable parts.
Jun 10, 2020 12:48:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-10T00:48:49.460Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jun 10, 2020 12:48:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-10T00:48:49.553Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jun 10, 2020 12:48:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-10T00:48:49.589Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Jun 10, 2020 12:48:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-10T00:48:49.625Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Jun 10, 2020 12:48:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-10T00:48:49.662Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Jun 10, 2020 12:48:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-10T00:48:49.999Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Jun 10, 2020 12:48:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-10T00:48:50.086Z: Starting 5 workers in us-central1-a...
Jun 10, 2020 12:49:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-06-10T00:49:18.784Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jun 10, 2020 12:49:24 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-10T00:49:23.730Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Jun 10, 2020 12:49:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-10T00:49:40.437Z: Workers have started successfully.
Jun 10, 2020 12:49:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-10T00:49:40.473Z: Workers have started successfully.
Jun 10, 2020 12:50:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-10T00:50:17.243Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Jun 10, 2020 12:50:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-10T00:50:17.475Z: Cleaning up.
Jun 10, 2020 12:50:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-10T00:50:17.568Z: Stopping worker pool...
Jun 10, 2020 12:52:14 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-10T00:52:13.527Z: Autoscaling: Resized worker pool from 5 to 0.
Jun 10, 2020 12:52:14 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-10T00:52:13.575Z: Worker pool stopped.
Jun 10, 2020 12:52:19 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-06-09_17_48_41-7707399744199220533 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 3978cd6a-6337-475c-be32-b33cced0ad58 and timestamp: 2020-06-10T00:52:19.628000000Z:
Metric: Value:
fields_read 4375276.0
read_time 19.099
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Jun 10, 2020 12:52:20 AM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 2 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.025 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.034 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 11,5,main]) completed. Took 3 mins 57.81 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 6s
104 actionable tasks: 70 executed, 34 from cache
Publishing build scan...
https://gradle.com/s/fqtx75yfaitwy
Stopped 1 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #607
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/607/display/redirect?page=changes>
Changes:
[robertwb] [BEAM-6215] Additional tests for FlatMap label.
[github] [BEAM-2939] Fix splittable DoFn lifecycle. (#11941)
------------------------------------------
[...truncated 292.46 KB...]
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:164)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Jun 09, 2020 6:51:51 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Jun 09, 2020 6:51:51 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Jun 09, 2020 6:51:51 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Jun 09, 2020 6:51:51 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Jun 09, 2020 6:51:51 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Jun 09, 2020 6:51:51 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Jun 09, 2020 6:51:52 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Jun 09, 2020 6:51:53 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 209 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Jun 09, 2020 6:51:54 PM org.apache.beam.runners.dataflow.util.PackageUtil$PackageAttributes forFileToStage
INFO: Staging custom dataflow-worker.jar as beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-Nap3owiwtjFicCxPsHdSMzFVukr9Gwh61Ps5qlgbv-c.jar
Jun 09, 2020 6:51:54 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT-Dr0A3JyuXMgoW1CmWKHIy0bpF9xrt45YuPSoixta0cE.jar
Jun 09, 2020 6:51:54 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-Z74Web-eLRUVbjlOfPWsbwrxWf4sHk_Qu6DCF2eK_Ps.jar
Jun 09, 2020 6:51:54 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-tests-tbu2PY7o5HwE8mviPv_8wF7qDmEPFYRtuoHwh0fIygQ.jar
Jun 09, 2020 6:51:54 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tests-cVtvgyY8TqLNGpnClNmU-jEBNBPvhqB_TQaXFgGVOho.jar
Jun 09, 2020 6:51:54 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-tests-S7bhnF55gXpH6a6CKhq1W65M7I9GE57H8ndmOtBYsC4.jar
Jun 09, 2020 6:51:54 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-Nap3owiwtjFicCxPsHdSMzFVukr9Gwh61Ps5qlgbv-c.jar
Jun 09, 2020 6:51:54 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.23.0-SNAPSHOT-nJuFoIla_eUKqyLwuCYYN_a30X_qouTZ4GHMxm_klZI.jar
Jun 09, 2020 6:51:54 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-tests-aTCBEhNRtnA5jceRW89XGh1krbzoG2HvO5TkGEAgOSg.jar
Jun 09, 2020 6:51:54 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-6c65bdZj7l6btzZ1EG2v8lfZJ7qYbbHuUkhAXL0TbBc.jar
Jun 09, 2020 6:51:54 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.23.0-SNAPSHOT-PpeeksToroQMu-66STQ3aazkxeYgvF9kzw8uuRbQldg.jar
Jun 09, 2020 6:51:54 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.23.0-SNAPSHOT-L6I9iZwM54WQsWSIOhk3GwG64-qE63QQT94eYgqSBZI.jar
Jun 09, 2020 6:51:54 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-HkV7pbzCbBU2fZ-jXR0VWciWkn4vDXvQdXHkVYAYr28.jar
Jun 09, 2020 6:51:54 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.23.0-SNAPSHOT-bYUeNHfUG4xQujep3Wp46Pm1i7zV4lfZqQFuZMjq128.jar
Jun 09, 2020 6:51:54 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.23.0-SNAPSHOT-5Sn_EwcLn4rAjmuqkogq4S28e18Nl0GprKlKK1nqW9s.jar
Jun 09, 2020 6:51:54 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-fBV79VV9CAy5wlAQ09rss0DY-aup-XuPNL7TqGIfkEU.jar
Jun 09, 2020 6:51:54 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-1Nr1E5D79kr-xYgznt_RBJ51z1dfEbWayv8ZH6EbDus.jar
Jun 09, 2020 6:51:54 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-rrvzJKIfemjQw5yw5lFyYqskBims0WzFQiUQ__HhtEY.jar
Jun 09, 2020 6:51:54 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-tests-UbF5k3shHJuIo3P2GiDmFJXwZtB6kl-g7kCH66E8O_c.jar
Jun 09, 2020 6:51:54 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /tmp/test7575231699018744383.zip to gs://temp-storage-for-perf-tests/loadtests/staging/test-gS4PGMdRaSGEzd3_tFXJznxhCYiw6IzX03sMk13JJHs.jar
Jun 09, 2020 6:51:54 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-unshaded-BGKLm5bLu0cBOpY436Uva5JG3VafdrMMverN-ADB-kw.jar
Jun 09, 2020 6:51:54 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-tests-_k5b5QY618jI1iI-K7Z9RLhp_mJwnh9xKU8giAQ75fk.jar
Jun 09, 2020 6:51:54 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-93Y7OhjYItuUhTfT1DihU1LI-pwlKkMDl8oLmpjbzp0.jar
Jun 09, 2020 6:51:54 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.23.0-SNAPSHOT-tests-LaURU7CEldL53I5_-1eAUWPmNifNSFNJw7DEHLf7SBA.jar
Jun 09, 2020 6:51:54 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-tests-cl8EFq-vh8vyLXKJ2bxs3mZTtnWCMUZd9wNLUDga92g.jar
Jun 09, 2020 6:51:54 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-fn-execution-2.23.0-SNAPSHOT-ygCmf3onssPCfb1GX2jH0QpALWAZM_ZCOwigVH_bzPM.jar
Jun 09, 2020 6:51:54 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT-TkQg7YihgBrI66UtQJxj9wu7SOfHCxA-FOHdt1xkC2s.jar
Jun 09, 2020 6:51:54 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/java-fn-execution/build/libs/beam-runners-java-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-java-fn-execution-2.23.0-SNAPSHOT-gf307kamuv71tQvX9X7GXDaCi8FE9bk3w0rjclxh7Xo.jar
Jun 09, 2020 6:51:54 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-Nap3owiwtjFicCxPsHdSMzFVukr9Gwh61Ps5qlgbv-c.jar
Jun 09, 2020 6:51:54 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.23.0-SNAPSHOT-Qha-f2bgOGpgkLjACuANwHJTwPqZeqxHybqKT1kbp2w.jar
Jun 09, 2020 6:51:54 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.23.0-SNAPSHOT-HdcjnUSRJhh4mO_x9F1I5wNsRT-v8M5wQPK0M1T3CgY.jar
Jun 09, 2020 6:51:54 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.23.0-SNAPSHOT-jMPzusCivLruoBkm_TVwnSQu6a7D1cGE9shaJxtbzLE.jar
Jun 09, 2020 6:51:55 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-Nap3owiwtjFicCxPsHdSMzFVukr9Gwh61Ps5qlgbv-c.jar&uploadType=resumable&upload_id=AAANsUlpGm9wA0SKGpv-v7opVKfauEzCgWoxraF1q6ZDuYy7lnBVpyhVMi1Uhpu8bMVxPPSkqLdahGThK2SB5qDwiB-9a8j7gQ.
Jun 09, 2020 6:51:55 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar>
java.io.IOException: Upload failed for 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-Nap3owiwtjFicCxPsHdSMzFVukr9Gwh61Ps5qlgbv-c.jar'
at com.google.cloud.hadoop.util.BaseAbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(BaseAbstractGoogleAsyncWriteChannel.java:297)
at com.google.cloud.hadoop.util.BaseAbstractGoogleAsyncWriteChannel.close(BaseAbstractGoogleAsyncWriteChannel.java:214)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:221)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:168)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:152)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:136)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Jun 09, 2020 6:52:02 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-Nap3owiwtjFicCxPsHdSMzFVukr9Gwh61Ps5qlgbv-c.jar
Jun 09, 2020 6:52:02 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 31 files newly uploaded in 9 seconds
Jun 09, 2020 6:52:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Jun 09, 2020 6:52:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Jun 09, 2020 6:52:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Jun 09, 2020 6:52:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Jun 09, 2020 6:52:03 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Jun 09, 2020 6:52:03 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <91292 bytes, hash 81d71900250b9bcade860a2daea6b0428975a0b9b7a1e85d984b6451c68a91a6> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-gdcZACULm8rehgotrqawQol1oLm3oehdmEtkUcaKkaY.pb
Jun 09, 2020 6:52:03 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.23.0-SNAPSHOT
Jun 09, 2020 6:52:04 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-06-09_11_52_03-14525815750164960731?project=apache-beam-testing
Jun 09, 2020 6:52:04 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-06-09_11_52_03-14525815750164960731
Jun 09, 2020 6:52:04 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-06-09_11_52_03-14525815750164960731
Jun 09, 2020 6:52:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-06-09T18:52:03.897Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Jun 09, 2020 6:52:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-09T18:52:11.634Z: Worker configuration: n1-standard-1 in us-central1-a.
Jun 09, 2020 6:52:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-09T18:52:12.542Z: Expanding CoGroupByKey operations into optimizable parts.
Jun 09, 2020 6:52:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-09T18:52:12.572Z: Expanding GroupByKey operations into optimizable parts.
Jun 09, 2020 6:52:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-09T18:52:12.605Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jun 09, 2020 6:52:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-09T18:52:12.667Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jun 09, 2020 6:52:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-09T18:52:12.694Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Jun 09, 2020 6:52:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-09T18:52:12.725Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Jun 09, 2020 6:52:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-09T18:52:12.753Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Jun 09, 2020 6:52:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-09T18:52:13.442Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Jun 09, 2020 6:52:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-09T18:52:13.523Z: Starting 5 workers in us-central1-a...
Jun 09, 2020 6:52:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-06-09T18:52:24.794Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jun 09, 2020 6:52:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-09T18:52:39.031Z: Autoscaling: Raised the number of workers to 2 based on the rate of progress in the currently running stage(s).
Jun 09, 2020 6:52:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-09T18:52:39.110Z: Resized worker pool to 2, though goal was 5. This could be a quota issue.
Jun 09, 2020 6:52:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-09T18:52:44.444Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Jun 09, 2020 6:52:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-09T18:52:58.034Z: Workers have started successfully.
Jun 09, 2020 6:52:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-09T18:52:58.060Z: Workers have started successfully.
Jun 09, 2020 6:53:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-09T18:53:26.456Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Jun 09, 2020 6:53:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-09T18:53:26.661Z: Cleaning up.
Jun 09, 2020 6:53:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-09T18:53:26.756Z: Stopping worker pool...
Jun 09, 2020 6:55:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-09T18:55:21.386Z: Autoscaling: Resized worker pool from 5 to 0.
Jun 09, 2020 6:55:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-09T18:55:21.430Z: Worker pool stopped.
Jun 09, 2020 6:55:26 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-06-09_11_52_03-14525815750164960731 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 3ff19cb6-5d54-479f-8e84-a149a35a8e27 and timestamp: 2020-06-09T18:55:26.882000000Z:
Metric: Value:
fields_read 4375276.0
read_time 11.259
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Jun 09, 2020 6:55:27 PM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.027 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.054 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 7,5,main]) completed. Took 3 mins 44.464 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 36s
104 actionable tasks: 68 executed, 36 from cache
Publishing build scan...
https://gradle.com/s/zrexro25276na
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #606
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/606/display/redirect?page=changes>
Changes:
[github] Set LabelDetectionConfig.model to “builtin/latest” (#11946)
------------------------------------------
[...truncated 293.42 KB...]
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:164)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Jun 09, 2020 12:45:35 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Jun 09, 2020 12:45:35 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Jun 09, 2020 12:45:35 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Jun 09, 2020 12:45:35 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Jun 09, 2020 12:45:35 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Jun 09, 2020 12:45:36 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Jun 09, 2020 12:45:36 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Jun 09, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 209 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Jun 09, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil$PackageAttributes forFileToStage
INFO: Staging custom dataflow-worker.jar as beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-TrDowNBSpt72djG-yWIFbwP4N7ebldkWmRqx7E6CteU.jar
Jun 09, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-TrDowNBSpt72djG-yWIFbwP4N7ebldkWmRqx7E6CteU.jar
Jun 09, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.23.0-SNAPSHOT-tests-nBwrauZK1IKg9DPb3F5RGZUUT57nGxLCLEEWciDWkqc.jar
Jun 09, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-Mmjx585dDyB3k0-p3ObC83kvQIJEqfaUPm1THxrWFMQ.jar
Jun 09, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT-Dc5rtDa-bBYDbTY8CWtKmu6BOhtU13wcnGa3ggllT8I.jar
Jun 09, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-tests-74kvACErLcCMFiDckxVrxCHvsckdRn9K3ZG7KC13rZY.jar
Jun 09, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-tests-DHDysV4kcv9LvU3w4IqNNYbgTJCScDj9IB13krL8IkQ.jar
Jun 09, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.23.0-SNAPSHOT-pHRZJpfha8yPFf0wPS1sKKW_22VQf_lCLKobJxIB__Y.jar
Jun 09, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.23.0-SNAPSHOT--BEGxhWpXAMNccjLJrMlrIIMVN4U1o3Uo3EOZMT05f4.jar
Jun 09, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-tests-3Kil_c0LiVM28ZQhFSOif_fCWTDsXq3UAPi-PZCmBoM.jar
Jun 09, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-tests-bMTt2Udmzw_dt_bhrhOiZ_9RG8k6V5GS9ld9tJRKBeE.jar
Jun 09, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.23.0-SNAPSHOT-ALn2sFWppvJr4Vv7_0p1WLQU3_hI0uJQ_RfguGpnfi0.jar
Jun 09, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tests-b6lEYkSIOWU-yVqMkz4jCZ8JKbidUMLJDqmTBN6OBYA.jar
Jun 09, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-EoyjxplVlQ-78fT7dCIu_eyaCq9C8GYUtY9CsCfKF9Q.jar
Jun 09, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT-_2dpEInQMDZf6OnjubseESHHVsdQR68e5CkUQeGmh5k.jar
Jun 09, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.23.0-SNAPSHOT-DP7ktbtHLnt8UvxsKcrcMTd4onoHIJWRAh4PR2ZbdnY.jar
Jun 09, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-fn-execution-2.23.0-SNAPSHOT-w4KG_ICR2k0MKNsw6kH2lcsmtoXEAP32oiclJej1-6M.jar
Jun 09, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.23.0-SNAPSHOT-NBXgg-8UKruxx3qhH7thkOWCU4RFIbzTkyUb1QnDER0.jar
Jun 09, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /tmp/test1871704927542397592.zip to gs://temp-storage-for-perf-tests/loadtests/staging/test-EIkNS_gmb9XBouGeJ3yvixjmj82pf2uMPaMCc8Q8OqU.jar
Jun 09, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/java-fn-execution/build/libs/beam-runners-java-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-java-fn-execution-2.23.0-SNAPSHOT-WyCo0oIGSHfFHlulXIY3CZxRSUa7KzngYQzsNgsLzVg.jar
Jun 09, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-uC7oDF7HJjzcW65CTC0Yk2cuYwZ4IKCBp_otJvGxpKo.jar
Jun 09, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-QxrX-g_lwqvlsKeF5j-ROJjiHZMaczfQVbIWd3GTXeg.jar
Jun 09, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-tests-M4V-Aif_eTOyYLpIsPon-R44odpPkCV74QQlKJNqTOM.jar
Jun 09, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-AVbv47JLsaZkUabyGp0-xiadhPKczzx6KLm_tyVMO3s.jar
Jun 09, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.23.0-SNAPSHOT-SUnlzbnbmks4-A1Qge_qdQCJ-XrO8llaW4xvpekURsk.jar
Jun 09, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-TrDowNBSpt72djG-yWIFbwP4N7ebldkWmRqx7E6CteU.jar
Jun 09, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-unshaded-QUARvgyaV9vSyATkrynpTWJFANfcPl5aV_jWvLxn7aY.jar
Jun 09, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-tests-7fJpOe4Cwk7PIWHT7gwp_PuYtKnxQ4shnAGKUDoaWWc.jar
Jun 09, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-Ye1jyeCrYf-lk0rqZmfjxDk7C0NzJdq3aYM0QCiuZSw.jar
Jun 09, 2020 12:45:39 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.23.0-SNAPSHOT-HHo3799VYTh0dUydJCOxTd_ZsS7ZKlMGCQg5FWoYlfQ.jar
Jun 09, 2020 12:45:39 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.23.0-SNAPSHOT-xJK_xkHNyaVTLELueOGURKsC2nYEB8e4MkJCDVkEAjc.jar
Jun 09, 2020 12:45:39 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.23.0-SNAPSHOT-NX4bI-qjN9Uibf3S_Y6zXBhxfcd_mz10SLXzFm6CKmA.jar
Jun 09, 2020 12:45:39 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-TrDowNBSpt72djG-yWIFbwP4N7ebldkWmRqx7E6CteU.jar&uploadType=resumable&upload_id=AAANsUnknew07xJIspBPlqD7sjKOcY2jIPCE4g0tztEm7ujCQID6ReXDRFnKsD9TceIdxVChOJA_KDe2zhBpZK1Vuco.
Jun 09, 2020 12:45:39 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar>
java.io.IOException: Upload failed for 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-TrDowNBSpt72djG-yWIFbwP4N7ebldkWmRqx7E6CteU.jar'
at com.google.cloud.hadoop.util.BaseAbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(BaseAbstractGoogleAsyncWriteChannel.java:297)
at com.google.cloud.hadoop.util.BaseAbstractGoogleAsyncWriteChannel.close(BaseAbstractGoogleAsyncWriteChannel.java:214)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:221)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:168)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:152)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:136)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Jun 09, 2020 12:45:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-TrDowNBSpt72djG-yWIFbwP4N7ebldkWmRqx7E6CteU.jar
Jun 09, 2020 12:45:44 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 31 files newly uploaded in 6 seconds
Jun 09, 2020 12:45:44 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Jun 09, 2020 12:45:45 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Jun 09, 2020 12:45:45 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Jun 09, 2020 12:45:45 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Jun 09, 2020 12:45:45 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Jun 09, 2020 12:45:45 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <91292 bytes, hash c68958cbd12ac4d5f1eefdbbf6bbfdd545ab6b6d27339e9785a9b184da0bcf49> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-xolYy9EqxNXx7v279rv91UWra20nM56XhamxhNoLz0k.pb
Jun 09, 2020 12:45:45 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.23.0-SNAPSHOT
Jun 09, 2020 12:45:47 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-06-09_05_45_45-1234718410829936561?project=apache-beam-testing
Jun 09, 2020 12:45:47 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-06-09_05_45_45-1234718410829936561
Jun 09, 2020 12:45:47 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-06-09_05_45_45-1234718410829936561
Jun 09, 2020 12:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-06-09T12:45:45.847Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Jun 09, 2020 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-09T12:45:53.711Z: Worker configuration: n1-standard-1 in us-central1-a.
Jun 09, 2020 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-09T12:45:54.617Z: Expanding CoGroupByKey operations into optimizable parts.
Jun 09, 2020 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-09T12:45:54.655Z: Expanding GroupByKey operations into optimizable parts.
Jun 09, 2020 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-09T12:45:54.697Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jun 09, 2020 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-09T12:45:54.792Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jun 09, 2020 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-09T12:45:54.831Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Jun 09, 2020 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-09T12:45:54.865Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Jun 09, 2020 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-09T12:45:54.893Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Jun 09, 2020 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-09T12:45:55.381Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Jun 09, 2020 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-09T12:45:55.456Z: Starting 5 workers in us-central1-a...
Jun 09, 2020 12:46:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-06-09T12:46:09.830Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jun 09, 2020 12:46:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-09T12:46:24.546Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Jun 09, 2020 12:46:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-09T12:46:41.348Z: Workers have started successfully.
Jun 09, 2020 12:46:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-09T12:46:41.382Z: Workers have started successfully.
Jun 09, 2020 12:47:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-09T12:47:19.408Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Jun 09, 2020 12:47:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-09T12:47:19.614Z: Cleaning up.
Jun 09, 2020 12:47:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-09T12:47:19.681Z: Stopping worker pool...
Jun 09, 2020 12:49:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-09T12:49:31.023Z: Autoscaling: Resized worker pool from 5 to 0.
Jun 09, 2020 12:49:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-09T12:49:31.068Z: Worker pool stopped.
Jun 09, 2020 12:49:37 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-06-09_05_45_45-1234718410829936561 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 57bacfa0-97a8-469d-a778-5383a32d4726 and timestamp: 2020-06-09T12:49:37.322000000Z:
Metric: Value:
read_time 19.688
fields_read 4375276.0
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Jun 09, 2020 12:49:37 PM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 2 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.023 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.032 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 2,5,main]) completed. Took 4 mins 10.285 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 17s
104 actionable tasks: 70 executed, 34 from cache
Publishing build scan...
https://gradle.com/s/cawykddgff46o
Stopped 1 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #605
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/605/display/redirect>
Changes:
------------------------------------------
[...truncated 295.44 KB...]
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:164)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Jun 09, 2020 6:45:43 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Jun 09, 2020 6:45:44 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Jun 09, 2020 6:45:44 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Jun 09, 2020 6:45:44 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Jun 09, 2020 6:45:44 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Jun 09, 2020 6:45:44 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Jun 09, 2020 6:45:45 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Jun 09, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 209 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Jun 09, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil$PackageAttributes forFileToStage
INFO: Staging custom dataflow-worker.jar as beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-1cVOvTNqLLFi2Z0vsfX2SQTTTZLqnGT-tccxJPdgLkQ.jar
Jun 09, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-tests--3qbyTn_2UF4LOLV-779Q3mm5_xX1zJGWo-NRgeVFRk.jar
Jun 09, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-tests-8F1lT-xvPm6RhD6cAWXxPy2qPAH00pJdCX4Euijrx2U.jar
Jun 09, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tests-qnoDGvI_NQDMa15V-RWx6yF1MZE7IHorZ62K7zYzCZw.jar
Jun 09, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT-Nctx-wYj1pW26CToMzcIZt0jS0bm8Q8jt_xa0vE7aG4.jar
Jun 09, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-tests-_CG5jEgoePRZf_NxAB4gll6SnRcjk8eX0700GcYaDXo.jar
Jun 09, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-fn-execution-2.23.0-SNAPSHOT-p0aaJipHTqh8oqj24FUwdJQwwJQysibsqOpuVm4VaaI.jar
Jun 09, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-LL63ks-h-3re86a5BUufzDzSuvKGKVnHE-F-FuQ4vXI.jar
Jun 09, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-sK0maVZZKjVps2hqGmVJ-3Om-f3vneyK0XqgpKqQZGg.jar
Jun 09, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.23.0-SNAPSHOT-FoFibXEsk8xm2lz5Rcp5higefUQYhkTI5rs9zUVDnr0.jar
Jun 09, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.23.0-SNAPSHOT-89mIfW5-9qab11XNW-_gbggnUwL8S-tHrioXer0geSA.jar
Jun 09, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.23.0-SNAPSHOT-Vi_tirxep04dtyYWaXz5kCmtBWX_KCsKENUaxSR5yAY.jar
Jun 09, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-LtaJ7_veWndvzzajcrbkYvb65GOUl498Q8cmCOkiYs4.jar
Jun 09, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-tests-6ktPOgCjcYq7xqzIvDUGCqFlH0R2luSSV7ogqt3USng.jar
Jun 09, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-HdyEOwymPKwmtG8hiPApufZOMiKjAiEGFQ1eHXzss1A.jar
Jun 09, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-MWQ4o7VGjfZNBZvrMR5vHxJLafrs4_aRR6K-PmF9qVI.jar
Jun 09, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-cFyRLlbRPy5DIOR9_GUj6W-AlB3gWN775MySK0YivL8.jar
Jun 09, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-tests-R3Oh2eonb-bd8Weyx0kI9MpZr1-4f5hgzetRkveG0Yw.jar
Jun 09, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.23.0-SNAPSHOT-tests-AjPvsk7Bez3iKG_RM37hAnZPNv4KicaVg6CzOBw22dU.jar
Jun 09, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.23.0-SNAPSHOT-twLR3mmT1cNy_mqZ8RF5Uf0XOJWyIXHIwKRyhFCReik.jar
Jun 09, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-1cVOvTNqLLFi2Z0vsfX2SQTTTZLqnGT-tccxJPdgLkQ.jar
Jun 09, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT-grkO0g1b4ooFT3JiV3D3Es5REHUSuElqY7IHWTt9s1M.jar
Jun 09, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-QBSby358Lk1Afc0uoo9Lkf-cqOWM6lNFDVnv3wjCec0.jar
Jun 09, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/java-fn-execution/build/libs/beam-runners-java-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-java-fn-execution-2.23.0-SNAPSHOT-ASNAV96QwXT9rxaF1fQEe8ywJF_gHlP35mGRcHSzokw.jar
Jun 09, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /tmp/test6234272118311264893.zip to gs://temp-storage-for-perf-tests/loadtests/staging/test-k_MEqlDnXsvAtN9rrkKAYjgMCKdsYZoXzYaAf4hItZM.jar
Jun 09, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-unshaded-99LcbhjAD9WDEhWfsbO93XWdigsExE5AuN7uwFyC48w.jar
Jun 09, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-tests-_xBHMhjnDP1ZHSxyMAqjveVqQLaceOWs8Ilc6WP7lOg.jar
Jun 09, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.23.0-SNAPSHOT-netxTMdITUsStZFGD90a0j3y-oZOq0I6HT8bkHYn5WQ.jar
Jun 09, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-1cVOvTNqLLFi2Z0vsfX2SQTTTZLqnGT-tccxJPdgLkQ.jar
Jun 09, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.23.0-SNAPSHOT-B7Ra5CITWraprBHdMcUFbiAPQzVu485QaoVl6UGpDXs.jar
Jun 09, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.23.0-SNAPSHOT-RxHdqozbCJAcnfkV6M7xKpMXGC7fBYS73MaSzmAx_qE.jar
Jun 09, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.23.0-SNAPSHOT-XpI7FLKnbVuemYDCvfPNdrpI_ac6P1dEnqKrWFhnHiU.jar
Jun 09, 2020 6:45:48 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-1cVOvTNqLLFi2Z0vsfX2SQTTTZLqnGT-tccxJPdgLkQ.jar&uploadType=resumable&upload_id=AAANsUnNfk8TD0Zbs6AgMlg0E4isbCFXY3oRbH5TfehNuPimrxUs6KOjkfzTE4_siL49Ct1bEWWTRD5aZCD8QZ3oCsILYoKocg.
Jun 09, 2020 6:45:48 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar>
java.io.IOException: Upload failed for 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-1cVOvTNqLLFi2Z0vsfX2SQTTTZLqnGT-tccxJPdgLkQ.jar'
at com.google.cloud.hadoop.util.BaseAbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(BaseAbstractGoogleAsyncWriteChannel.java:297)
at com.google.cloud.hadoop.util.BaseAbstractGoogleAsyncWriteChannel.close(BaseAbstractGoogleAsyncWriteChannel.java:214)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:221)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:168)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:152)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:136)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Jun 09, 2020 6:45:52 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-1cVOvTNqLLFi2Z0vsfX2SQTTTZLqnGT-tccxJPdgLkQ.jar
Jun 09, 2020 6:45:53 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 31 files newly uploaded in 6 seconds
Jun 09, 2020 6:45:53 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Jun 09, 2020 6:45:53 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Jun 09, 2020 6:45:53 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Jun 09, 2020 6:45:53 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Jun 09, 2020 6:45:53 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Jun 09, 2020 6:45:53 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <91292 bytes, hash 96f9bdb60ca2308f0fce474f08946bf1c4532b2649cd2260550a1f7921505ec3> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-lvm9tgyiMI8PzkdPCJRr8cRTKyZJzSJgVQofeSFQXsM.pb
Jun 09, 2020 6:45:54 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.23.0-SNAPSHOT
Jun 09, 2020 6:45:55 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-06-08_23_45_54-1321795041532301499?project=apache-beam-testing
Jun 09, 2020 6:45:55 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-06-08_23_45_54-1321795041532301499
Jun 09, 2020 6:45:55 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-06-08_23_45_54-1321795041532301499
Jun 09, 2020 6:45:55 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-06-09T06:45:54.417Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Jun 09, 2020 6:46:04 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-09T06:46:02.432Z: Worker configuration: n1-standard-1 in us-central1-a.
Jun 09, 2020 6:46:04 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-09T06:46:03.327Z: Expanding CoGroupByKey operations into optimizable parts.
Jun 09, 2020 6:46:04 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-09T06:46:03.370Z: Expanding GroupByKey operations into optimizable parts.
Jun 09, 2020 6:46:04 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-09T06:46:03.404Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jun 09, 2020 6:46:04 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-09T06:46:03.477Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jun 09, 2020 6:46:04 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-09T06:46:03.502Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Jun 09, 2020 6:46:04 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-09T06:46:03.524Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Jun 09, 2020 6:46:04 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-09T06:46:03.546Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Jun 09, 2020 6:46:04 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-09T06:46:03.982Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Jun 09, 2020 6:46:04 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-09T06:46:04.038Z: Starting 5 workers in us-central1-a...
Jun 09, 2020 6:46:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-06-09T06:46:15.535Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jun 09, 2020 6:46:28 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-09T06:46:28.030Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running stage(s).
Jun 09, 2020 6:46:28 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-09T06:46:28.053Z: Resized worker pool to 4, though goal was 5. This could be a quota issue.
Jun 09, 2020 6:46:34 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-09T06:46:33.399Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Jun 09, 2020 6:46:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-09T06:46:45.401Z: Workers have started successfully.
Jun 09, 2020 6:46:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-09T06:46:45.424Z: Workers have started successfully.
Jun 09, 2020 6:47:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-09T06:47:15.249Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Jun 09, 2020 6:47:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-09T06:47:15.468Z: Cleaning up.
Jun 09, 2020 6:47:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-09T06:47:15.539Z: Stopping worker pool...
Jun 09, 2020 6:49:15 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-09T06:49:15.704Z: Autoscaling: Resized worker pool from 5 to 0.
Jun 09, 2020 6:49:15 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-09T06:49:15.753Z: Worker pool stopped.
Jun 09, 2020 6:49:25 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-06-08_23_45_54-1321795041532301499 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 174527f9-d24b-4318-944b-4f87b4cc17ea and timestamp: 2020-06-09T06:49:25.684000000Z:
Metric: Value:
fields_read 4375276.0
read_time 11.533
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Jun 09, 2020 6:49:26 AM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 2 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.021 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.033 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':',5,main]) completed. Took 3 mins 51.318 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 6s
104 actionable tasks: 70 executed, 34 from cache
Publishing build scan...
https://gradle.com/s/et5q3wu7alumw
Stopped 1 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #604
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/604/display/redirect?page=changes>
Changes:
[robertwb] Cleanup ToString transforms.
[iemejia] [BEAM-10211] Upgrade Spark to version 2.4.6
[github] Update Beam website to release 2.22.0 (#11904)
[amaliujia] [BEAM-10215] @Ignore: Concat now works with varargs
[amaliujia] [BEAM-9191] Add Jira Link to empty @Ignore.
[github] Add blog post announcing the 2.22.0 release (#11910)
[github] [BEAM-10213] @Ignore: fix the test for testCastToDateWithCase. (#11948)
------------------------------------------
[...truncated 293.21 KB...]
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:164)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Jun 09, 2020 1:49:35 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Jun 09, 2020 1:49:35 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Jun 09, 2020 1:49:35 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Jun 09, 2020 1:49:35 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Jun 09, 2020 1:49:35 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Jun 09, 2020 1:49:35 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Jun 09, 2020 1:49:36 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Jun 09, 2020 1:49:37 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 209 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Jun 09, 2020 1:49:37 AM org.apache.beam.runners.dataflow.util.PackageUtil$PackageAttributes forFileToStage
INFO: Staging custom dataflow-worker.jar as beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-b682nR4Tb_Sc9qD7RuEGub8846SZH_f6b6VesCZ3TNE.jar
Jun 09, 2020 1:49:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-tests-zizqcHQftGCk9pSaoXq-FdqwOaRmcAKDj9p6Qq9BjCs.jar
Jun 09, 2020 1:49:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-b682nR4Tb_Sc9qD7RuEGub8846SZH_f6b6VesCZ3TNE.jar
Jun 09, 2020 1:49:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.23.0-SNAPSHOT-mCpfDyyzgCgAZzLTEm9u0UPOmuTIlIzLuGCWKH_KaV0.jar
Jun 09, 2020 1:49:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.23.0-SNAPSHOT-QRIp-om9ka_a_i1yCD-ZXNvNIPWYTootJR3ugaKnzT0.jar
Jun 09, 2020 1:49:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-tests-KNAYy5gBYuyJBH6dqkJPTCGxB8FgjRQz3-_GR-R_-mA.jar
Jun 09, 2020 1:49:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.23.0-SNAPSHOT-90Qt4f-11R0F0MoToubbaLaI10zW5tVDdWC_IZ5Ba5M.jar
Jun 09, 2020 1:49:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-jwj-eq0TtdiKSCdJ2P7e_2sy78qlFm8wjzSGI35ujKk.jar
Jun 09, 2020 1:49:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /tmp/test1227048931457528110.zip to gs://temp-storage-for-perf-tests/loadtests/staging/test-6JZCWjTviH4MemzBW8LMH9uUTDayaKxGPJudaTbtXpk.jar
Jun 09, 2020 1:49:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-v8SKYwi-dxQPcLx2SbcC6JSiHPPkneVVeVJK62MfSiE.jar
Jun 09, 2020 1:49:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-BK7Vua7Ba2XuboDdY4SNIOgYCkoZFxOBUGJzb1NgEQw.jar
Jun 09, 2020 1:49:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-5ORHfSF5Xt5qEY_pMNsvN0UEksJf_z0EHjVjV59e3yk.jar
Jun 09, 2020 1:49:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/java-fn-execution/build/libs/beam-runners-java-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-java-fn-execution-2.23.0-SNAPSHOT-ApB_J4N9oaMnG2roXEkXVLzGBIDl41VfzxOhJdgOlFw.jar
Jun 09, 2020 1:49:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.23.0-SNAPSHOT-tests-acptswSSu1_-nMvMRiDQWTmTFTlfw8Qc7qd5KoO6Et0.jar
Jun 09, 2020 1:49:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-tests-oDQrO3BNdkzkF9TUyyfbybCWM9E_-oaEVX7UPBoeHKk.jar
Jun 09, 2020 1:49:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.23.0-SNAPSHOT-V9Hx6crfYNshcrnPW0IcSA1or02JtSfFItxHTgM1UOc.jar
Jun 09, 2020 1:49:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-tests-kOi6_7W249mBZwYsIzyYX4fVt5-EDd0nw4VSglzCnEs.jar
Jun 09, 2020 1:49:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT-3PCPgQw9loQWtSV_PJdnk3KGBfI_c1SKUGMV6VscM88.jar
Jun 09, 2020 1:49:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-tests-WPpKpEsCO9O_KJRZ-RFlncTuI_1QI3q9HOkjpqY80c8.jar
Jun 09, 2020 1:49:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-kvktQLlZnBjp09XvtN0KUYTBnyW4LGRSgw89XMDDVoE.jar
Jun 09, 2020 1:49:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-tests-AbGF3XGN4QtqaGp1ArsfXWwOkQ5qh69csPEHeMzotYA.jar
Jun 09, 2020 1:49:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-unshaded-nZLwBFk7hh0HzWSsxzhX5fopHWfAsPQHlZFsG8CYpss.jar
Jun 09, 2020 1:49:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT-YFVTGwe6lTGIN8GYMoAZCLgX4PaqY3zmzJiI4ydLUlI.jar
Jun 09, 2020 1:49:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tests-x_7w3VdHTpoovUBOTEe3CGKJK5HsvNDqvNp1LH78bjg.jar
Jun 09, 2020 1:49:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-YJuIIGkcwY1TdepJPtTXwcAJzDnxycsc_o0vhcUUSHI.jar
Jun 09, 2020 1:49:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-fn-execution-2.23.0-SNAPSHOT-vJoqNYsCplII7PhE33eQWU0frMnyvanNViCPBGSncCk.jar
Jun 09, 2020 1:49:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.23.0-SNAPSHOT-KvNqtnowYoZx2w3JNltdrfjJsBol64IbtsN7Shgu7Hk.jar
Jun 09, 2020 1:49:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-bkONzHWKpE6kUvCc75oPz6uXVETeSJGMm_85OejSp1o.jar
Jun 09, 2020 1:49:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-b682nR4Tb_Sc9qD7RuEGub8846SZH_f6b6VesCZ3TNE.jar
Jun 09, 2020 1:49:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.23.0-SNAPSHOT-KY_sI2cgOUdnz7kwRP_XOFg0w6r-J5vKQqhvMDaP8Vo.jar
Jun 09, 2020 1:49:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.23.0-SNAPSHOT-EzgbDAyqwlIIIiTQbAwHdYC-lvSvyWKdtdM-oBv1zYE.jar
Jun 09, 2020 1:49:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.23.0-SNAPSHOT-ae-V7s-bcYgdJ3yZIwcgEWSiDhVhDIAcXtU44HQJfV4.jar
Jun 09, 2020 1:49:39 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-b682nR4Tb_Sc9qD7RuEGub8846SZH_f6b6VesCZ3TNE.jar&uploadType=resumable&upload_id=AAANsUnj_HsIcBzJ9cjD9yCSwoePDiqrzFSehgO65c83uDO4yzyNpeZguVArePE-1xwmftQQKXU87AEJzgO0-5v1Vl8.
Jun 09, 2020 1:49:39 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar>
java.io.IOException: Upload failed for 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-b682nR4Tb_Sc9qD7RuEGub8846SZH_f6b6VesCZ3TNE.jar'
at com.google.cloud.hadoop.util.BaseAbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(BaseAbstractGoogleAsyncWriteChannel.java:297)
at com.google.cloud.hadoop.util.BaseAbstractGoogleAsyncWriteChannel.close(BaseAbstractGoogleAsyncWriteChannel.java:214)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:221)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:168)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:152)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:136)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Jun 09, 2020 1:49:46 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-b682nR4Tb_Sc9qD7RuEGub8846SZH_f6b6VesCZ3TNE.jar
Jun 09, 2020 1:49:47 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 31 files newly uploaded in 9 seconds
Jun 09, 2020 1:49:47 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Jun 09, 2020 1:49:47 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Jun 09, 2020 1:49:47 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Jun 09, 2020 1:49:47 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Jun 09, 2020 1:49:47 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Jun 09, 2020 1:49:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <91292 bytes, hash 2b107ba9b49b96dfc5ebaba1962deb9441f2b4b8158ad816cb02cf94d4703432> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-KxB7qbSblt_F66uhli3rlEHytLgVitgWywLPlNRwNDI.pb
Jun 09, 2020 1:49:47 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.23.0-SNAPSHOT
Jun 09, 2020 1:49:49 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-06-08_18_49_47-9151558150073267613?project=apache-beam-testing
Jun 09, 2020 1:49:49 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-06-08_18_49_47-9151558150073267613
Jun 09, 2020 1:49:49 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-06-08_18_49_47-9151558150073267613
Jun 09, 2020 1:49:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-06-09T01:49:47.995Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Jun 09, 2020 1:49:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-09T01:49:54.720Z: Worker configuration: n1-standard-1 in us-central1-a.
Jun 09, 2020 1:49:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-09T01:49:55.479Z: Expanding CoGroupByKey operations into optimizable parts.
Jun 09, 2020 1:49:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-09T01:49:55.540Z: Expanding GroupByKey operations into optimizable parts.
Jun 09, 2020 1:49:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-09T01:49:55.577Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jun 09, 2020 1:49:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-09T01:49:55.658Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jun 09, 2020 1:49:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-09T01:49:55.694Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Jun 09, 2020 1:49:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-09T01:49:55.753Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Jun 09, 2020 1:49:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-09T01:49:55.794Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Jun 09, 2020 1:49:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-09T01:49:56.263Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Jun 09, 2020 1:49:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-09T01:49:56.350Z: Starting 5 workers in us-central1-a...
Jun 09, 2020 1:50:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-06-09T01:50:02.456Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jun 09, 2020 1:50:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-09T01:50:20.239Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running stage(s).
Jun 09, 2020 1:50:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-09T01:50:20.268Z: Resized worker pool to 4, though goal was 5. This could be a quota issue.
Jun 09, 2020 1:50:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-09T01:50:25.679Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Jun 09, 2020 1:50:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-09T01:50:42.846Z: Workers have started successfully.
Jun 09, 2020 1:50:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-09T01:50:42.876Z: Workers have started successfully.
Jun 09, 2020 1:51:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-09T01:51:17.708Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Jun 09, 2020 1:51:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-09T01:51:17.878Z: Cleaning up.
Jun 09, 2020 1:51:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-09T01:51:17.945Z: Stopping worker pool...
Jun 09, 2020 1:52:57 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-09T01:52:55.279Z: Autoscaling: Resized worker pool from 5 to 0.
Jun 09, 2020 1:52:57 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-09T01:52:55.326Z: Worker pool stopped.
Jun 09, 2020 1:53:01 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-06-08_18_49_47-9151558150073267613 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): f3b0e8b0-8cd1-40ba-8c57-3fe640ff48f8 and timestamp: 2020-06-09T01:53:01.835000000Z:
Metric: Value:
read_time 15.187
fields_read 4375276.0
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Jun 09, 2020 1:53:07 AM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 2 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.023 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.031 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 4,5,main]) completed. Took 3 mins 40.514 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 54s
104 actionable tasks: 70 executed, 34 from cache
Publishing build scan...
https://gradle.com/s/33djzttpiybiw
Stopped 1 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #603
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/603/display/redirect?page=changes>
Changes:
[github] Fix links for type hint changes in 2.21 blog post (#11947)
------------------------------------------
[...truncated 293.58 KB...]
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:164)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Jun 08, 2020 7:01:16 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Jun 08, 2020 7:01:16 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Jun 08, 2020 7:01:16 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Jun 08, 2020 7:01:16 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Jun 08, 2020 7:01:17 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Jun 08, 2020 7:01:17 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Jun 08, 2020 7:01:17 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Jun 08, 2020 7:01:20 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 209 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Jun 08, 2020 7:01:20 PM org.apache.beam.runners.dataflow.util.PackageUtil$PackageAttributes forFileToStage
INFO: Staging custom dataflow-worker.jar as beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-fkDIieUiwzhna_en6vsSafLAeeFMS4mpb6P-pW0vmpc.jar
Jun 08, 2020 7:01:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-SOxcM6D8E1EgE0k_bGPOTxT74DWhxz_PHpEo2v21rbY.jar
Jun 08, 2020 7:01:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.23.0-SNAPSHOT-XD0a-52r0yxGOYdq-WW3fa_2d1RjblPTf4s2cGgXuaQ.jar
Jun 08, 2020 7:01:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-fkDIieUiwzhna_en6vsSafLAeeFMS4mpb6P-pW0vmpc.jar
Jun 08, 2020 7:01:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.23.0-SNAPSHOT-IwNALJRA0t9ccuGWtsJhd92RHOZKj79IOgiiU9x1UGw.jar
Jun 08, 2020 7:01:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-tests-g2koTdNks6Ro45SoI6gtPSrin1-zDeWIQs7vS3Q-Wc0.jar
Jun 08, 2020 7:01:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-y13nzVR78Zk7M-0M1dCMSz5Kui7E3ds7us3mN8m8vW0.jar
Jun 08, 2020 7:01:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-tests-8JAmNqRsVpIxvIHpyloVw0-L8wb2qj_wP6U1MX9LQso.jar
Jun 08, 2020 7:01:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/java-fn-execution/build/libs/beam-runners-java-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-java-fn-execution-2.23.0-SNAPSHOT-ZqC1QiUGh-keNwy4EPnaUCF9vZNCmwndXDZvza7XMvo.jar
Jun 08, 2020 7:01:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.23.0-SNAPSHOT-tests-OLDxB9uIPYI-3ZaFV-wn0MDsLcgFQ-8lHM0YKy60IMg.jar
Jun 08, 2020 7:01:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.23.0-SNAPSHOT-EXyqcyXipM3JQ1aOeaqudK-Nf-Vko4m23Ok6Qx6AxH4.jar
Jun 08, 2020 7:01:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-70SkoSnhNezEB9qdUx79ShnXqFY1QPZ7u1Ct2mwVhvc.jar
Jun 08, 2020 7:01:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-8n-WkCnxN5F-p26gnccvSnq8_MIGMosZnO49ROAKOZM.jar
Jun 08, 2020 7:01:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /tmp/test7535795950439910344.zip to gs://temp-storage-for-perf-tests/loadtests/staging/test-BRf424-LB6ZliXuMmLkTvy69-qq7JFsM9DospjAf8fs.jar
Jun 08, 2020 7:01:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.23.0-SNAPSHOT-D9hrxlobmign5mhCOuRWSK1_c6MlY25ga7w2YUL9398.jar
Jun 08, 2020 7:01:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT-HO8Z0pBD99UmKplIOj5e6ywOeZilgc9Kr-v1WIEht3U.jar
Jun 08, 2020 7:01:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-tests-4VJKWonFGiW6OsGyWD-Twejlp4hw0zd96GG7nZl7GgE.jar
Jun 08, 2020 7:01:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT-IYz2KZ1fhtFJnb4QFP6w97YFVyMg3IhJHFSwDPt0N1M.jar
Jun 08, 2020 7:01:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-dtcWmaWyb3PNn7ij3XOgJDAFzLgCOjrJsXY-UzBpkzE.jar
Jun 08, 2020 7:01:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-QY7pcI1r3WJXrSPGEcaeHrlaKZ56tTn-JR2i1CgKD9U.jar
Jun 08, 2020 7:01:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-fn-execution-2.23.0-SNAPSHOT-KR_h7i4mSZ98n6CcCiI_aDDc3YYK-wvxBWpjun5h4d4.jar
Jun 08, 2020 7:01:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-tests-PQHmHAm-c0vYEBDEg-1qc7_jcBzSmy0T3A27-OSVRas.jar
Jun 08, 2020 7:01:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tests-_Ex9L-rG0lknoLdGHUcpsuvyIYK_UxOgxUrL0VhJ8Uw.jar
Jun 08, 2020 7:01:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-1dzfh2-p5-okXEQ8wapOlqI4K1wXNQ9Kmn0K9UcHAn8.jar
Jun 08, 2020 7:01:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-tests-3aZ3KARHdPxgr6AIyVEihB8QzjAU94CYQSDHmaPQe-A.jar
Jun 08, 2020 7:01:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-tests-mrpfwcOE0Ydt5UAFGIkZgR1p0pnyk8NjDu8OtU3A__g.jar
Jun 08, 2020 7:01:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-fkDIieUiwzhna_en6vsSafLAeeFMS4mpb6P-pW0vmpc.jar
Jun 08, 2020 7:01:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-unshaded-lKxD7SYygxuPBtMV7ZDOuyVUM5LOItMON6-TNM4ZxCE.jar
Jun 08, 2020 7:01:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.23.0-SNAPSHOT-CIaSnaT9shzWvcMkDtMpOOguBK5qTnggoxUwR5IvisA.jar
Jun 08, 2020 7:01:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.23.0-SNAPSHOT-s_HXz2tHVjyBY7GQQEgIpbm8yO5uz8HZnMpb_1ilGm4.jar
Jun 08, 2020 7:01:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.23.0-SNAPSHOT--50AOwjdouc3tDPgNhMAVrn95KIZBjFd_bIoETpV3JI.jar
Jun 08, 2020 7:01:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.23.0-SNAPSHOT-1yK9K3DY6K-UStwA4RrTZcKBEhfCIpsR2JeYLe6y7aI.jar
Jun 08, 2020 7:01:21 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-fkDIieUiwzhna_en6vsSafLAeeFMS4mpb6P-pW0vmpc.jar&uploadType=resumable&upload_id=AAANsUmJ8WYTIoDtLjbJOy5cESMgiitE79pombO7BtP2oCqSPkx2cl6xiQJ18g5jDb5YlM2iqz1P3o0DVbCWCUpVZ-Vhv0jxxg.
Jun 08, 2020 7:01:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar>
java.io.IOException: Upload failed for 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-fkDIieUiwzhna_en6vsSafLAeeFMS4mpb6P-pW0vmpc.jar'
at com.google.cloud.hadoop.util.BaseAbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(BaseAbstractGoogleAsyncWriteChannel.java:297)
at com.google.cloud.hadoop.util.BaseAbstractGoogleAsyncWriteChannel.close(BaseAbstractGoogleAsyncWriteChannel.java:214)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:221)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:168)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:152)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:136)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Jun 08, 2020 7:01:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-fkDIieUiwzhna_en6vsSafLAeeFMS4mpb6P-pW0vmpc.jar
Jun 08, 2020 7:01:25 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 31 files newly uploaded in 5 seconds
Jun 08, 2020 7:01:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Jun 08, 2020 7:01:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Jun 08, 2020 7:01:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Jun 08, 2020 7:01:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Jun 08, 2020 7:01:26 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Jun 08, 2020 7:01:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <91292 bytes, hash e8af55a2e8691faabfa6ef784f0a8ff3e3ff40401b422ed90852222af16eb8ab> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-6K9VouhpH6q_pu94TwqP8-P_QEAbQi7ZCFIiKvFuuKs.pb
Jun 08, 2020 7:01:26 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.23.0-SNAPSHOT
Jun 08, 2020 7:01:28 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-06-08_12_01_27-18213788567503483502?project=apache-beam-testing
Jun 08, 2020 7:01:28 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-06-08_12_01_27-18213788567503483502
Jun 08, 2020 7:01:28 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-06-08_12_01_27-18213788567503483502
Jun 08, 2020 7:01:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-06-08T19:01:27.098Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Jun 08, 2020 7:01:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-08T19:01:34.031Z: Worker configuration: n1-standard-1 in us-central1-f.
Jun 08, 2020 7:01:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-08T19:01:34.755Z: Expanding CoGroupByKey operations into optimizable parts.
Jun 08, 2020 7:01:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-08T19:01:34.857Z: Expanding GroupByKey operations into optimizable parts.
Jun 08, 2020 7:01:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-08T19:01:34.892Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jun 08, 2020 7:01:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-08T19:01:34.986Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jun 08, 2020 7:01:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-08T19:01:35.033Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Jun 08, 2020 7:01:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-08T19:01:35.073Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Jun 08, 2020 7:01:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-08T19:01:35.115Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Jun 08, 2020 7:01:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-08T19:01:35.703Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Jun 08, 2020 7:01:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-08T19:01:35.786Z: Starting 5 workers in us-central1-f...
Jun 08, 2020 7:01:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-06-08T19:01:48.573Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jun 08, 2020 7:02:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-08T19:02:02.992Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Jun 08, 2020 7:02:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-08T19:02:24.417Z: Workers have started successfully.
Jun 08, 2020 7:02:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-08T19:02:24.463Z: Workers have started successfully.
Jun 08, 2020 7:02:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-08T19:02:56.501Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Jun 08, 2020 7:02:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-08T19:02:56.718Z: Cleaning up.
Jun 08, 2020 7:02:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-08T19:02:56.811Z: Stopping worker pool...
Jun 08, 2020 7:05:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-08T19:05:05.566Z: Autoscaling: Resized worker pool from 5 to 0.
Jun 08, 2020 7:05:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-08T19:05:05.620Z: Worker pool stopped.
Jun 08, 2020 7:05:10 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-06-08_12_01_27-18213788567503483502 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): ac6931c4-0512-4efb-a329-965bbc1474f8 and timestamp: 2020-06-08T19:05:10.906000000Z:
Metric: Value:
read_time 13.413
fields_read 4375276.0
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Jun 08, 2020 7:05:11 PM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 6 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.002 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.006 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 10,5,main]) completed. Took 4 mins 3.799 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 5s
104 actionable tasks: 70 executed, 34 from cache
Publishing build scan...
https://gradle.com/s/avkx4xh2s5upq
Stopped 1 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #602
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/602/display/redirect?page=changes>
Changes:
[kcweaver] [BEAM-10024] Mark testOutputTimestampDefault with
[kcweaver] Create bounded/unbounded variants of testOutputTimestampDefault.
------------------------------------------
[...truncated 302.34 KB...]
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:164)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Jun 08, 2020 1:26:24 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Jun 08, 2020 1:26:24 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Jun 08, 2020 1:26:24 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Jun 08, 2020 1:26:24 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Jun 08, 2020 1:26:25 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Jun 08, 2020 1:26:25 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Jun 08, 2020 1:26:27 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Jun 08, 2020 1:26:37 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 209 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Jun 08, 2020 1:26:38 PM org.apache.beam.runners.dataflow.util.PackageUtil$PackageAttributes forFileToStage
INFO: Staging custom dataflow-worker.jar as beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-qGPG_6tHTlExbRW6tHk4eVp17q40pZvgJoQxG11j5aI.jar
Jun 08, 2020 1:26:40 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT-x5voszOUB163DDPqUGgPlHKtNqrzdDCuIHgGvR79HbI.jar
Jun 08, 2020 1:26:40 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.23.0-SNAPSHOT-tests-k4GUb-sJx4SuJ-2s7jFmXCL9Ntgd2Rct_LbSExooFa4.jar
Jun 08, 2020 1:26:40 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.23.0-SNAPSHOT-5tLkU22gh7cG9m628NKI4S2zzIoMUTRA7Is2G6jfybM.jar
Jun 08, 2020 1:26:40 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-tests-geGPmCa36TZyUz91N40DDx5nsYCLtKY6dKqSJHuyYvc.jar
Jun 08, 2020 1:26:40 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-fn-execution-2.23.0-SNAPSHOT-bcPMZ_rO6Ep6Ela3Tm4sdCcOv9HgOIEnAfAn8tIhwoQ.jar
Jun 08, 2020 1:26:40 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-NjHR-dc_kUCZHi7h2vLS0UdlGWEPlyYe3vc0n8MJAeI.jar
Jun 08, 2020 1:26:40 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-uyeqy4PBAOVZJkUAvhlQR2SUKvCaDPUiB4nSUS_9mco.jar
Jun 08, 2020 1:26:40 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-LYJL18k6AcL5m5GCnlMg1Q_AU9Y4yXlnw9buko3Zmos.jar
Jun 08, 2020 1:26:40 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /tmp/test5433956549536382803.zip to gs://temp-storage-for-perf-tests/loadtests/staging/test-cA7Jt0ugHX3Jh31S7Yse509MA5gp1BExanvMqM0Pm60.jar
Jun 08, 2020 1:26:40 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-qGPG_6tHTlExbRW6tHk4eVp17q40pZvgJoQxG11j5aI.jar
Jun 08, 2020 1:26:40 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT-PQa2LsGNWPrFJOLAlaaeefSBkwYTd3pvwvw87doPpqA.jar
Jun 08, 2020 1:26:40 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-tests-1wP6fZ_-vO5ExRUKFjhQqQtBgADTrLJcan34nM2wpmk.jar
Jun 08, 2020 1:26:40 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-kOEYR9Oc7h21FTNLqfYU5BR10JZyBQui7717JylJPQo.jar
Jun 08, 2020 1:26:40 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-tests-5CxhtEd6Fg_gegPd-2HN-IVJ3P5kelCUGFt4KOklVf0.jar
Jun 08, 2020 1:26:40 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/java-fn-execution/build/libs/beam-runners-java-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-java-fn-execution-2.23.0-SNAPSHOT-0p0LN1l8X8a3tvtkqPtBfnWVyjyndCL_SqdYNdmxPiE.jar
Jun 08, 2020 1:26:40 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-tests-zqEI_U20kzrF-PobVV4Z5JarsIR5zE63dIHzEMWRuFw.jar
Jun 08, 2020 1:26:40 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.23.0-SNAPSHOT-DYsgxQLe9O03em1ddGNXmkgS3q3dLzitCSNm4gQcNuA.jar
Jun 08, 2020 1:26:40 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.23.0-SNAPSHOT-PdVvkOwFTZwzD3rcCRFNhR3vjm_Pk_deSpv9zjHqZrY.jar
Jun 08, 2020 1:26:40 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-Zl0Nle16_ZDGzHxsKuEvL0eHSvEFVG70Xs7-VVS2M4Q.jar
Jun 08, 2020 1:26:40 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-tests-jvsBPHyw4ePkVa23Rcljit5GcTKz0Ybd1i4CuD7ZuNU.jar
Jun 08, 2020 1:26:40 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-IbHVLRp4ptCV1qWUL-uIVR1Giv0bzyMqGaPCxd4S3dU.jar
Jun 08, 2020 1:26:40 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-tests-9K187Oq8NwIjqbURf7e0MIeWaAPtfV83jjVM8tbnFeQ.jar
Jun 08, 2020 1:26:40 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-oiRZ8hEJav2JeX-RiHlOY37FvQtZCpt1tu5QVRbcihk.jar
Jun 08, 2020 1:26:40 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.23.0-SNAPSHOT-vTAGrGUeHcWz5mcL8G1zEz8YSVIzueSWK1ig25WBuhA.jar
Jun 08, 2020 1:26:41 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.23.0-SNAPSHOT-HZmTUVsaB4fCt1P8E-riGd4RbHMkKmaZ00vJIzOjvGw.jar
Jun 08, 2020 1:26:41 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-unshaded-9z9m5GF13XofhjOVZo4BB-6qetFOHMXk6nAHVQVwO7k.jar
Jun 08, 2020 1:26:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tests-3SLAS12BtQm-yc2hrNTlRBqbj0u85MV5E1c23bqX02c.jar
Jun 08, 2020 1:26:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-qGPG_6tHTlExbRW6tHk4eVp17q40pZvgJoQxG11j5aI.jar
Jun 08, 2020 1:26:45 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.23.0-SNAPSHOT-waTnouTzJY57x5wgsFwUtfZ6E53P9ZTKYMV7fCIh38Y.jar
Jun 08, 2020 1:26:45 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.23.0-SNAPSHOT-jZlTveC3amwoltatGjhLXN0efJXM24ivewyjvMZK_xI.jar
Jun 08, 2020 1:26:45 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.23.0-SNAPSHOT-9I6mV3kSq7sNrJfC-dUz7GzjU1CcB_zGutXUj9uJmDU.jar
Jun 08, 2020 1:26:45 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-qGPG_6tHTlExbRW6tHk4eVp17q40pZvgJoQxG11j5aI.jar&uploadType=resumable&upload_id=AAANsUntSp6__CRkwsJ5lpeotudotYLYJkhQvijy-6u9DnQgLh8Vix4KyP4_2s4tOcacdtNQ_6F14NlxCmizf4EZqSXApZwZkw.
Jun 08, 2020 1:26:45 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar>
java.io.IOException: Upload failed for 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-qGPG_6tHTlExbRW6tHk4eVp17q40pZvgJoQxG11j5aI.jar'
at com.google.cloud.hadoop.util.BaseAbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(BaseAbstractGoogleAsyncWriteChannel.java:297)
at com.google.cloud.hadoop.util.BaseAbstractGoogleAsyncWriteChannel.close(BaseAbstractGoogleAsyncWriteChannel.java:214)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:221)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:168)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:152)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:136)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Jun 08, 2020 1:26:50 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-qGPG_6tHTlExbRW6tHk4eVp17q40pZvgJoQxG11j5aI.jar
Jun 08, 2020 1:26:50 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 31 files newly uploaded in 13 seconds
Jun 08, 2020 1:26:51 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Jun 08, 2020 1:26:51 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Jun 08, 2020 1:26:52 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Jun 08, 2020 1:26:52 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Jun 08, 2020 1:26:52 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Jun 08, 2020 1:26:52 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <91292 bytes, hash 50497b89c7f61cc00d008bfe436f928dea4c28e9ee05f1b0d8c6aca866cc64e3> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-UEl7icf2HMANAIv-Q2-SjepMKOnuBfGw2MasqGbMZOM.pb
Jun 08, 2020 1:26:53 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.23.0-SNAPSHOT
Jun 08, 2020 1:26:54 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-06-08_06_26_53-17001055089433930251?project=apache-beam-testing
Jun 08, 2020 1:26:54 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-06-08_06_26_53-17001055089433930251
Jun 08, 2020 1:26:54 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-06-08_06_26_53-17001055089433930251
Jun 08, 2020 1:26:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-06-08T13:26:53.419Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Jun 08, 2020 1:27:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-08T13:27:03.757Z: Worker configuration: n1-standard-1 in us-central1-a.
Jun 08, 2020 1:27:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-08T13:27:04.663Z: Expanding CoGroupByKey operations into optimizable parts.
Jun 08, 2020 1:27:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-08T13:27:04.706Z: Expanding GroupByKey operations into optimizable parts.
Jun 08, 2020 1:27:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-08T13:27:04.747Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jun 08, 2020 1:27:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-08T13:27:04.848Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jun 08, 2020 1:27:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-08T13:27:04.884Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Jun 08, 2020 1:27:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-08T13:27:04.910Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Jun 08, 2020 1:27:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-08T13:27:04.946Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Jun 08, 2020 1:27:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-08T13:27:05.360Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Jun 08, 2020 1:27:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-08T13:27:05.495Z: Starting 5 workers in us-central1-a...
Jun 08, 2020 1:27:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-06-08T13:27:24.280Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jun 08, 2020 1:27:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-08T13:27:32.856Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running stage(s).
Jun 08, 2020 1:27:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-08T13:27:32.901Z: Resized worker pool to 4, though goal was 5. This could be a quota issue.
Jun 08, 2020 1:27:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-08T13:27:38.266Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Jun 08, 2020 1:27:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-08T13:27:53.804Z: Workers have started successfully.
Jun 08, 2020 1:27:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-08T13:27:53.839Z: Workers have started successfully.
Jun 08, 2020 1:28:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-08T13:28:28.386Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Jun 08, 2020 1:28:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-08T13:28:28.647Z: Cleaning up.
Jun 08, 2020 1:28:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-08T13:28:28.752Z: Stopping worker pool...
Jun 08, 2020 1:30:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-08T13:30:30.537Z: Autoscaling: Resized worker pool from 5 to 0.
Jun 08, 2020 1:30:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-08T13:30:30.600Z: Worker pool stopped.
Jun 08, 2020 1:30:36 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-06-08_06_26_53-17001055089433930251 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 5243ba90-9330-4a53-8375-093088b86a54 and timestamp: 2020-06-08T13:30:36.532000000Z:
Metric: Value:
read_time 17.004
fields_read 4375276.0
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Jun 08, 2020 1:30:37 PM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 2 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.091 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.151 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Daemon worker,5,main]) completed. Took 5 mins 2.678 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 10m 59s
104 actionable tasks: 70 executed, 34 from cache
Publishing build scan...
https://gradle.com/s/hgkhbs2morqto
Stopped 1 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #601
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/601/display/redirect>
Changes:
------------------------------------------
[...truncated 291.95 KB...]
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:164)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Jun 08, 2020 6:45:20 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Jun 08, 2020 6:45:20 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Jun 08, 2020 6:45:20 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Jun 08, 2020 6:45:20 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Jun 08, 2020 6:45:20 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Jun 08, 2020 6:45:20 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Jun 08, 2020 6:45:21 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Jun 08, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 209 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Jun 08, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil$PackageAttributes forFileToStage
INFO: Staging custom dataflow-worker.jar as beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-vIMf1AUNGzvwzMFM_GDUQ7xs1DJ3WeE_cCK70Q5uGBY.jar
Jun 08, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-tests-EAYVtgmaY9xSzdJwu8bVww4YHIhFfazB33qmkCX7ZbU.jar
Jun 08, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-7wS226Eg15fJT3qS2hqpe6KqFW19gQwsb-gSoXK72GM.jar
Jun 08, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-tests-qsdkbQcj04kSd-sBgIxNE9L9xFzrFhtaoZElBAY3wOE.jar
Jun 08, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-vIMf1AUNGzvwzMFM_GDUQ7xs1DJ3WeE_cCK70Q5uGBY.jar
Jun 08, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-Cu3o_nYTBlhC0qBQbdkiPTeSCE03jBLopoI2R3RXX7g.jar
Jun 08, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.23.0-SNAPSHOT-Ly5_SjNZ-KW73lyOQZqynuogPsSqO0cPpoF1fxSzN9w.jar
Jun 08, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-1WIQ9-cgSsWvfLqPGqsshWYyq-PIPvwlS7ScpitDUiY.jar
Jun 08, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-fn-execution-2.23.0-SNAPSHOT-Fug3IFpIjM0P0BhtNdk014RkLPLhP9aqJT2zWIITZgI.jar
Jun 08, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.23.0-SNAPSHOT--Yq9mILU6V64yhZnT_qiFAy2MwWduIJKSdEjVBy91hQ.jar
Jun 08, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.23.0-SNAPSHOT-RVQJMzM16x99Sb7u0k59Jr64R1_S1W8nUsI27JWrCsI.jar
Jun 08, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-N5k7-l9c6yf4Y2rHcCYyiBQTGzm44ub8sCNhMlSyvz0.jar
Jun 08, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.23.0-SNAPSHOT-tests-yi9h-3xD-8_FTqD5gq2a7zOIgE7CusrGGMs9sgKMB5k.jar
Jun 08, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-tests-J2E0lHG7I7dxP9CSr1xdpFDuwT8nXHgFZZGQ96Zmbyo.jar
Jun 08, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.23.0-SNAPSHOT-4SDmY8GPB1L7x-CCTxVKAnvidw4yl2eGHRFbLUAOzG4.jar
Jun 08, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.23.0-SNAPSHOT-Ie0HhNwKz64RzWmVpVoCpQ8sQCDrKo-jOb_49S__gfY.jar
Jun 08, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT-8Vo8egJAXToSIvAMKOwh3Jg5LtLF8vpxWBDCqHYNRCA.jar
Jun 08, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT-2OPCFSMTQPobTSNGmxv8HMdH03HcZ-YB1VLMesWBR7c.jar
Jun 08, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /tmp/test2975940725784367076.zip to gs://temp-storage-for-perf-tests/loadtests/staging/test-TucJ0SDpQBYzX2BIZVnddrSwXFuIZ6ElrJa9njBeB-M.jar
Jun 08, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tests-hKmJoDP8VJw1WgGX9b1pEnUAyEjn3u9m8C6lbSTo9rc.jar
Jun 08, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-bsnSg0sRsp604GgTq2UJ0xQIBM_2AM4CyHXnwHUkRSg.jar
Jun 08, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/java-fn-execution/build/libs/beam-runners-java-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-java-fn-execution-2.23.0-SNAPSHOT-as5NagvcZTnE3M3TrM-6T-XlpUGs0dVlIeD9XZDIZXw.jar
Jun 08, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-tests-GkWVmm9AKY1GU3sUAtKmJVUN43qOgROzTFazuaZo5nY.jar
Jun 08, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-YnFVdv4sh1SsJh5jZe3XdmDbxs20l1rvfttLN9b_RUI.jar
Jun 08, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-tests-F7byugNyl0DkzJM8PUM19HdBbJS_CxTP9k-RSy9KnJE.jar
Jun 08, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-tests-ibdS5ls9AJTZ8YDXzUsZ4xZAyFixsLws_Yr4rKVVKrk.jar
Jun 08, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-unshaded-R2IwuYnixZf6ay_COaS6kNUurNRIcdIgW35Lly9PA-8.jar
Jun 08, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-vIMf1AUNGzvwzMFM_GDUQ7xs1DJ3WeE_cCK70Q5uGBY.jar
Jun 08, 2020 6:45:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-Ua6VqF_o3HZFX7JSh_SoBXv34fW5d-FraJdy_wDr7i4.jar
Jun 08, 2020 6:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.23.0-SNAPSHOT-bhGEBzGWlihEq5u0KPFQ5u_QIY7CzZnoongr3Y-TqsY.jar
Jun 08, 2020 6:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.23.0-SNAPSHOT-KxMTwU1ETJU45SnYOX8tVt_IOyNSLXNnVBE0fBbREMM.jar
Jun 08, 2020 6:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.23.0-SNAPSHOT-vIalR6A6sCx9BHqNCmxuVPTZm9KXjHddlq4pDp13IWg.jar
Jun 08, 2020 6:45:24 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-vIMf1AUNGzvwzMFM_GDUQ7xs1DJ3WeE_cCK70Q5uGBY.jar&uploadType=resumable&upload_id=AAANsUlFkNxRayQofJ9Srmj-C1-ieMYWe3aPYLOinR050aeZJk87avL2mWEhkh7KNKhSKgtmUfag0B4ZzfOo3i8x-CJX6Hq3pg.
Jun 08, 2020 6:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar>
java.io.IOException: Upload failed for 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-vIMf1AUNGzvwzMFM_GDUQ7xs1DJ3WeE_cCK70Q5uGBY.jar'
at com.google.cloud.hadoop.util.BaseAbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(BaseAbstractGoogleAsyncWriteChannel.java:297)
at com.google.cloud.hadoop.util.BaseAbstractGoogleAsyncWriteChannel.close(BaseAbstractGoogleAsyncWriteChannel.java:214)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:221)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:168)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:152)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:136)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Jun 08, 2020 6:45:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-vIMf1AUNGzvwzMFM_GDUQ7xs1DJ3WeE_cCK70Q5uGBY.jar
Jun 08, 2020 6:45:33 AM org.apache.beam.sdk.metrics.MetricsEnvironment getCurrentContainer
WARNING: Reporting metrics are not supported in the current execution environment.
Jun 08, 2020 6:45:33 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 31 files newly uploaded in 10 seconds
Jun 08, 2020 6:45:33 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Jun 08, 2020 6:45:33 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Jun 08, 2020 6:45:33 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Jun 08, 2020 6:45:33 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Jun 08, 2020 6:45:33 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Jun 08, 2020 6:45:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <91291 bytes, hash 7d70395510e7058428ba1ab5cedee23081588653d176921529d3f8957e2f202e> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-fXA5VRDnBYQouhq1zt7iMIFYhlPRdpIVKdP4lX4vIC4.pb
Jun 08, 2020 6:45:34 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.23.0-SNAPSHOT
Jun 08, 2020 6:45:35 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-06-07_23_45_34-17740389473736354975?project=apache-beam-testing
Jun 08, 2020 6:45:35 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-06-07_23_45_34-17740389473736354975
Jun 08, 2020 6:45:35 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-06-07_23_45_34-17740389473736354975
Jun 08, 2020 6:45:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-06-08T06:45:34.765Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Jun 08, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-08T06:45:42.277Z: Worker configuration: n1-standard-1 in us-central1-a.
Jun 08, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-08T06:45:43.026Z: Expanding CoGroupByKey operations into optimizable parts.
Jun 08, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-08T06:45:43.074Z: Expanding GroupByKey operations into optimizable parts.
Jun 08, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-08T06:45:43.116Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jun 08, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-08T06:45:43.217Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jun 08, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-08T06:45:43.251Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Jun 08, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-08T06:45:43.288Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Jun 08, 2020 6:45:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-08T06:45:43.317Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Jun 08, 2020 6:45:45 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-08T06:45:43.726Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Jun 08, 2020 6:45:45 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-08T06:45:43.852Z: Starting 5 workers in us-central1-a...
Jun 08, 2020 6:46:13 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-08T06:46:12.568Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running stage(s).
Jun 08, 2020 6:46:13 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-08T06:46:12.597Z: Resized worker pool to 4, though goal was 5. This could be a quota issue.
Jun 08, 2020 6:46:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-06-08T06:46:15.921Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jun 08, 2020 6:46:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-08T06:46:17.967Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Jun 08, 2020 6:46:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-08T06:46:30.991Z: Workers have started successfully.
Jun 08, 2020 6:46:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-08T06:46:31.029Z: Workers have started successfully.
Jun 08, 2020 6:47:02 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-08T06:47:02.565Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Jun 08, 2020 6:47:02 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-08T06:47:02.803Z: Cleaning up.
Jun 08, 2020 6:47:05 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-08T06:47:02.885Z: Stopping worker pool...
Jun 08, 2020 6:48:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-08T06:48:37.967Z: Autoscaling: Resized worker pool from 5 to 0.
Jun 08, 2020 6:48:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-08T06:48:38.060Z: Worker pool stopped.
Jun 08, 2020 6:48:43 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-06-07_23_45_34-17740389473736354975 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 1c70d2eb-570c-4d69-88b3-70b4981b988f and timestamp: 2020-06-08T06:48:43.679000000Z:
Metric: Value:
fields_read 4375276.0
read_time 14.24
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Jun 08, 2020 6:48:44 AM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.021 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.032 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Daemon worker,5,main]) completed. Took 3 mins 32.316 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 24s
104 actionable tasks: 68 executed, 36 from cache
Publishing build scan...
https://gradle.com/s/nckhcdyflleo2
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #600
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/600/display/redirect?page=changes>
Changes:
[iemejia] [BEAM-10066] Add support for ValueProviders in
------------------------------------------
[...truncated 292.78 KB...]
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:164)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Jun 08, 2020 12:45:21 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Jun 08, 2020 12:45:21 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Jun 08, 2020 12:45:21 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Jun 08, 2020 12:45:21 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Jun 08, 2020 12:45:21 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Jun 08, 2020 12:45:21 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Jun 08, 2020 12:45:22 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Jun 08, 2020 12:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 209 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Jun 08, 2020 12:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil$PackageAttributes forFileToStage
INFO: Staging custom dataflow-worker.jar as beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-i8Q2lcxfTNvMQdB4pbGC4kXboisFNJmjKf8q1LOX2Gc.jar
Jun 08, 2020 12:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.23.0-SNAPSHOT-yaehwbAG3SZUYWO_JIDt0VOCy-XJPd9n00qwH6TBAjM.jar
Jun 08, 2020 12:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.23.0-SNAPSHOT-gpQT5-A1GydMOc4VItXmx-ZXPoxIhRJZjHLCovSQexs.jar
Jun 08, 2020 12:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-ZJQijAJ9-0RdnnZBkNQLH8L6Z9t649m8DxrsJeT4lPQ.jar
Jun 08, 2020 12:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-tests-qggp5Sp-hlBn1AatmxEgUxnlB6EhHtcmkKfWIaj_c_U.jar
Jun 08, 2020 12:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.23.0-SNAPSHOT-g2QyZYDHB2-EMYRxJBSpQNUFDXLcb8592jIYMp1PreY.jar
Jun 08, 2020 12:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT-ckQQd-Z0iyXJzBZQi15HGQRHllVEk5r1pEgGbOrTUnQ.jar
Jun 08, 2020 12:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /tmp/test1698897465777741790.zip to gs://temp-storage-for-perf-tests/loadtests/staging/test-P8Xdb20rANiXT2S5RG-mBHBzwdomLXDYEh8ijWduyZ4.jar
Jun 08, 2020 12:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.23.0-SNAPSHOT-Wp5s6NSB9vRNrBWEEZ7Sakb6zeSnXT-FUrFI1pe-N2g.jar
Jun 08, 2020 12:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tests-5a_fSf0NTdZrtOUdwe4hwTjRVjwo89ZduXKPZ9zdGIA.jar
Jun 08, 2020 12:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-qHwWz36Dei-4vbx14svSWg64Ac-OOss0VUGNcsW9pNs.jar
Jun 08, 2020 12:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.23.0-SNAPSHOT-cq0zKPlf7jPUByM5Ac5YKAyFngkbMhGW_vKQWvcSxqA.jar
Jun 08, 2020 12:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-fim-UFpGFQVFJHR4iNBBlSLudEJ5urL-cA7Lbquxjgw.jar
Jun 08, 2020 12:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-8y-d9BTL6wgmcK8jhHjakvk9lfKl-BstHeY3mAa-TyU.jar
Jun 08, 2020 12:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-tests-QG1-HS21dvhzZCoAlgvvCTBURw_qTNAx7YWorK7eiSA.jar
Jun 08, 2020 12:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-cEMVCppFO3Dwfgzl_9QRtjZKi0uMQopnygAhzkNJBG4.jar
Jun 08, 2020 12:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT-SAEXMaCiCeE0N412cXoaIhb--sowvN0L9XDevuB3N_Q.jar
Jun 08, 2020 12:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-tests-uACOr4LkD97nbcdMz4fcs6ZzngI7ftW4YzGn_TE87QQ.jar
Jun 08, 2020 12:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-tests-5jL9EtvrXl-MokoFMobHdXL9CcBKccJm3RjNHHKxHFo.jar
Jun 08, 2020 12:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-GOc02-q1tfZWnc-7ix_uOGJIekb8JzosG4Y6O1p8ayc.jar
Jun 08, 2020 12:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.23.0-SNAPSHOT-tests-WMKOdJb9GxSlPfKIcJBPku7Ex7RhZnWtyAhVftZZ0zU.jar
Jun 08, 2020 12:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-i8Q2lcxfTNvMQdB4pbGC4kXboisFNJmjKf8q1LOX2Gc.jar
Jun 08, 2020 12:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-fn-execution-2.23.0-SNAPSHOT-R4MZ41b8HC4EtSOoIKUSow8rLkkUuOY-ab1R-QCW2Xw.jar
Jun 08, 2020 12:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-tests-WdS-_8868bFVlUbYKPIr8_lE__VsI53XrvHi6P2EFHc.jar
Jun 08, 2020 12:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/java-fn-execution/build/libs/beam-runners-java-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-java-fn-execution-2.23.0-SNAPSHOT-vbJ8z2jS85nw_sPc1ftqWba--TVokT2X6sb60sG021o.jar
Jun 08, 2020 12:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-IyUz1wAdsWEBv-xWRcsgT2QTLZ6xMRUVWqhpP87KyaM.jar
Jun 08, 2020 12:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-tests-dV4pfoAqKj0nBX7R8c42WL5tZJ0xd4zsZ8iCJkmDvSk.jar
Jun 08, 2020 12:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-unshaded-8_fsBkkRMUT-Y-P-mFiWpjx--YHJFCNRf6qG3q4yjQE.jar
Jun 08, 2020 12:45:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-i8Q2lcxfTNvMQdB4pbGC4kXboisFNJmjKf8q1LOX2Gc.jar
Jun 08, 2020 12:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.23.0-SNAPSHOT-ZscNv_NsPE2XFe5paGGOXXXGRdgwo1xtT73qw6ORfbA.jar
Jun 08, 2020 12:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.23.0-SNAPSHOT-bVhsphD01vyrHbz00RQ3P0C5j4ywz6YaBdiponADzWw.jar
Jun 08, 2020 12:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.23.0-SNAPSHOT-9Y2-x748fpT-QJ6fVnzv304AFEnip1DucXPUu4CsvmE.jar
Jun 08, 2020 12:45:25 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-i8Q2lcxfTNvMQdB4pbGC4kXboisFNJmjKf8q1LOX2Gc.jar&uploadType=resumable&upload_id=AAANsUmmgSs5tDO9rk2TRqnGHss-YfmxkP0USFYCZrGpBG_Tgg9iG0qYw2I903IG45p865u32-T-00AA-FqXFszQ4no18my-3A.
Jun 08, 2020 12:45:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar>
java.io.IOException: Upload failed for 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-i8Q2lcxfTNvMQdB4pbGC4kXboisFNJmjKf8q1LOX2Gc.jar'
at com.google.cloud.hadoop.util.BaseAbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(BaseAbstractGoogleAsyncWriteChannel.java:297)
at com.google.cloud.hadoop.util.BaseAbstractGoogleAsyncWriteChannel.close(BaseAbstractGoogleAsyncWriteChannel.java:214)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:221)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:168)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:152)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:136)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Jun 08, 2020 12:45:28 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-i8Q2lcxfTNvMQdB4pbGC4kXboisFNJmjKf8q1LOX2Gc.jar
Jun 08, 2020 12:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 31 files newly uploaded in 4 seconds
Jun 08, 2020 12:45:29 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Jun 08, 2020 12:45:29 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Jun 08, 2020 12:45:29 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Jun 08, 2020 12:45:29 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Jun 08, 2020 12:45:29 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Jun 08, 2020 12:45:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <91292 bytes, hash 0a6bcfe8586f5df61febd4eaaca8ef508acf871b21a4fa394e4f8865437262c5> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-CmvP6FhvXfYf69TqrKjvUIrPhxshpPo5Tk-IZUNyYsU.pb
Jun 08, 2020 12:45:29 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.23.0-SNAPSHOT
Jun 08, 2020 12:45:31 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-06-07_17_45_30-2910269694643723680?project=apache-beam-testing
Jun 08, 2020 12:45:31 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-06-07_17_45_30-2910269694643723680
Jun 08, 2020 12:45:31 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-06-07_17_45_30-2910269694643723680
Jun 08, 2020 12:45:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-06-08T00:45:30.074Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Jun 08, 2020 12:45:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-08T00:45:37.567Z: Worker configuration: n1-standard-1 in us-central1-a.
Jun 08, 2020 12:45:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-08T00:45:38.310Z: Expanding CoGroupByKey operations into optimizable parts.
Jun 08, 2020 12:45:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-08T00:45:38.348Z: Expanding GroupByKey operations into optimizable parts.
Jun 08, 2020 12:45:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-08T00:45:38.376Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jun 08, 2020 12:45:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-08T00:45:38.457Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jun 08, 2020 12:45:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-08T00:45:38.495Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Jun 08, 2020 12:45:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-08T00:45:38.529Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Jun 08, 2020 12:45:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-08T00:45:38.565Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Jun 08, 2020 12:45:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-08T00:45:38.969Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Jun 08, 2020 12:45:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-08T00:45:39.048Z: Starting 5 workers in us-central1-a...
Jun 08, 2020 12:46:04 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-06-08T00:46:03.408Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jun 08, 2020 12:46:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-08T00:46:07.784Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Jun 08, 2020 12:46:34 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-08T00:46:32.593Z: Workers have started successfully.
Jun 08, 2020 12:46:34 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-08T00:46:32.628Z: Workers have started successfully.
Jun 08, 2020 12:47:06 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-08T00:47:05.621Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Jun 08, 2020 12:47:06 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-08T00:47:05.819Z: Cleaning up.
Jun 08, 2020 12:47:06 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-08T00:47:05.891Z: Stopping worker pool...
Jun 08, 2020 12:48:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-08T00:48:47.177Z: Autoscaling: Resized worker pool from 5 to 0.
Jun 08, 2020 12:48:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-08T00:48:47.224Z: Worker pool stopped.
Jun 08, 2020 12:48:53 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-06-07_17_45_30-2910269694643723680 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): eed1a6b3-4940-47e3-8393-40b1e4e017a7 and timestamp: 2020-06-08T00:48:53.921000000Z:
Metric: Value:
fields_read 4375276.0
read_time 14.894
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Jun 08, 2020 12:48:54 AM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.022 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.031 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 2,5,main]) completed. Took 3 mins 41.493 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 33s
104 actionable tasks: 68 executed, 36 from cache
Publishing build scan...
https://gradle.com/s/kdz74lph6nkac
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #599
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/599/display/redirect>
Changes:
------------------------------------------
[...truncated 292.42 KB...]
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278)
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:164)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Jun 07, 2020 6:46:01 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Jun 07, 2020 6:46:01 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Jun 07, 2020 6:46:01 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Jun 07, 2020 6:46:01 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Jun 07, 2020 6:46:01 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Jun 07, 2020 6:46:01 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Jun 07, 2020 6:46:02 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Jun 07, 2020 6:46:04 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 209 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Jun 07, 2020 6:46:04 PM org.apache.beam.runners.dataflow.util.PackageUtil$PackageAttributes forFileToStage
INFO: Staging custom dataflow-worker.jar as beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-PI5zXzpM46VR1lmdrRUKrJ6Nn25E94sM6vGlYGoC3F8.jar
Jun 07, 2020 6:46:05 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.23.0-SNAPSHOT-WuECY0rCHVggAditn-TN5ISkAKvo9dnm0BQ3P3-r3ZI.jar
Jun 07, 2020 6:46:05 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-tests-sD97TDqzfWGmWkHei_ewEt9XL9aN0_Xu5E8k_BcVDzM.jar
Jun 07, 2020 6:46:05 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-Fh7CoA4sBLuqpC1l0Nop-a6Ayr9rsbpO-Kc3q586xN0.jar
Jun 07, 2020 6:46:05 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-GsBQ6g6ZVOdqYWN5hK6BEAc6abkUSLneg47-6nSJ7MU.jar
Jun 07, 2020 6:46:05 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.23.0-SNAPSHOT-sQqA7rHCrM9za4lICn70wcO7rqU2zw4clcwrCEcsoQs.jar
Jun 07, 2020 6:46:05 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-Wk8pzPFBpdzsJUbO6lZiQ0y_g1wpF0lVBxXWFyul-fo.jar
Jun 07, 2020 6:46:05 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-tests-WquHIR-W37TDmXN1aa6IAHwBYTtQlkBz6Kn7NL17V3Y.jar
Jun 07, 2020 6:46:05 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-tests-Sp9z624RPL6hyMXZt5lCpSbuVbN8POXW8TA1fhWnPr8.jar
Jun 07, 2020 6:46:05 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-PI5zXzpM46VR1lmdrRUKrJ6Nn25E94sM6vGlYGoC3F8.jar
Jun 07, 2020 6:46:05 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT-WFaB3TB0PAWviE5wY_SrBTS-R5qqr7ShcB5zydE05Dw.jar
Jun 07, 2020 6:46:05 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-tests-EO9NsHJSK5n3SIelO3G8NXfn8ljoIxwYnubt3XHa4T8.jar
Jun 07, 2020 6:46:05 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-pKlVdnV9hQx73HFtup-iBC-BXoiO361XUg4EFCf_OEs.jar
Jun 07, 2020 6:46:05 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.23.0-SNAPSHOT-2tpzPCFh46Zmy8I8p4hzbHG6n-UIQ0ztwsqMB8iu_aY.jar
Jun 07, 2020 6:46:05 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-KPUQoQ0pnkv2QVuv8s5__Yu72AQE8wrce2-1kUdCRvs.jar
Jun 07, 2020 6:46:05 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tests-aeRAywJOSNVgIugRyci7-7Z6QGeWG1dlZVnxfVb99u4.jar
Jun 07, 2020 6:46:05 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-tests-eomXwZh93zTp_6OgARHRLPN0GXaAoIhXTUuoKtRM83M.jar
Jun 07, 2020 6:46:05 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /tmp/test8218042934746401552.zip to gs://temp-storage-for-perf-tests/loadtests/staging/test-9ilShvVy7giTsOcm84vOxbN38ho2MmUQI6DpXLFXw-4.jar
Jun 07, 2020 6:46:05 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.23.0-SNAPSHOT-n2NamAmDJGhRZ5xutReSnWYYbLHJXrrZkXDrdXmV5l4.jar
Jun 07, 2020 6:46:05 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.23.0-SNAPSHOT-tests-Gax_LKCY6dpvuRffvDPTSQRQjLlqnww88d0DWtomxiw.jar
Jun 07, 2020 6:46:05 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-fn-execution-2.23.0-SNAPSHOT-g_3BmhjwDiP2MaODtgPenx-Lf950TxiOgM0Ew1_LQkU.jar
Jun 07, 2020 6:46:05 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-7EWso_S7c3vbEePtek1IKEnD_ButlfqCKnVUYvZHbyQ.jar
Jun 07, 2020 6:46:05 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.23.0-SNAPSHOT-ONN0E3RVxIY9eFVMxJe--pDJgZZphCjQBzkUPyciwsM.jar
Jun 07, 2020 6:46:05 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/java-fn-execution/build/libs/beam-runners-java-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-java-fn-execution-2.23.0-SNAPSHOT-3kOAf7DUg_2DQB2fs6JJMrU_T-ucq45BWv7cXlusDSk.jar
Jun 07, 2020 6:46:05 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-tests-NjOMJUVffDO0V6B4PYThQ2JO1t3se9XWgZTbUPLSj3E.jar
Jun 07, 2020 6:46:05 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-unshaded-ePegScFH-5suQxE2zGmKHssb2VovVd-rST3ZrZ46Yhs.jar
Jun 07, 2020 6:46:05 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-2r-A0SETamkx8WcH67wOzPrQ-ym-Jun7bQt9ZnsHchQ.jar
Jun 07, 2020 6:46:05 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT-p-QFWl3Q4e0wcupXf_WJxhM-A6jJ6Fnjc4P9208wNLg.jar
Jun 07, 2020 6:46:05 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-PI5zXzpM46VR1lmdrRUKrJ6Nn25E94sM6vGlYGoC3F8.jar
Jun 07, 2020 6:46:06 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.23.0-SNAPSHOT-0E_49VWq_-G3A_8h086qSy_d_G3-jVf7q4gxKGi2KSU.jar
Jun 07, 2020 6:46:06 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.23.0-SNAPSHOT-lPkno1LsvksL0o5bL_y14cWzqtFKxDksWuvlo57oezE.jar
Jun 07, 2020 6:46:06 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.23.0-SNAPSHOT-6My0ApGlgTm0M_nEE26_vIqL9gx7GUbM1rGNKSQO224.jar
Jun 07, 2020 6:46:06 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-PI5zXzpM46VR1lmdrRUKrJ6Nn25E94sM6vGlYGoC3F8.jar&uploadType=resumable&upload_id=AAANsUmU_2TduqkBnGL0S5Mc-me_95tAh_-qxkhBGrJh4PEV8yriKEiVm98HUCXLPcqp8vhZ75u-oYFB2C7PgItv0sw.
Jun 07, 2020 6:46:06 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar>
java.io.IOException: Upload failed for 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-PI5zXzpM46VR1lmdrRUKrJ6Nn25E94sM6vGlYGoC3F8.jar'
at com.google.cloud.hadoop.util.BaseAbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(BaseAbstractGoogleAsyncWriteChannel.java:297)
at com.google.cloud.hadoop.util.BaseAbstractGoogleAsyncWriteChannel.close(BaseAbstractGoogleAsyncWriteChannel.java:214)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:221)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:168)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:152)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:136)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Jun 07, 2020 6:46:10 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-PI5zXzpM46VR1lmdrRUKrJ6Nn25E94sM6vGlYGoC3F8.jar
Jun 07, 2020 6:46:11 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 31 files newly uploaded in 6 seconds
Jun 07, 2020 6:46:11 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Jun 07, 2020 6:46:11 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Jun 07, 2020 6:46:11 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Jun 07, 2020 6:46:11 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Jun 07, 2020 6:46:11 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Jun 07, 2020 6:46:11 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <91292 bytes, hash 5d26eb7476d544b9ebf59cf66639ce0f324dab48ea61c7a4036baf9d07672eb3> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-XSbrdHbVRLnr9Zz2ZjnODzJNq0jqYcekA2uvnQdnLrM.pb
Jun 07, 2020 6:46:12 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.23.0-SNAPSHOT
Jun 07, 2020 6:46:13 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-06-07_11_46_12-3025072561693460765?project=apache-beam-testing
Jun 07, 2020 6:46:13 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-06-07_11_46_12-3025072561693460765
Jun 07, 2020 6:46:13 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-06-07_11_46_12-3025072561693460765
Jun 07, 2020 6:46:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-06-07T18:46:12.469Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Jun 07, 2020 6:46:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-07T18:46:19.800Z: Worker configuration: n1-standard-1 in us-central1-a.
Jun 07, 2020 6:46:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-07T18:46:20.739Z: Expanding CoGroupByKey operations into optimizable parts.
Jun 07, 2020 6:46:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-07T18:46:20.769Z: Expanding GroupByKey operations into optimizable parts.
Jun 07, 2020 6:46:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-07T18:46:20.793Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jun 07, 2020 6:46:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-07T18:46:20.850Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jun 07, 2020 6:46:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-07T18:46:20.876Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Jun 07, 2020 6:46:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-07T18:46:20.896Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Jun 07, 2020 6:46:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-07T18:46:20.924Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Jun 07, 2020 6:46:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-07T18:46:21.348Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Jun 07, 2020 6:46:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-07T18:46:21.424Z: Starting 5 workers in us-central1-a...
Jun 07, 2020 6:46:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-06-07T18:46:43.834Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jun 07, 2020 6:46:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-07T18:46:49.752Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Jun 07, 2020 6:47:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-07T18:47:10.711Z: Workers have started successfully.
Jun 07, 2020 6:47:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-07T18:47:10.739Z: Workers have started successfully.
Jun 07, 2020 6:47:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-07T18:47:41.871Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Jun 07, 2020 6:47:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-07T18:47:42.047Z: Cleaning up.
Jun 07, 2020 6:47:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-07T18:47:42.116Z: Stopping worker pool...
Jun 07, 2020 6:49:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-07T18:49:27.493Z: Autoscaling: Resized worker pool from 5 to 0.
Jun 07, 2020 6:49:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-07T18:49:27.531Z: Worker pool stopped.
Jun 07, 2020 6:49:33 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-06-07_11_46_12-3025072561693460765 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 273b8906-2971-45f5-a22d-f903f95f9193 and timestamp: 2020-06-07T18:49:33.350000000Z:
Metric: Value:
read_time 13.962
fields_read 4375276.0
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Jun 07, 2020 6:49:33 PM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.025 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.034 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 9,5,main]) completed. Took 3 mins 42.799 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 56s
104 actionable tasks: 68 executed, 36 from cache
Publishing build scan...
https://gradle.com/s/zxvw3f46ufqze
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #598
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/598/display/redirect>
Changes:
------------------------------------------
[...truncated 292.96 KB...]
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:164)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Jun 07, 2020 12:45:26 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Jun 07, 2020 12:45:26 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Jun 07, 2020 12:45:26 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Jun 07, 2020 12:45:26 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Jun 07, 2020 12:45:27 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Jun 07, 2020 12:45:27 PM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Jun 07, 2020 12:45:28 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Jun 07, 2020 12:45:29 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 209 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Jun 07, 2020 12:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil$PackageAttributes forFileToStage
INFO: Staging custom dataflow-worker.jar as beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-nPU1ow7nnaquaksvBZVYN8NDJLdub-fXWiQ74hTRQ50.jar
Jun 07, 2020 12:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-unshaded-1M2AuVF7H7Svu1EkwmEJ4kn5szKIAlHfReDMoLoBXbI.jar
Jun 07, 2020 12:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.23.0-SNAPSHOT-YPZ2oAtfvD72MHnWqH5NPeVQ9im120QUjvA_mLvSes4.jar
Jun 07, 2020 12:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-tests-QDZMuckVLtc_j2VBek14E1yxEjucYTrwv1piegDniGw.jar
Jun 07, 2020 12:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-tests--5ga0jtL17zhJL4l-HjPoMcyRZDkWYCuVj5tzgYBYe8.jar
Jun 07, 2020 12:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.23.0-SNAPSHOT-elN4SByRnBF7f3lz9LeAzIjDhmj0avyJaXWFrRf2dc4.jar
Jun 07, 2020 12:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-nPU1ow7nnaquaksvBZVYN8NDJLdub-fXWiQ74hTRQ50.jar
Jun 07, 2020 12:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-tests-oxAewsnTgeeQWi5kAv5FG1aLD36Uf_pjeoHS27dmDkw.jar
Jun 07, 2020 12:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.23.0-SNAPSHOT-2wuHGQFbmRJZaBRDTvjKG5vy95cgiBsMrFPE47XjyZ4.jar
Jun 07, 2020 12:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-ez8Q8zjPjxbQ7qkWdtrb6XDpLPoN-2SZvZHthoIsjkw.jar
Jun 07, 2020 12:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT-HBpQqiB2j5TLX2U7EsBQoZA5eVbyospe55m5VBcrVv4.jar
Jun 07, 2020 12:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/java-fn-execution/build/libs/beam-runners-java-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-java-fn-execution-2.23.0-SNAPSHOT-vhRV_dBNO7BR35o12Ky7Vbxe_9A7nx5ZB_JICiAtUFU.jar
Jun 07, 2020 12:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tests-VWD88NuNYgMlyF65KaFOjFec7QWSoxWpjaD76382Lpk.jar
Jun 07, 2020 12:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT-j0jXv0cJsOsK2FjyYoRneSWKK5zBj08LobAUI-aPbpw.jar
Jun 07, 2020 12:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /tmp/test3511965468851296173.zip to gs://temp-storage-for-perf-tests/loadtests/staging/test-wdPHQSqQL5F7hjQa5-iQRkblKVi-N9vf2B_xSo8v3q0.jar
Jun 07, 2020 12:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-tests-S1BsJliyBr1b9PilmXoqYZ-jtsQUA-cDpQOd6qVe4vY.jar
Jun 07, 2020 12:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-ppsLrB9C_eGziCoNuZiU8WTY4SAoyFo8NarHIF3BOjs.jar
Jun 07, 2020 12:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-UAgB8QsqFtFiZG9JwSC9sswaoFzXPFwgfqX8NBkMaA4.jar
Jun 07, 2020 12:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-DFF57ZFwHRTn3UTmxkwN0JdS2ALF_mqyjca2hRMKX5k.jar
Jun 07, 2020 12:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.23.0-SNAPSHOT-ya3PrYdoQmwHXcDW28wnsdaa6m4xYQya6iwi-EbvOLY.jar
Jun 07, 2020 12:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-tests-Lt651GBOAQWEaYZ2tIa9KWMtj-CcMkDtILN_dQBeU80.jar
Jun 07, 2020 12:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.23.0-SNAPSHOT-tests-nkf5ILmUqIYYRDjvyU8F41-Q4KBnXcW-8kL5TeK4ZZI.jar
Jun 07, 2020 12:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-f-w51Y10mBgO4ZUvzvO6Bg3cW_WEIaPkim54R-OxGzk.jar
Jun 07, 2020 12:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-L_M9XIytC31uUeBmNyric4DYq97kGh_XXCL3ov5Zh4k.jar
Jun 07, 2020 12:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-tests-J5D-jYJjLzd3uDWwaiKzVCOawl2ZA5I9gswbTCkU82Y.jar
Jun 07, 2020 12:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.23.0-SNAPSHOT-B8QOS7q7M8YvKF8HRiAOGQTcZ8RTYxtn71Su0fDyycE.jar
Jun 07, 2020 12:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-lovmrFz3865nrvguzYl_n0KYACjvFlr8Z-6JQ50dv10.jar
Jun 07, 2020 12:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-nPU1ow7nnaquaksvBZVYN8NDJLdub-fXWiQ74hTRQ50.jar
Jun 07, 2020 12:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-fn-execution-2.23.0-SNAPSHOT-TNwEBnKk8tG04ZhIuikNDZVxWajlA_VWpO3eDYpZAPQ.jar
Jun 07, 2020 12:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.23.0-SNAPSHOT-oFBHP6fHsWSvmCi5yLPMToC4wF_1IFQ_wa4itzfOUmA.jar
Jun 07, 2020 12:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.23.0-SNAPSHOT-sy5ZReXiWANvuLWE24S8Jm6OZui3EK5Q-6lg025xnls.jar
Jun 07, 2020 12:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.23.0-SNAPSHOT-Ix5o7GaYOiG99L2YPWiLGEaEXdEUmiatzYEi2F1jNlw.jar
Jun 07, 2020 12:45:31 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-nPU1ow7nnaquaksvBZVYN8NDJLdub-fXWiQ74hTRQ50.jar&uploadType=resumable&upload_id=AAANsUmfDfBfDIbqlyi7LlTvRpGNRHoCteI8XSzkvPjSskEiIhikPqlqYQexIr38irf_pbkO5NS9OnAuhuguZeBZ8T0HTzuaRA.
Jun 07, 2020 12:45:31 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar>
java.io.IOException: Upload failed for 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-nPU1ow7nnaquaksvBZVYN8NDJLdub-fXWiQ74hTRQ50.jar'
at com.google.cloud.hadoop.util.BaseAbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(BaseAbstractGoogleAsyncWriteChannel.java:297)
at com.google.cloud.hadoop.util.BaseAbstractGoogleAsyncWriteChannel.close(BaseAbstractGoogleAsyncWriteChannel.java:214)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:221)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:168)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:152)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:136)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Jun 07, 2020 12:45:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-nPU1ow7nnaquaksvBZVYN8NDJLdub-fXWiQ74hTRQ50.jar
Jun 07, 2020 12:45:36 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 31 files newly uploaded in 6 seconds
Jun 07, 2020 12:45:36 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Jun 07, 2020 12:45:36 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Jun 07, 2020 12:45:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Jun 07, 2020 12:45:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Jun 07, 2020 12:45:37 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Jun 07, 2020 12:45:37 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <91292 bytes, hash e8ea23aed0881039a1858e1acc831255778bd4177cd80bdb27e8afa0f977d6bf> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-6OojrtCIEDmhhY4azIMSVXeL1Bd82AvbJ-ivoPl31r8.pb
Jun 07, 2020 12:45:37 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.23.0-SNAPSHOT
Jun 07, 2020 12:45:38 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-06-07_05_45_37-8902117900992356487?project=apache-beam-testing
Jun 07, 2020 12:45:38 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-06-07_05_45_37-8902117900992356487
Jun 07, 2020 12:45:38 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-06-07_05_45_37-8902117900992356487
Jun 07, 2020 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-06-07T12:45:37.684Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Jun 07, 2020 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-07T12:45:44.836Z: Worker configuration: n1-standard-1 in us-central1-a.
Jun 07, 2020 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-07T12:45:45.534Z: Expanding CoGroupByKey operations into optimizable parts.
Jun 07, 2020 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-07T12:45:45.568Z: Expanding GroupByKey operations into optimizable parts.
Jun 07, 2020 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-07T12:45:45.599Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jun 07, 2020 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-07T12:45:45.681Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jun 07, 2020 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-07T12:45:45.717Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Jun 07, 2020 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-07T12:45:45.750Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Jun 07, 2020 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-07T12:45:45.777Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Jun 07, 2020 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-07T12:45:46.158Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Jun 07, 2020 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-07T12:45:46.233Z: Starting 5 workers in us-central1-a...
Jun 07, 2020 12:45:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-06-07T12:45:57.170Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jun 07, 2020 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-07T12:46:16.969Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running stage(s).
Jun 07, 2020 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-07T12:46:16.993Z: Resized worker pool to 4, though goal was 5. This could be a quota issue.
Jun 07, 2020 12:46:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-07T12:46:22.538Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Jun 07, 2020 12:46:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-07T12:46:40.852Z: Workers have started successfully.
Jun 07, 2020 12:46:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-07T12:46:40.877Z: Workers have started successfully.
Jun 07, 2020 12:47:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-07T12:47:15.620Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Jun 07, 2020 12:47:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-07T12:47:15.799Z: Cleaning up.
Jun 07, 2020 12:47:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-07T12:47:15.870Z: Stopping worker pool...
Jun 07, 2020 12:48:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-07T12:48:46.139Z: Autoscaling: Resized worker pool from 5 to 0.
Jun 07, 2020 12:48:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-07T12:48:46.186Z: Worker pool stopped.
Jun 07, 2020 12:48:51 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-06-07_05_45_37-8902117900992356487 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 37e44fa5-4d3f-4c0f-95f2-977f6170c8ec and timestamp: 2020-06-07T12:48:51.796000000Z:
Metric: Value:
fields_read 4375276.0
read_time 16.215
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Jun 07, 2020 12:48:52 PM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.027 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.032 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker for ':' Thread 9,5,main]) completed. Took 3 mins 34.348 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 27s
104 actionable tasks: 68 executed, 36 from cache
Publishing build scan...
https://gradle.com/s/xlqn2hbtfsreo
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_SQLBigQueryIO_Batch_Performance_Test_Java #597
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/597/display/redirect>
Changes:
------------------------------------------
[...truncated 293.44 KB...]
at org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:164)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Jun 07, 2020 6:45:38 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Jun 07, 2020 6:45:38 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`, `HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND `HACKER_NEWS`.`score` > 2
Jun 07, 2020 6:45:38 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Jun 07, 2020 6:45:38 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Jun 07, 2020 6:45:38 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by, type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8, 'job'))>($5, 2)}, unsupported{}]])
Jun 07, 2020 6:45:38 AM org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` = 'job') AND `score` > 2
Jun 07, 2020 6:45:39 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Jun 07, 2020 6:45:41 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 209 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Jun 07, 2020 6:45:41 AM org.apache.beam.runners.dataflow.util.PackageUtil$PackageAttributes forFileToStage
INFO: Staging custom dataflow-worker.jar as beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-TDk-0YA8DeOvYwmAjukD_-Q59VuGpdcTb1Tj1sZnGjs.jar
Jun 07, 2020 6:45:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.23.0-SNAPSHOT-TAbLWyef9YIEMrih11gEwyslu4wbuJoEFOlxN-4Y3nc.jar
Jun 07, 2020 6:45:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-tests-B1ku24dPIiex5smytBi1K-zKOL0ZvTjWbSBanUrCasU.jar
Jun 07, 2020 6:45:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.23.0-SNAPSHOT-64nckog5StiN6GrAz4VOu4FNUtsN9WAj2uWET5KjSqY.jar
Jun 07, 2020 6:45:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /tmp/test3797716128178161643.zip to gs://temp-storage-for-perf-tests/loadtests/staging/test-Z-rA076arVJ4WQKpI51uHzjyr3kOEJUnqyOpuSdFOkc.jar
Jun 07, 2020 6:45:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-GkMVLcWTf0wvAS2pv-lo_r9DrPFUIu04Iu6po96IjFE.jar
Jun 07, 2020 6:45:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-tests-gESDML16K0BoiEKWFxj_TGcZSUknWvS8OCqc-wZ88v4.jar
Jun 07, 2020 6:45:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-zTTe58il4QNOVIk3WOXz6HRtXufV1z5xrb9nOWFJ8K4.jar
Jun 07, 2020 6:45:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.23.0-SNAPSHOT-5akWXtxQ7ujUROVFBUjNSYjI2UPPqFc2TlcNQ3Fe-gc.jar
Jun 07, 2020 6:45:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/java-fn-execution/build/libs/beam-runners-java-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-java-fn-execution-2.23.0-SNAPSHOT-g6ErKNp-58DMa9Bs3dumIbNje5m-GADpV8CbbNFdCWk.jar
Jun 07, 2020 6:45:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-tests-GOKyz442gICI_ZQ-oT_PMspydLMDODGm2LRWgQKkmiM.jar
Jun 07, 2020 6:45:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-TDk-0YA8DeOvYwmAjukD_-Q59VuGpdcTb1Tj1sZnGjs.jar
Jun 07, 2020 6:45:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.23.0-SNAPSHOT-MG65hWCnXoDfIBCfQl-bdNV_UtKAY3X52oVzWO9UmpE.jar
Jun 07, 2020 6:45:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-fn-execution-2.23.0-SNAPSHOT--vyZh7EAUu5_mGq3TI9AOVqRFTsvqvpOcncBF5tqvB4.jar
Jun 07, 2020 6:45:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-xX6taBM3Q7my7PbLS89Jf5UJCFf70N9s5GBqjAhdP3g.jar
Jun 07, 2020 6:45:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT-8SIYIZfu--wd7L_v5_9d9KyvM3jhDT92StejXpdW95Q.jar
Jun 07, 2020 6:45:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.23.0-SNAPSHOT-tests-uNCgcGgy0lTtQ47YYDQmemI4qa4m6PF7rhd2pwkIvkQ.jar
Jun 07, 2020 6:45:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.23.0-SNAPSHOT-pxdOLaD_k6hkWws8IOnD-00qsx7Qfzoo_p3Z4x1HyFw.jar
Jun 07, 2020 6:45:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-tests-ioiwnppjjMNreRg5Y_mrmNgCwbx6GUOkAHm9A8_7f24.jar
Jun 07, 2020 6:45:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.23.0-SNAPSHOT-tests--h5IpFCnGQ53Q4WBEk5HPlSMUeWkeJYAh4gWJsWyMxw.jar
Jun 07, 2020 6:45:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.23.0-SNAPSHOT-tests-hlmhN4i6_oAdmkGsqMM0DISzxJj5ywsUG6QqITu5CTc.jar
Jun 07, 2020 6:45:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.23.0-SNAPSHOT-NMzpHgSWZy2hzGwekQo4fgU8AH_C1di7bM7H83RkoGk.jar
Jun 07, 2020 6:45:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT-tests.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-tests-tFvdO7BRWoX_fQTG3_6EoHpl2aV1t6Gx8cC9KiJWEiA.jar
Jun 07, 2020 6:45:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.23.0-SNAPSHOT-sf80JWLAmqNtxbSy_uNo16hbIblhkWoezsoDkGK0Qx4.jar
Jun 07, 2020 6:45:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.23.0-SNAPSHOT-unshaded.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.23.0-SNAPSHOT-unshaded-LmtoCFdi1QbPBWdwiVZrKQTDxcp_rtFpKGyce8aWmFU.jar
Jun 07, 2020 6:45:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.23.0-SNAPSHOT-dFrEYKXDNc4nZu1dSK4c5pw9Nryt_47NyPcUZ6P5xfY.jar
Jun 07, 2020 6:45:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.23.0-SNAPSHOT-5fI0uHlnExa5yJ0rfG-X_n-UARoDaPnMZG2Urvc4-Z8.jar
Jun 07, 2020 6:45:41 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.23.0-SNAPSHOT-uaZxAuTCGdXN9NiGQ_cXnxdoxfjOiyF_4OZ1jqLMlnc.jar
Jun 07, 2020 6:45:42 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-TDk-0YA8DeOvYwmAjukD_-Q59VuGpdcTb1Tj1sZnGjs.jar
Jun 07, 2020 6:45:42 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.23.0-SNAPSHOT-HopY2kwH7wT56SzLGJJJPQtRPVuN_XEkrsfLqDjXjZ8.jar
Jun 07, 2020 6:45:42 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.23.0-SNAPSHOT-lpTtZJdMoWmP6V0Gju5OU65AmsRuFFrvo5bhNHgCd0U.jar
Jun 07, 2020 6:45:42 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.23.0-SNAPSHOT-PULDRYGH6nH77DEuu6qgg0I7kSUSF-iFGl67qfkHGkY.jar
Jun 07, 2020 6:45:42 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 412, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?ifGenerationMatch=0&name=loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-TDk-0YA8DeOvYwmAjukD_-Q59VuGpdcTb1Tj1sZnGjs.jar&uploadType=resumable&upload_id=AAANsUlY3QifyVd4cxZgvknKy2MculB75QGpTHos_W12-3dqjvKRgg8Gr6gYwpb-8QCKkbY9H1pbkcyViX2nx7WiWL-Ck6khKQ.
Jun 07, 2020 6:45:42 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar>
java.io.IOException: Upload failed for 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-TDk-0YA8DeOvYwmAjukD_-Q59VuGpdcTb1Tj1sZnGjs.jar'
at com.google.cloud.hadoop.util.BaseAbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(BaseAbstractGoogleAsyncWriteChannel.java:297)
at com.google.cloud.hadoop.util.BaseAbstractGoogleAsyncWriteChannel.close(BaseAbstractGoogleAsyncWriteChannel.java:214)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:221)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:168)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:152)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:136)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 412 Precondition Failed
{
"code" : 412,
"errors" : [ {
"domain" : "global",
"location" : "If-Match",
"locationType" : "header",
"message" : "Precondition Failed",
"reason" : "conditionNotMet"
} ],
"message" : "Precondition Failed"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Jun 07, 2020 6:45:46 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.23.0-SNAPSHOT-TDk-0YA8DeOvYwmAjukD_-Q59VuGpdcTb1Tj1sZnGjs.jar
Jun 07, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 178 files cached, 31 files newly uploaded in 5 seconds
Jun 07, 2020 6:45:47 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource) as step s1
Jun 07, 2020 6:45:47 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Jun 07, 2020 6:45:47 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Jun 07, 2020 6:45:47 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Jun 07, 2020 6:45:47 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/
Jun 07, 2020 6:45:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <91292 bytes, hash dcafabefe26c472e9ee2b5a416d64b4bf9d7df723b7a48a72eac88b4fd75117e> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-3K-r7-JsRy6e4rWkFtZLS_nX33I7ekinLqyItP11EX4.pb
Jun 07, 2020 6:45:48 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.23.0-SNAPSHOT
Jun 07, 2020 6:45:49 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-06-06_23_45_48-8339189321410440014?project=apache-beam-testing
Jun 07, 2020 6:45:49 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-06-06_23_45_48-8339189321410440014
Jun 07, 2020 6:45:49 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-06-06_23_45_48-8339189321410440014
Jun 07, 2020 6:45:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-06-07T06:45:48.156Z: The requested max number of workers (5) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Jun 07, 2020 6:45:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-07T06:45:54.862Z: Worker configuration: n1-standard-1 in us-central1-a.
Jun 07, 2020 6:45:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-07T06:45:55.925Z: Expanding CoGroupByKey operations into optimizable parts.
Jun 07, 2020 6:45:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-07T06:45:55.974Z: Expanding GroupByKey operations into optimizable parts.
Jun 07, 2020 6:45:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-07T06:45:56.014Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jun 07, 2020 6:45:57 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-07T06:45:56.106Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jun 07, 2020 6:45:57 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-07T06:45:56.142Z: Fusing consumer BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)
Jun 07, 2020 6:45:57 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-07T06:45:56.185Z: Fusing consumer BeamCalcRel_285/ParDo(Calc) into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Jun 07, 2020 6:45:57 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-07T06:45:56.223Z: Fusing consumer ParDo(TimeMonitor) into BeamCalcRel_285/ParDo(Calc)
Jun 07, 2020 6:45:57 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-07T06:45:56.718Z: Executing operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Jun 07, 2020 6:45:57 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-07T06:45:56.798Z: Starting 5 workers in us-central1-a...
Jun 07, 2020 6:46:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-06-07T06:46:10.677Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jun 07, 2020 6:46:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-07T06:46:23.046Z: Autoscaling: Raised the number of workers to 5 based on the rate of progress in the currently running stage(s).
Jun 07, 2020 6:46:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-07T06:46:40.613Z: Workers have started successfully.
Jun 07, 2020 6:46:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-07T06:46:40.646Z: Workers have started successfully.
Jun 07, 2020 6:47:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-07T06:47:18.429Z: Finished operation BeamPushDownIOSourceRel_229/Read Input BQ Rows with push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Jun 07, 2020 6:47:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-07T06:47:18.628Z: Cleaning up.
Jun 07, 2020 6:47:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-07T06:47:18.729Z: Stopping worker pool...
Jun 07, 2020 6:49:13 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-07T06:49:12.648Z: Autoscaling: Resized worker pool from 5 to 0.
Jun 07, 2020 6:49:13 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-06-07T06:49:12.693Z: Worker pool stopped.
Jun 07, 2020 6:49:20 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-06-06_23_45_48-8339189321410440014 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): 68a080c2-830f-404e-9184-77a869d45c6a and timestamp: 2020-06-07T06:49:20.550000000Z:
Metric: Value:
read_time 19.431
fields_read 4375276.0
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT > readUsingDirectReadMethodPushDown STANDARD_ERROR
Jun 07, 2020 6:49:20 AM org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithSettings
WARNING: Missing property -- measurement/database. Metrics won't be published.
Gradle Test Executor 2 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.022 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.032 secs) into: <https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Daemon worker,5,main]) completed. Took 3 mins 51.098 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 0s
104 actionable tasks: 70 executed, 34 from cache
Publishing build scan...
https://gradle.com/s/or4puiquwgs3i
Stopped 1 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org