You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@beam.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2022/05/20 14:58:37 UTC
Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Batch_Java11 #336
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Batch_Java11/336/display/redirect?page=changes>
Changes:
[chamikaramj] Corrects I/O connectors availability status in Beam Website.
[singh.vikash2310] fixed typos in README.md
[noreply] Update the PTransform and associated APIs to be less class-based.
[noreply] Vortex performance improvement: Enable multiple stream clients per
[noreply] [BEAM-14488] Alias async flags. (#17711)
[noreply] [BEAM-14487] Make drain & update terminal states. (#17710)
[noreply] [BEAM-14484] Improve behavior surrounding primary roots in
[noreply] Improve validation error message (#17719)
[noreply] Remove unused validation configurations. (#17705)
[bulat.safiullin] [BEAM-14418] added arrows to slider
[noreply] Minor: Bump Dataflow container versions (#17684)
------------------------------------------
[...truncated 49.81 KB...]
a7934564e6b9: Preparing
1b7cceb6a07c: Preparing
b274e8788e0c: Preparing
78658088978a: Preparing
834cc28b31a7: Waiting
a7934564e6b9: Waiting
3b2356b88239: Waiting
1b7cceb6a07c: Waiting
b274e8788e0c: Waiting
bd8eda81a84b: Waiting
78658088978a: Waiting
b9ce114f9aa9: Waiting
cbd16b72e80e: Waiting
07df3576ab77: Waiting
444ee16e3285: Waiting
c751f44f7ef6: Waiting
bfebdf477ef9: Waiting
0d3f6119f339: Waiting
a6bcf17321cd: Waiting
6e29b7dabdba: Waiting
13b4c87e4181: Pushed
5b5041b680d9: Pushed
f00f1e6070bd: Pushed
94e56ba7beef: Pushed
1c2c9df99a58: Pushed
6e29b7dabdba: Pushed
444ee16e3285: Pushed
0d3f6119f339: Pushed
bfebdf477ef9: Pushed
1e97e8e00e2f: Pushed
a6bcf17321cd: Pushed
cbd16b72e80e: Layer already exists
07df3576ab77: Layer already exists
c751f44f7ef6: Pushed
3b2356b88239: Layer already exists
a7934564e6b9: Layer already exists
1b7cceb6a07c: Layer already exists
b274e8788e0c: Layer already exists
78658088978a: Layer already exists
bd8eda81a84b: Pushed
b9ce114f9aa9: Pushed
834cc28b31a7: Pushed
20220520145239: digest: sha256:700b785f90f906fc9875715012894fd6122db9e5a681bcfd47154886a7254c7a size: 4935
> Task :sdks:java:testing:load-tests:run
May 20, 2022 2:54:40 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
May 20, 2022 2:54:41 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 222 files. Enable logging at DEBUG level to see which files will be staged.
May 20, 2022 2:54:42 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
May 20, 2022 2:54:42 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
May 20, 2022 2:54:45 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 222 files from PipelineOptions.filesToStage to staging location to prepare for execution.
May 20, 2022 2:54:47 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 222 files cached, 0 files newly uploaded in 2 seconds
May 20, 2022 2:54:47 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
May 20, 2022 2:54:47 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <114842 bytes, hash 7b37bc93197625deb5095a0a15f7e018c893e27fef3668f90abdb6b333fe67c6> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-eze8kxl2Jd61CVoKFffgGMiT4n_vNmj5Cr22szP-Z8Y.pb
May 20, 2022 2:54:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input as step s1
May 20, 2022 2:54:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s2
May 20, 2022 2:54:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s3
May 20, 2022 2:54:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input as step s4
May 20, 2022 2:54:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s5
May 20, 2022 2:54:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s6
May 20, 2022 2:54:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s7
May 20, 2022 2:54:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s8
May 20, 2022 2:54:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s9
May 20, 2022 2:54:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s10
May 20, 2022 2:54:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s11
May 20, 2022 2:54:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s12
May 20, 2022 2:54:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s13
May 20, 2022 2:54:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s14
May 20, 2022 2:54:50 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.40.0-SNAPSHOT
May 20, 2022 2:54:51 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-05-20_07_54_50-4752207528063837205?project=apache-beam-testing
May 20, 2022 2:54:51 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2022-05-20_07_54_50-4752207528063837205
May 20, 2022 2:54:51 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2022-05-20_07_54_50-4752207528063837205
May 20, 2022 2:54:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2022-05-20T14:54:52.675Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20batch0cogbk01-jenkins-052014-r9qp. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
May 20, 2022 2:54:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-20T14:54:58.865Z: Worker configuration: e2-standard-2 in us-central1-b.
May 20, 2022 2:55:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-20T14:54:59.462Z: Expanding SplittableParDo operations into optimizable parts.
May 20, 2022 2:55:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-20T14:54:59.622Z: Expanding CollectionToSingleton operations into optimizable parts.
May 20, 2022 2:55:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-20T14:54:59.678Z: Expanding CoGroupByKey operations into optimizable parts.
May 20, 2022 2:55:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-20T14:54:59.749Z: Expanding GroupByKey operations into optimizable parts.
May 20, 2022 2:55:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-20T14:54:59.806Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 20, 2022 2:55:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-20T14:54:59.840Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
May 20, 2022 2:55:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-20T14:54:59.873Z: Fusing unzipped copy of CoGroupByKey/GBK/Write, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
May 20, 2022 2:55:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-20T14:54:59.915Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/Read
May 20, 2022 2:55:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-20T14:54:59.941Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
May 20, 2022 2:55:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-20T14:54:59.977Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
May 20, 2022 2:55:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-20T14:55:00.001Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
May 20, 2022 2:55:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-20T14:55:00.033Z: Fusing consumer CoGroupByKey/GBK/Write into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
May 20, 2022 2:55:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-20T14:55:00.063Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
May 20, 2022 2:55:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-20T14:55:00.098Z: Fusing consumer Read-input-ParDo-BoundedSourceAsSDFWrapper--ParMultiDo-BoundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
May 20, 2022 2:55:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-20T14:55:00.122Z: Fusing consumer Read-input-ParDo-BoundedSourceAsSDFWrapper--ParMultiDo-BoundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-BoundedSourceAsSDFWrapper--ParMultiDo-BoundedSourceAsSDFWrapper-/PairWithRestriction
May 20, 2022 2:55:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-20T14:55:00.148Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read-input-ParDo-BoundedSourceAsSDFWrapper--ParMultiDo-BoundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
May 20, 2022 2:55:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-20T14:55:00.175Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
May 20, 2022 2:55:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-20T14:55:00.203Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
May 20, 2022 2:55:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-20T14:55:00.246Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
May 20, 2022 2:55:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-20T14:55:00.272Z: Fusing consumer Read-co-input-ParDo-BoundedSourceAsSDFWrapper--ParMultiDo-BoundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
May 20, 2022 2:55:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-20T14:55:00.300Z: Fusing consumer Read-co-input-ParDo-BoundedSourceAsSDFWrapper--ParMultiDo-BoundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-BoundedSourceAsSDFWrapper--ParMultiDo-BoundedSourceAsSDFWrapper-/PairWithRestriction
May 20, 2022 2:55:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-20T14:55:00.329Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read-co-input-ParDo-BoundedSourceAsSDFWrapper--ParMultiDo-BoundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
May 20, 2022 2:55:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-20T14:55:00.355Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
May 20, 2022 2:55:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-20T14:55:00.388Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
May 20, 2022 2:55:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-20T14:55:00.772Z: Executing operation Read co-input/Impulse+Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)+Read-co-input-ParDo-BoundedSourceAsSDFWrapper--ParMultiDo-BoundedSourceAsSDFWrapper-/PairWithRestriction+Read-co-input-ParDo-BoundedSourceAsSDFWrapper--ParMultiDo-BoundedSourceAsSDFWrapper-/SplitWithSizing
May 20, 2022 2:55:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-20T14:55:00.808Z: Executing operation Read input/Impulse+Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)+Read-input-ParDo-BoundedSourceAsSDFWrapper--ParMultiDo-BoundedSourceAsSDFWrapper-/PairWithRestriction+Read-input-ParDo-BoundedSourceAsSDFWrapper--ParMultiDo-BoundedSourceAsSDFWrapper-/SplitWithSizing
May 20, 2022 2:55:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-20T14:55:00.842Z: Starting 5 ****s in us-central1-b...
May 20, 2022 2:55:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-20T14:55:13.262Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 20, 2022 2:55:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-20T14:55:22.868Z: Autoscaling: Raised the number of ****s to 5 based on the rate of progress in the currently running stage(s).
May 20, 2022 2:56:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-20T14:56:34.494Z: Workers have started successfully.
May 20, 2022 2:56:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-20T14:56:51.737Z: Finished operation Read co-input/Impulse+Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)+Read-co-input-ParDo-BoundedSourceAsSDFWrapper--ParMultiDo-BoundedSourceAsSDFWrapper-/PairWithRestriction+Read-co-input-ParDo-BoundedSourceAsSDFWrapper--ParMultiDo-BoundedSourceAsSDFWrapper-/SplitWithSizing
May 20, 2022 2:56:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-20T14:56:51.772Z: Finished operation Read input/Impulse+Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)+Read-input-ParDo-BoundedSourceAsSDFWrapper--ParMultiDo-BoundedSourceAsSDFWrapper-/PairWithRestriction+Read-input-ParDo-BoundedSourceAsSDFWrapper--ParMultiDo-BoundedSourceAsSDFWrapper-/SplitWithSizing
May 20, 2022 2:56:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-20T14:56:51.895Z: Executing operation CoGroupByKey/GBK/Create
May 20, 2022 2:56:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-20T14:56:54.454Z: Finished operation CoGroupByKey/GBK/Create
May 20, 2022 2:56:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-20T14:56:54.596Z: Executing operation Read-co-input-ParDo-BoundedSourceAsSDFWrapper--ParMultiDo-BoundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing+Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)+Window.Into()2/Window.Assign+CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)+CoGroupByKey/GBK/Write
May 20, 2022 2:56:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-20T14:56:54.621Z: Executing operation Read-input-ParDo-BoundedSourceAsSDFWrapper--ParMultiDo-BoundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing+Collect start time metrics (input)/ParMultiDo(TimeMonitor)+Window.Into()/Window.Assign+CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)+CoGroupByKey/GBK/Write
May 20, 2022 2:57:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-20T14:57:05.411Z: Finished operation Read-co-input-ParDo-BoundedSourceAsSDFWrapper--ParMultiDo-BoundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing+Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)+Window.Into()2/Window.Assign+CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)+CoGroupByKey/GBK/Write
May 20, 2022 2:57:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-20T14:57:16.417Z: Finished operation Read-input-ParDo-BoundedSourceAsSDFWrapper--ParMultiDo-BoundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing+Collect start time metrics (input)/ParMultiDo(TimeMonitor)+Window.Into()/Window.Assign+CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)+CoGroupByKey/GBK/Write
May 20, 2022 2:57:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-20T14:57:16.501Z: Executing operation CoGroupByKey/GBK/Close
May 20, 2022 2:57:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-20T14:57:16.552Z: Finished operation CoGroupByKey/GBK/Close
May 20, 2022 2:57:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-20T14:57:16.632Z: Executing operation CoGroupByKey/GBK/Read+CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)+Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)+Collect total bytes/ParMultiDo(ByteMonitor)+Collect end time metrics/ParMultiDo(TimeMonitor)
May 20, 2022 2:57:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-20T14:57:46.647Z: Finished operation CoGroupByKey/GBK/Read+CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)+Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)+Collect total bytes/ParMultiDo(ByteMonitor)+Collect end time metrics/ParMultiDo(TimeMonitor)
May 20, 2022 2:57:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-20T14:57:46.776Z: Cleaning up.
May 20, 2022 2:57:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-20T14:57:46.837Z: Stopping **** pool...
May 20, 2022 2:58:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-20T14:58:22.669Z: Autoscaling: Resized **** pool from 5 to 0.
May 20, 2022 2:58:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-20T14:58:22.707Z: Worker pool stopped.
May 20, 2022 2:58:28 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2022-05-20_07_54_50-4752207528063837205 finished with status DONE.
Load test results for test (ID): 28cbab45-0e9b-4b5b-beaf-40c5150e4656 and timestamp: 2022-05-20T14:54:42.257000000Z:
Metric: Value:
dataflow_v2_java11_runtime_sec 48.086
dataflow_v2_java11_total_bytes_count 2.2E9
> Task :runners:google-cloud-dataflow-java:cleanUpDockerJavaImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220520145239
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:700b785f90f906fc9875715012894fd6122db9e5a681bcfd47154886a7254c7a
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220520145239]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:700b785f90f906fc9875715012894fd6122db9e5a681bcfd47154886a7254c7a]
Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220520145239] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:700b785f90f906fc9875715012894fd6122db9e5a681bcfd47154886a7254c7a])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:f93a14275cef1f0369d1790e5468a4b630a3a908a0b6ca2c70df86c68532b872
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:f93a14275cef1f0369d1790e5468a4b630a3a908a0b6ca2c70df86c68532b872
ERROR: (gcloud.container.images.delete) Not found: response: {'docker-distribution-api-version': 'registry/2.0', 'content-type': 'application/json', 'date': 'Fri, 20 May 2022 14:58:35 GMT', 'server': 'Docker Registry', 'cache-control': 'private', 'x-xss-protection': '0', 'x-frame-options': 'SAMEORIGIN', 'transfer-encoding': 'chunked', 'status': '404', 'content-length': '168', '-content-encoding': 'gzip'}
Failed to compute blob liveness for manifest: 'sha256:f93a14275cef1f0369d1790e5468a4b630a3a908a0b6ca2c70df86c68532b872': None
> Task :runners:google-cloud-dataflow-java:cleanUpDockerJavaImages FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Batch_Java11/ws/src/runners/google-cloud-dataflow-java/build.gradle'> line: 297
* What went wrong:
Execution failed for task ':runners:google-cloud-dataflow-java:cleanUpDockerJavaImages'.
> Process 'command './scripts/cleanup_untagged_gcr_images.sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.
BUILD FAILED in 6m 10s
110 actionable tasks: 77 executed, 29 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/unmqflzv6dic6
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Jenkins build is back to normal : beam_LoadTests_Java_CoGBK_Dataflow_V2_Batch_Java11 #338
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Batch_Java11/338/display/redirect>
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Batch_Java11 #337
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Batch_Java11/337/display/redirect?page=changes>
Changes:
[yathu] Add labels for typescript PRs
[noreply] Bump google.golang.org/grpc from 1.45.0 to 1.46.2 in /sdks (#17677)
[noreply] [BEAM-13015] Only create a TimerBundleTracker if there are timers.
------------------------------------------
[...truncated 114.08 KB...]
1b7cceb6a07c: Preparing
b274e8788e0c: Preparing
78658088978a: Preparing
451e392a70f1: Waiting
76956504921c: Waiting
f128dcb01b33: Waiting
65aca8f763c3: Waiting
b274e8788e0c: Waiting
3b2356b88239: Waiting
1c08204d53bf: Waiting
f4dd63856a44: Waiting
9069e693a75d: Waiting
b39a4e644434: Waiting
a7934564e6b9: Waiting
3c2a589ad52e: Waiting
66dd2119aab4: Waiting
07df3576ab77: Waiting
1b7cceb6a07c: Waiting
cbd16b72e80e: Waiting
78658088978a: Waiting
9c5e935dc05a: Pushed
fa3c01fc9f8b: Pushed
3b27adf2dba7: Pushed
95e8eaf0c547: Pushed
69998f28bf33: Pushed
451e392a70f1: Pushed
76956504921c: Pushed
65aca8f763c3: Pushed
f128dcb01b33: Pushed
3c2a589ad52e: Pushed
1c08204d53bf: Pushed
cbd16b72e80e: Layer already exists
07df3576ab77: Layer already exists
3b2356b88239: Layer already exists
a7934564e6b9: Layer already exists
1b7cceb6a07c: Layer already exists
b274e8788e0c: Layer already exists
78658088978a: Layer already exists
f4dd63856a44: Pushed
66dd2119aab4: Pushed
9069e693a75d: Pushed
b39a4e644434: Pushed
20220521145753: digest: sha256:9be913687b8e7a4c32eb1bae71bb0b9bc2a6deeb5b334a1c6f3db6ffa6f61666 size: 4935
> Task :sdks:java:testing:load-tests:run
May 21, 2022 2:58:23 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
May 21, 2022 2:58:23 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 222 files. Enable logging at DEBUG level to see which files will be staged.
May 21, 2022 2:58:24 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
May 21, 2022 2:58:24 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
May 21, 2022 2:58:26 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 222 files from PipelineOptions.filesToStage to staging location to prepare for execution.
May 21, 2022 2:58:26 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 222 files cached, 0 files newly uploaded in 0 seconds
May 21, 2022 2:58:26 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
May 21, 2022 2:58:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <114842 bytes, hash 9f7d67059d16d0607612cb451caa88c8cda3bc9646540416ef994ed973fa6736> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-n31nBZ0W0GB2EstFHKqIyM2jvJZGVAQW75lO2XP6ZzY.pb
May 21, 2022 2:58:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input as step s1
May 21, 2022 2:58:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s2
May 21, 2022 2:58:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s3
May 21, 2022 2:58:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input as step s4
May 21, 2022 2:58:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s5
May 21, 2022 2:58:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s6
May 21, 2022 2:58:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s7
May 21, 2022 2:58:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s8
May 21, 2022 2:58:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s9
May 21, 2022 2:58:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s10
May 21, 2022 2:58:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s11
May 21, 2022 2:58:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s12
May 21, 2022 2:58:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s13
May 21, 2022 2:58:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s14
May 21, 2022 2:58:28 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.40.0-SNAPSHOT
May 21, 2022 2:58:29 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-05-21_07_58_28-4210429078762022098?project=apache-beam-testing
May 21, 2022 2:58:29 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2022-05-21_07_58_28-4210429078762022098
May 21, 2022 2:58:29 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2022-05-21_07_58_28-4210429078762022098
May 21, 2022 2:58:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2022-05-21T14:58:30.702Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20batch0cogbk02-jenkins-052114-klfq. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
May 21, 2022 2:58:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-21T14:58:34.845Z: Worker configuration: e2-standard-2 in us-central1-b.
May 21, 2022 2:58:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-21T14:58:35.599Z: Expanding SplittableParDo operations into optimizable parts.
May 21, 2022 2:58:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-21T14:58:35.631Z: Expanding CollectionToSingleton operations into optimizable parts.
May 21, 2022 2:58:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-21T14:58:35.695Z: Expanding CoGroupByKey operations into optimizable parts.
May 21, 2022 2:58:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-21T14:58:35.757Z: Expanding GroupByKey operations into optimizable parts.
May 21, 2022 2:58:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-21T14:58:35.825Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 21, 2022 2:58:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-21T14:58:35.882Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
May 21, 2022 2:58:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-21T14:58:35.916Z: Fusing unzipped copy of CoGroupByKey/GBK/Write, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
May 21, 2022 2:58:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-21T14:58:35.950Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/Read
May 21, 2022 2:58:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-21T14:58:35.981Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
May 21, 2022 2:58:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-21T14:58:36.009Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
May 21, 2022 2:58:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-21T14:58:36.037Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
May 21, 2022 2:58:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-21T14:58:36.067Z: Fusing consumer CoGroupByKey/GBK/Write into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
May 21, 2022 2:58:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-21T14:58:36.097Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
May 21, 2022 2:58:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-21T14:58:36.163Z: Fusing consumer Read-input-ParDo-BoundedSourceAsSDFWrapper--ParMultiDo-BoundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
May 21, 2022 2:58:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-21T14:58:36.196Z: Fusing consumer Read-input-ParDo-BoundedSourceAsSDFWrapper--ParMultiDo-BoundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-BoundedSourceAsSDFWrapper--ParMultiDo-BoundedSourceAsSDFWrapper-/PairWithRestriction
May 21, 2022 2:58:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-21T14:58:36.225Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read-input-ParDo-BoundedSourceAsSDFWrapper--ParMultiDo-BoundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
May 21, 2022 2:58:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-21T14:58:36.260Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
May 21, 2022 2:58:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-21T14:58:36.294Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
May 21, 2022 2:58:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-21T14:58:36.328Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
May 21, 2022 2:58:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-21T14:58:36.362Z: Fusing consumer Read-co-input-ParDo-BoundedSourceAsSDFWrapper--ParMultiDo-BoundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
May 21, 2022 2:58:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-21T14:58:36.408Z: Fusing consumer Read-co-input-ParDo-BoundedSourceAsSDFWrapper--ParMultiDo-BoundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-BoundedSourceAsSDFWrapper--ParMultiDo-BoundedSourceAsSDFWrapper-/PairWithRestriction
May 21, 2022 2:58:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-21T14:58:36.434Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read-co-input-ParDo-BoundedSourceAsSDFWrapper--ParMultiDo-BoundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
May 21, 2022 2:58:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-21T14:58:36.460Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
May 21, 2022 2:58:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-21T14:58:36.481Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
May 21, 2022 2:58:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-21T14:58:36.843Z: Executing operation Read co-input/Impulse+Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)+Read-co-input-ParDo-BoundedSourceAsSDFWrapper--ParMultiDo-BoundedSourceAsSDFWrapper-/PairWithRestriction+Read-co-input-ParDo-BoundedSourceAsSDFWrapper--ParMultiDo-BoundedSourceAsSDFWrapper-/SplitWithSizing
May 21, 2022 2:58:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-21T14:58:36.876Z: Executing operation Read input/Impulse+Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)+Read-input-ParDo-BoundedSourceAsSDFWrapper--ParMultiDo-BoundedSourceAsSDFWrapper-/PairWithRestriction+Read-input-ParDo-BoundedSourceAsSDFWrapper--ParMultiDo-BoundedSourceAsSDFWrapper-/SplitWithSizing
May 21, 2022 2:58:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-21T14:58:36.925Z: Starting 5 ****s in us-central1-b...
May 21, 2022 2:59:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-21T14:58:58.982Z: Autoscaling: Raised the number of ****s to 5 based on the rate of progress in the currently running stage(s).
May 21, 2022 2:59:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-21T14:59:08.887Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 21, 2022 3:00:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-21T15:00:03.485Z: Workers have started successfully.
May 21, 2022 3:00:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-21T15:00:17.314Z: Finished operation Read co-input/Impulse+Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)+Read-co-input-ParDo-BoundedSourceAsSDFWrapper--ParMultiDo-BoundedSourceAsSDFWrapper-/PairWithRestriction+Read-co-input-ParDo-BoundedSourceAsSDFWrapper--ParMultiDo-BoundedSourceAsSDFWrapper-/SplitWithSizing
May 21, 2022 3:00:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-21T15:00:17.411Z: Finished operation Read input/Impulse+Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)+Read-input-ParDo-BoundedSourceAsSDFWrapper--ParMultiDo-BoundedSourceAsSDFWrapper-/PairWithRestriction+Read-input-ParDo-BoundedSourceAsSDFWrapper--ParMultiDo-BoundedSourceAsSDFWrapper-/SplitWithSizing
May 21, 2022 3:00:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-21T15:00:17.542Z: Executing operation CoGroupByKey/GBK/Create
May 21, 2022 3:00:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-21T15:00:20.081Z: Finished operation CoGroupByKey/GBK/Create
May 21, 2022 3:00:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-21T15:00:20.222Z: Executing operation Read-co-input-ParDo-BoundedSourceAsSDFWrapper--ParMultiDo-BoundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing+Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)+Window.Into()2/Window.Assign+CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)+CoGroupByKey/GBK/Write
May 21, 2022 3:00:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-21T15:00:20.243Z: Executing operation Read-input-ParDo-BoundedSourceAsSDFWrapper--ParMultiDo-BoundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing+Collect start time metrics (input)/ParMultiDo(TimeMonitor)+Window.Into()/Window.Assign+CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)+CoGroupByKey/GBK/Write
May 21, 2022 3:00:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-21T15:00:39.838Z: Finished operation Read-co-input-ParDo-BoundedSourceAsSDFWrapper--ParMultiDo-BoundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing+Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)+Window.Into()2/Window.Assign+CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)+CoGroupByKey/GBK/Write
May 21, 2022 3:00:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-21T15:00:48.104Z: Finished operation Read-input-ParDo-BoundedSourceAsSDFWrapper--ParMultiDo-BoundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing+Collect start time metrics (input)/ParMultiDo(TimeMonitor)+Window.Into()/Window.Assign+CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)+CoGroupByKey/GBK/Write
May 21, 2022 3:00:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-21T15:00:48.178Z: Executing operation CoGroupByKey/GBK/Close
May 21, 2022 3:00:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-21T15:00:48.221Z: Finished operation CoGroupByKey/GBK/Close
May 21, 2022 3:00:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-21T15:00:48.286Z: Executing operation CoGroupByKey/GBK/Read+CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)+Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)+Collect total bytes/ParMultiDo(ByteMonitor)+Collect end time metrics/ParMultiDo(TimeMonitor)
May 21, 2022 3:01:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-21T15:01:13.846Z: Finished operation CoGroupByKey/GBK/Read+CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)+Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)+Collect total bytes/ParMultiDo(ByteMonitor)+Collect end time metrics/ParMultiDo(TimeMonitor)
May 21, 2022 3:01:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-21T15:01:13.991Z: Cleaning up.
May 21, 2022 3:01:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-21T15:01:14.071Z: Stopping **** pool...
May 21, 2022 3:01:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-21T15:01:46.929Z: Autoscaling: Resized **** pool from 5 to 0.
May 21, 2022 3:01:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-21T15:01:46.972Z: Worker pool stopped.
May 21, 2022 3:01:58 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2022-05-21_07_58_28-4210429078762022098 finished with status DONE.
Load test results for test (ID): 7b263020-d27e-449d-a60f-763b7b0aa15c and timestamp: 2022-05-21T14:58:23.788000000Z:
Metric: Value:
dataflow_v2_java11_runtime_sec 46.563
dataflow_v2_java11_total_bytes_count 2.2E9
> Task :runners:google-cloud-dataflow-java:cleanUpDockerJavaImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220521145753
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:9be913687b8e7a4c32eb1bae71bb0b9bc2a6deeb5b334a1c6f3db6ffa6f61666
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220521145753]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:9be913687b8e7a4c32eb1bae71bb0b9bc2a6deeb5b334a1c6f3db6ffa6f61666]
Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220521145753] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:9be913687b8e7a4c32eb1bae71bb0b9bc2a6deeb5b334a1c6f3db6ffa6f61666])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:f6df6c9940e670a1c3edc23625057b497e62ca993b3782c0435c6dd9dd24cd30
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:f6df6c9940e670a1c3edc23625057b497e62ca993b3782c0435c6dd9dd24cd30
ERROR: (gcloud.container.images.delete) Not found: response: {'docker-distribution-api-version': 'registry/2.0', 'content-type': 'application/json', 'date': 'Sat, 21 May 2022 15:02:04 GMT', 'server': 'Docker Registry', 'cache-control': 'private', 'x-xss-protection': '0', 'x-frame-options': 'SAMEORIGIN', 'transfer-encoding': 'chunked', 'status': '404', 'content-length': '168', '-content-encoding': 'gzip'}
Failed to compute blob liveness for manifest: 'sha256:f6df6c9940e670a1c3edc23625057b497e62ca993b3782c0435c6dd9dd24cd30': None
> Task :runners:google-cloud-dataflow-java:cleanUpDockerJavaImages FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Batch_Java11/ws/src/runners/google-cloud-dataflow-java/build.gradle'> line: 297
* What went wrong:
Execution failed for task ':runners:google-cloud-dataflow-java:cleanUpDockerJavaImages'.
> Process 'command './scripts/cleanup_untagged_gcr_images.sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.
BUILD FAILED in 4m 12s
105 actionable tasks: 8 executed, 97 up-to-date
Publishing build scan...
https://gradle.com/s/gdgq6vdeuetqs
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org