You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@beam.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2022/05/08 16:02:04 UTC
Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17 #139
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17/139/display/redirect>
Changes:
------------------------------------------
[...truncated 50.84 KB...]
08fa02ce37eb: Preparing
a037458de4e0: Preparing
bafdbe68e4ae: Preparing
a13c519c6361: Preparing
fced9607bc47: Waiting
e91b08f16562: Waiting
713082863eee: Waiting
08e8a29c256a: Waiting
00d4a583dbe4: Waiting
875a60575306: Waiting
c92dc92e5b42: Waiting
721b8ef4b4ba: Waiting
b900d4df52f8: Waiting
5b26c8721c69: Waiting
3bc383470c05: Waiting
bafdbe68e4ae: Waiting
a13c519c6361: Waiting
e93827457889: Waiting
08fa02ce37eb: Waiting
a037458de4e0: Waiting
fbe1b45afd3e: Pushed
c8df78b28fbb: Pushed
e023246e7a9a: Pushed
d580cb531531: Pushed
de4ea1bb1e74: Pushed
875a60575306: Pushed
721b8ef4b4ba: Pushed
b900d4df52f8: Pushed
5b26c8721c69: Pushed
713082863eee: Pushed
e91b08f16562: Pushed
fced9607bc47: Pushed
e93827457889: Layer already exists
08fa02ce37eb: Layer already exists
3bc383470c05: Layer already exists
a037458de4e0: Layer already exists
bafdbe68e4ae: Layer already exists
a13c519c6361: Layer already exists
00d4a583dbe4: Pushed
c92dc92e5b42: Pushed
08e8a29c256a: Pushed
20220508123043: digest: sha256:3e2bc6f9b92f30686d7581e866cc3457a9e65cb36f5ee4f62689dd7b0f578fb4 size: 4729
> Task :sdks:java:testing:load-tests:run
May 08, 2022 12:33:00 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
May 08, 2022 12:33:01 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 222 files. Enable logging at DEBUG level to see which files will be staged.
May 08, 2022 12:33:01 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
May 08, 2022 12:33:01 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
May 08, 2022 12:33:05 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 222 files from PipelineOptions.filesToStage to staging location to prepare for execution.
May 08, 2022 12:33:06 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 222 files cached, 0 files newly uploaded in 0 seconds
May 08, 2022 12:33:06 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
May 08, 2022 12:33:06 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <120277 bytes, hash 6c4231b8b37219127e2cbe678df05f694b012a4e308249a674ec8a16db5c1949> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-bEIxuLNyGRJ-LL5njfBfaUsBKk4wgkmmdOyKFttcGUk.pb
May 08, 2022 12:33:08 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
May 08, 2022 12:33:08 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@751ae8a4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@235d659c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4232b34a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2da16263, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@f5ce0bb, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@47e51549, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@101a461c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@360e9c06, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5ebffb44, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@311ff287, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7377781e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@31db34da, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@109f8c7e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5edacf20, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@16a5eb6d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7e307087, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1220ef43, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1a8b81e8, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@234cff57, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5e8507f1]
May 08, 2022 12:33:08 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
May 08, 2022 12:33:08 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
May 08, 2022 12:33:08 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
May 08, 2022 12:33:08 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
May 08, 2022 12:33:08 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3bf54172, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2c9a6717, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b3cde6f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6d091cad, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7c663eaf, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3bb5ceb, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1e692555, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ba0ae41, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@76fe6cdc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2ffb3aec, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@786ff1cb, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@46039a21, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@431e86b1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@35c4e864, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@32a2a6be, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@682af059, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5f36c8e3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4da39ca9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6a9344f5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5584d9c6]
May 08, 2022 12:33:08 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
May 08, 2022 12:33:08 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
May 08, 2022 12:33:08 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
May 08, 2022 12:33:08 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
May 08, 2022 12:33:08 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
May 08, 2022 12:33:08 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
May 08, 2022 12:33:08 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
May 08, 2022 12:33:08 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
May 08, 2022 12:33:08 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
May 08, 2022 12:33:08 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
May 08, 2022 12:33:08 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
May 08, 2022 12:33:08 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.40.0-SNAPSHOT
May 08, 2022 12:33:09 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-05-08_05_33_08-12380385818833013014?project=apache-beam-testing
May 08, 2022 12:33:09 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2022-05-08_05_33_08-12380385818833013014
May 08, 2022 12:33:09 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2022-05-08_05_33_08-12380385818833013014
May 08, 2022 12:33:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2022-05-08T12:33:14.423Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java170dataflow0v20streaming0cogbk01-jenkins-05-v0tj. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
May 08, 2022 12:33:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-08T12:33:25.065Z: Worker configuration: e2-standard-2 in us-central1-b.
May 08, 2022 12:33:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-08T12:33:25.642Z: Expanding SplittableParDo operations into optimizable parts.
May 08, 2022 12:33:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-08T12:33:25.737Z: Expanding CollectionToSingleton operations into optimizable parts.
May 08, 2022 12:33:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-08T12:33:25.818Z: Expanding CoGroupByKey operations into optimizable parts.
May 08, 2022 12:33:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-08T12:33:25.888Z: Expanding SplittableProcessKeyed operations into optimizable parts.
May 08, 2022 12:33:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-08T12:33:25.917Z: Expanding GroupByKey operations into streaming Read/Write steps
May 08, 2022 12:33:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-08T12:33:25.983Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 08, 2022 12:33:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-08T12:33:26.099Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 08, 2022 12:33:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-08T12:33:26.130Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
May 08, 2022 12:33:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-08T12:33:26.164Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
May 08, 2022 12:33:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-08T12:33:26.199Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
May 08, 2022 12:33:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-08T12:33:26.230Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
May 08, 2022 12:33:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-08T12:33:26.270Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
May 08, 2022 12:33:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-08T12:33:26.306Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
May 08, 2022 12:33:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-08T12:33:26.340Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
May 08, 2022 12:33:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-08T12:33:26.371Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
May 08, 2022 12:33:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-08T12:33:26.410Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
May 08, 2022 12:33:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-08T12:33:26.443Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
May 08, 2022 12:33:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-08T12:33:26.476Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
May 08, 2022 12:33:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-08T12:33:26.509Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
May 08, 2022 12:33:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-08T12:33:26.542Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
May 08, 2022 12:33:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-08T12:33:26.568Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
May 08, 2022 12:33:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-08T12:33:26.605Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
May 08, 2022 12:33:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-08T12:33:26.639Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
May 08, 2022 12:33:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-08T12:33:26.736Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
May 08, 2022 12:33:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-08T12:33:26.771Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
May 08, 2022 12:33:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-08T12:33:26.802Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
May 08, 2022 12:33:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-08T12:33:26.836Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
May 08, 2022 12:33:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-08T12:33:26.872Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
May 08, 2022 12:33:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-08T12:33:26.897Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
May 08, 2022 12:33:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-08T12:33:27.139Z: Running job using Streaming Engine
May 08, 2022 12:33:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-08T12:33:27.534Z: Starting 5 ****s in us-central1-b...
May 08, 2022 12:33:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-08T12:33:41.960Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 08, 2022 12:33:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-08T12:33:49.913Z: Autoscaling: Raised the number of ****s to 4 so that the pipeline can catch up with its backlog and keep up with its input rate.
May 08, 2022 12:33:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-08T12:33:49.939Z: Resized **** pool to 4, though goal was 5. This could be a quota issue.
May 08, 2022 12:34:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-08T12:34:00.147Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
May 08, 2022 12:34:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-08T12:34:57.660Z: Workers have started successfully.
May 08, 2022 4:01:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-08T16:01:12.049Z: Cancel request is committed for workflow job: 2022-05-08_05_33_08-12380385818833013014.
May 08, 2022 4:01:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-08T16:01:12.227Z: Cleaning up.
May 08, 2022 4:01:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-08T16:01:12.337Z: Stopping **** pool...
May 08, 2022 4:01:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-08T16:01:12.380Z: Stopping **** pool...
May 08, 2022 4:01:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-08T16:01:45.060Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
May 08, 2022 4:01:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-08T16:01:45.128Z: Worker pool stopped.
May 08, 2022 4:01:52 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2022-05-08_05_33_08-12380385818833013014 finished with status CANCELLED.
Load test results for test (ID): 6898ae29-0c8e-440a-aed1-3e8098cdf3b0 and timestamp: 2022-05-08T12:33:01.540000000Z:
Metric: Value:
dataflow_v2_java17_runtime_sec 12302.848
dataflow_v2_java17_total_bytes_count 2.44242771E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)
> Task :sdks:java:testing:load-tests:run FAILED
> Task :runners:google-cloud-dataflow-java:cleanUpDockerJavaImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220508123043
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:3e2bc6f9b92f30686d7581e866cc3457a9e65cb36f5ee4f62689dd7b0f578fb4
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220508123043]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:3e2bc6f9b92f30686d7581e866cc3457a9e65cb36f5ee4f62689dd7b0f578fb4]
Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220508123043] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:3e2bc6f9b92f30686d7581e866cc3457a9e65cb36f5ee4f62689dd7b0f578fb4])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:3e2bc6f9b92f30686d7581e866cc3457a9e65cb36f5ee4f62689dd7b0f578fb4
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:3e2bc6f9b92f30686d7581e866cc3457a9e65cb36f5ee4f62689dd7b0f578fb4
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:3e2bc6f9b92f30686d7581e866cc3457a9e65cb36f5ee4f62689dd7b0f578fb4].
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.
BUILD FAILED in 3h 31m 31s
110 actionable tasks: 73 executed, 33 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/a3nj7pq4szqmc
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Jenkins build is back to normal : beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17 #142
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17/142/display/redirect?page=changes>
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17 #141
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17/141/display/redirect?page=changes>
Changes:
[andyye333] Add extra details to PubSub matcher errors
[noreply] Merge pull request #17559 from [BEAM-14423] Add exception injection
[noreply] [BEAM-11104] Allow self-checkpointing SDFs to return without finishing
[noreply] Merge pull request #17544 from [BEAM-14415] Exception handling tests for
[noreply] Merge pull request #17565 from [BEAM-14413] add Kafka exception test
[noreply] Merge pull request #17555 from [BEAM-14417] Adding exception handling
[noreply] [BEAM-14433] Improve Go split error message. (#17575)
[noreply] [BEAM-14429] Force java load test on dataflow runner v2
[noreply] Merge pull request #17577 from [BEAM-14435] Adding exception handling
[noreply] [BEAM-14347] Add generic registration functions for iters and emitters
[noreply] [BEAM-14169] Add Credentials rotation cron job for clusters (#17383)
[noreply] [BEAM-14347] Add generic registration for accumulators (#17579)
------------------------------------------
[...truncated 50.68 KB...]
8d3c4b8a056c: Preparing
779e60d40a60: Preparing
6e94b4ebcad2: Preparing
3a7cd99b8479: Preparing
c49c9f3cb678: Preparing
5a27e519e570: Waiting
3fe3663cf18e: Preparing
3634662e7534: Preparing
779e60d40a60: Waiting
6e94b4ebcad2: Waiting
267e49f181ed: Preparing
3bc383470c05: Preparing
e93827457889: Preparing
08fa02ce37eb: Preparing
a037458de4e0: Preparing
bafdbe68e4ae: Preparing
a13c519c6361: Preparing
08fa02ce37eb: Waiting
a13c519c6361: Waiting
a037458de4e0: Waiting
8d3c4b8a056c: Waiting
e93827457889: Waiting
3a7cd99b8479: Waiting
267e49f181ed: Waiting
c49c9f3cb678: Waiting
3bc383470c05: Waiting
83bac70d0466: Waiting
aa1c4990f94b: Pushed
5d41c137d1a8: Pushed
7d9293365344: Pushed
38be9522fc38: Pushed
a51548066d49: Pushed
8d3c4b8a056c: Pushed
5a27e519e570: Pushed
779e60d40a60: Pushed
6e94b4ebcad2: Pushed
c49c9f3cb678: Pushed
3a7cd99b8479: Pushed
3bc383470c05: Layer already exists
83bac70d0466: Pushed
e93827457889: Layer already exists
08fa02ce37eb: Layer already exists
a037458de4e0: Layer already exists
bafdbe68e4ae: Layer already exists
a13c519c6361: Layer already exists
3634662e7534: Pushed
267e49f181ed: Pushed
3fe3663cf18e: Pushed
20220510123038: digest: sha256:681c0d5f2a230de72485a61d99b98d34569d0303dda01320734c2ebe64c4a3da size: 4729
> Task :sdks:java:testing:load-tests:run
May 10, 2022 12:32:49 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
May 10, 2022 12:32:50 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 222 files. Enable logging at DEBUG level to see which files will be staged.
May 10, 2022 12:32:51 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
May 10, 2022 12:32:51 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
May 10, 2022 12:32:53 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 222 files from PipelineOptions.filesToStage to staging location to prepare for execution.
May 10, 2022 12:32:54 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 222 files cached, 0 files newly uploaded in 0 seconds
May 10, 2022 12:32:54 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
May 10, 2022 12:32:54 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <120279 bytes, hash 90d5250ac2615b6f15499695641c4bbc2e25c31d4352ec686073b7925252605d> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-kNUlCsJhW28VSZaVZBxLvC4lwx1DUuxoYHO3klJSYF0.pb
May 10, 2022 12:32:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
May 10, 2022 12:32:56 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 1 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4232b34a]
May 10, 2022 12:32:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
May 10, 2022 12:32:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
May 10, 2022 12:32:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
May 10, 2022 12:32:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
May 10, 2022 12:32:56 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 1 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@ffd4cba]
May 10, 2022 12:32:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
May 10, 2022 12:32:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
May 10, 2022 12:32:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
May 10, 2022 12:32:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
May 10, 2022 12:32:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
May 10, 2022 12:32:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
May 10, 2022 12:32:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
May 10, 2022 12:32:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
May 10, 2022 12:32:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
May 10, 2022 12:32:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
May 10, 2022 12:32:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
May 10, 2022 12:32:56 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.40.0-SNAPSHOT
May 10, 2022 12:32:57 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-05-10_05_32_56-7371717862294852586?project=apache-beam-testing
May 10, 2022 12:32:57 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2022-05-10_05_32_56-7371717862294852586
May 10, 2022 12:32:57 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2022-05-10_05_32_56-7371717862294852586
May 10, 2022 12:33:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2022-05-10T12:33:02.262Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java170dataflow0v20streaming0cogbk01-jenkins-05-ahvo. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
May 10, 2022 12:33:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-10T12:33:12.593Z: Worker configuration: e2-standard-2 in us-central1-b.
May 10, 2022 12:33:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-10T12:33:13.927Z: Expanding SplittableParDo operations into optimizable parts.
May 10, 2022 12:33:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-10T12:33:13.960Z: Expanding CollectionToSingleton operations into optimizable parts.
May 10, 2022 12:33:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-10T12:33:14.024Z: Expanding CoGroupByKey operations into optimizable parts.
May 10, 2022 12:33:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-10T12:33:14.093Z: Expanding SplittableProcessKeyed operations into optimizable parts.
May 10, 2022 12:33:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-10T12:33:14.125Z: Expanding GroupByKey operations into streaming Read/Write steps
May 10, 2022 12:33:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-10T12:33:14.186Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 10, 2022 12:33:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-10T12:33:14.308Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 10, 2022 12:33:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-10T12:33:14.336Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
May 10, 2022 12:33:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-10T12:33:14.371Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
May 10, 2022 12:33:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-10T12:33:14.405Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
May 10, 2022 12:33:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-10T12:33:14.439Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
May 10, 2022 12:33:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-10T12:33:14.476Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
May 10, 2022 12:33:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-10T12:33:14.506Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
May 10, 2022 12:33:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-10T12:33:14.543Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
May 10, 2022 12:33:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-10T12:33:14.575Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
May 10, 2022 12:33:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-10T12:33:14.606Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
May 10, 2022 12:33:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-10T12:33:14.659Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
May 10, 2022 12:33:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-10T12:33:14.694Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
May 10, 2022 12:33:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-10T12:33:14.738Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
May 10, 2022 12:33:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-10T12:33:14.772Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
May 10, 2022 12:33:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-10T12:33:14.797Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
May 10, 2022 12:33:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-10T12:33:14.827Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
May 10, 2022 12:33:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-10T12:33:14.864Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
May 10, 2022 12:33:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-10T12:33:14.889Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
May 10, 2022 12:33:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-10T12:33:14.918Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
May 10, 2022 12:33:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-10T12:33:14.952Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
May 10, 2022 12:33:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-10T12:33:14.987Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
May 10, 2022 12:33:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-10T12:33:15.020Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
May 10, 2022 12:33:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-10T12:33:15.047Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
May 10, 2022 12:33:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-10T12:33:15.240Z: Running job using Streaming Engine
May 10, 2022 12:33:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-10T12:33:15.523Z: Starting 5 ****s in us-central1-b...
May 10, 2022 12:33:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-10T12:33:21.076Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 10, 2022 12:33:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-10T12:33:38.137Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
May 10, 2022 12:34:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-10T12:34:46.474Z: Workers have started successfully.
May 10, 2022 2:13:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-10T14:13:36.440Z: Cleaning up.
May 10, 2022 2:13:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-10T14:13:36.646Z: Stopping **** pool...
May 10, 2022 2:13:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-10T14:13:36.700Z: Stopping **** pool...
May 10, 2022 2:14:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-10T14:14:11.917Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
May 10, 2022 2:14:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-10T14:14:11.960Z: Worker pool stopped.
May 10, 2022 2:14:18 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2022-05-10_05_32_56-7371717862294852586 finished with status DONE.
Load test results for test (ID): 8cdbdbfb-6fcb-49fa-a861-a1bb58fd0ad5 and timestamp: 2022-05-10T12:32:50.673000000Z:
Metric: Value:
dataflow_v2_java17_runtime_sec 5813.232
dataflow_v2_java17_total_bytes_count 6.0274294E9
> Task :runners:google-cloud-dataflow-java:cleanUpDockerJavaImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220510123038
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:681c0d5f2a230de72485a61d99b98d34569d0303dda01320734c2ebe64c4a3da
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220510123038]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:681c0d5f2a230de72485a61d99b98d34569d0303dda01320734c2ebe64c4a3da]
Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220510123038] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:681c0d5f2a230de72485a61d99b98d34569d0303dda01320734c2ebe64c4a3da])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:e016287d22625030f983cbf6f23e489ff06d1e82e3b1276c012f318dcaa366bf
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:e016287d22625030f983cbf6f23e489ff06d1e82e3b1276c012f318dcaa366bf
ERROR: (gcloud.container.images.delete) Not found: response: {'docker-distribution-api-version': 'registry/2.0', 'content-type': 'application/json', 'date': 'Tue, 10 May 2022 14:14:25 GMT', 'server': 'Docker Registry', 'cache-control': 'private', 'x-xss-protection': '0', 'x-frame-options': 'SAMEORIGIN', 'transfer-encoding': 'chunked', 'status': '404', 'content-length': '168', '-content-encoding': 'gzip'}
Failed to compute blob liveness for manifest: 'sha256:e016287d22625030f983cbf6f23e489ff06d1e82e3b1276c012f318dcaa366bf': None
> Task :runners:google-cloud-dataflow-java:cleanUpDockerJavaImages FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17/ws/src/runners/google-cloud-dataflow-java/build.gradle'> line: 297
* What went wrong:
Execution failed for task ':runners:google-cloud-dataflow-java:cleanUpDockerJavaImages'.
> Process 'command './scripts/cleanup_untagged_gcr_images.sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.
BUILD FAILED in 1h 44m
110 actionable tasks: 73 executed, 33 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/iters5gdlxc5s
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17 #140
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17/140/display/redirect?page=changes>
Changes:
[elias.segundo] Changing elegibility to AllNodeElegibility
[chamikaramj] Adds code reviewers for GCP I/O connectors and KafkaIO to Beam OWNERS
------------------------------------------
[...truncated 50.04 KB...]
e93827457889: Preparing
08fa02ce37eb: Preparing
a037458de4e0: Preparing
bafdbe68e4ae: Preparing
a13c519c6361: Preparing
4558b07d7c6c: Waiting
7572dcf244a7: Waiting
a037458de4e0: Waiting
bafdbe68e4ae: Waiting
a13c519c6361: Waiting
76cbfa6ab6de: Waiting
3bc383470c05: Waiting
212e3bed8e24: Waiting
a1cdc24dc423: Waiting
08fa02ce37eb: Waiting
e93827457889: Waiting
e71737e48960: Waiting
9eded4dea7e1: Waiting
d25a54d106dd: Waiting
65c0e27593f2: Waiting
b0a0cfc0c1c7: Pushed
99b1b0c017c9: Pushed
70d12e40179b: Pushed
74d7ad42a174: Pushed
a7b750c44030: Pushed
76cbfa6ab6de: Pushed
212e3bed8e24: Pushed
65c0e27593f2: Pushed
e71737e48960: Pushed
9eded4dea7e1: Pushed
a1cdc24dc423: Pushed
3bc383470c05: Layer already exists
e93827457889: Layer already exists
08fa02ce37eb: Layer already exists
4fdbf0a79d70: Pushed
a037458de4e0: Layer already exists
bafdbe68e4ae: Layer already exists
a13c519c6361: Layer already exists
4558b07d7c6c: Pushed
7572dcf244a7: Pushed
d25a54d106dd: Pushed
20220509123035: digest: sha256:ae661c08a40bbf8a47368c1570d2cbdb9dbb0865ef587bd35d60f43cbee2a87d size: 4729
> Task :sdks:java:testing:load-tests:run
May 09, 2022 12:32:39 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
May 09, 2022 12:32:39 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 222 files. Enable logging at DEBUG level to see which files will be staged.
May 09, 2022 12:32:40 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
May 09, 2022 12:32:40 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
May 09, 2022 12:32:43 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 222 files from PipelineOptions.filesToStage to staging location to prepare for execution.
May 09, 2022 12:32:44 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 222 files cached, 0 files newly uploaded in 0 seconds
May 09, 2022 12:32:44 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
May 09, 2022 12:32:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <120277 bytes, hash 703fa7d7284d03a6724f18a00f996b599f49c2d0600f6d8db28805bcee16d47a> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-cD-n1yhNA6ZyTxigD5lrWZ9JwtBgD22NsogFvO4W1Ho.pb
May 09, 2022 12:32:45 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
May 09, 2022 12:32:45 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@751ae8a4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@235d659c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4232b34a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2da16263, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@f5ce0bb, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@47e51549, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@101a461c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@360e9c06, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5ebffb44, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@311ff287, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7377781e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@31db34da, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@109f8c7e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5edacf20, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@16a5eb6d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7e307087, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1220ef43, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1a8b81e8, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@234cff57, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5e8507f1]
May 09, 2022 12:32:45 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
May 09, 2022 12:32:45 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
May 09, 2022 12:32:45 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
May 09, 2022 12:32:46 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
May 09, 2022 12:32:46 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3bf54172, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2c9a6717, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b3cde6f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6d091cad, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7c663eaf, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3bb5ceb, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1e692555, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ba0ae41, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@76fe6cdc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2ffb3aec, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@786ff1cb, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@46039a21, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@431e86b1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@35c4e864, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@32a2a6be, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@682af059, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5f36c8e3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4da39ca9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6a9344f5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5584d9c6]
May 09, 2022 12:32:46 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
May 09, 2022 12:32:46 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
May 09, 2022 12:32:46 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
May 09, 2022 12:32:46 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
May 09, 2022 12:32:46 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
May 09, 2022 12:32:46 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
May 09, 2022 12:32:46 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
May 09, 2022 12:32:46 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
May 09, 2022 12:32:46 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
May 09, 2022 12:32:46 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
May 09, 2022 12:32:46 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
May 09, 2022 12:32:46 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.40.0-SNAPSHOT
May 09, 2022 12:32:46 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-05-09_05_32_46-14665142041617253802?project=apache-beam-testing
May 09, 2022 12:32:46 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2022-05-09_05_32_46-14665142041617253802
May 09, 2022 12:32:46 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2022-05-09_05_32_46-14665142041617253802
May 09, 2022 12:32:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2022-05-09T12:32:51.088Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java170dataflow0v20streaming0cogbk01-jenkins-05-ithr. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
May 09, 2022 12:32:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-09T12:32:56.474Z: Worker configuration: e2-standard-2 in us-central1-b.
May 09, 2022 12:33:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-09T12:32:57.352Z: Expanding SplittableParDo operations into optimizable parts.
May 09, 2022 12:33:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-09T12:32:57.408Z: Expanding CollectionToSingleton operations into optimizable parts.
May 09, 2022 12:33:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-09T12:32:57.481Z: Expanding CoGroupByKey operations into optimizable parts.
May 09, 2022 12:33:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-09T12:32:57.573Z: Expanding SplittableProcessKeyed operations into optimizable parts.
May 09, 2022 12:33:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-09T12:32:57.613Z: Expanding GroupByKey operations into streaming Read/Write steps
May 09, 2022 12:33:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-09T12:32:57.689Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 09, 2022 12:33:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-09T12:32:57.803Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 09, 2022 12:33:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-09T12:32:57.838Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
May 09, 2022 12:33:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-09T12:32:57.881Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
May 09, 2022 12:33:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-09T12:32:57.914Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
May 09, 2022 12:33:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-09T12:32:57.954Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
May 09, 2022 12:33:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-09T12:32:57.985Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
May 09, 2022 12:33:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-09T12:32:58.023Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
May 09, 2022 12:33:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-09T12:32:58.058Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
May 09, 2022 12:33:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-09T12:32:58.093Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
May 09, 2022 12:33:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-09T12:32:58.124Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
May 09, 2022 12:33:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-09T12:32:58.158Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
May 09, 2022 12:33:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-09T12:32:58.194Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
May 09, 2022 12:33:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-09T12:32:58.227Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
May 09, 2022 12:33:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-09T12:32:58.251Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
May 09, 2022 12:33:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-09T12:32:58.284Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
May 09, 2022 12:33:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-09T12:32:58.319Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
May 09, 2022 12:33:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-09T12:32:58.353Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
May 09, 2022 12:33:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-09T12:32:58.400Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
May 09, 2022 12:33:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-09T12:32:58.429Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
May 09, 2022 12:33:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-09T12:32:58.456Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
May 09, 2022 12:33:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-09T12:32:58.492Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
May 09, 2022 12:33:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-09T12:32:58.525Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
May 09, 2022 12:33:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-09T12:32:58.575Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
May 09, 2022 12:33:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-09T12:32:58.758Z: Running job using Streaming Engine
May 09, 2022 12:33:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-09T12:32:58.982Z: Starting 5 ****s in us-central1-b...
May 09, 2022 12:33:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-09T12:33:21.818Z: Autoscaling: Raised the number of ****s to 4 so that the pipeline can catch up with its backlog and keep up with its input rate.
May 09, 2022 12:33:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-09T12:33:21.846Z: Resized **** pool to 4, though goal was 5. This could be a quota issue.
May 09, 2022 12:33:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-09T12:33:29.180Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 09, 2022 12:33:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-09T12:33:32.071Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
May 09, 2022 12:34:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-09T12:34:33.040Z: Workers have started successfully.
May 09, 2022 4:01:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-09T16:01:00.503Z: Cancel request is committed for workflow job: 2022-05-09_05_32_46-14665142041617253802.
May 09, 2022 4:01:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-09T16:01:00.620Z: Cleaning up.
May 09, 2022 4:01:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-09T16:01:00.718Z: Stopping **** pool...
May 09, 2022 4:01:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-09T16:01:00.768Z: Stopping **** pool...
May 09, 2022 4:01:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-09T16:01:32.713Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
May 09, 2022 4:01:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-09T16:01:32.750Z: Worker pool stopped.
May 09, 2022 4:01:41 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2022-05-09_05_32_46-14665142041617253802 finished with status CANCELLED.
Load test results for test (ID): a6fbb7f6-1e3f-46ae-9feb-1181e26d3207 and timestamp: 2022-05-09T12:32:40.038000000Z:
Metric: Value:
dataflow_v2_java17_runtime_sec 12338.085
dataflow_v2_java17_total_bytes_count 2.86964364E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)
> Task :sdks:java:testing:load-tests:run FAILED
> Task :runners:google-cloud-dataflow-java:cleanUpDockerJavaImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220509123035
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ae661c08a40bbf8a47368c1570d2cbdb9dbb0865ef587bd35d60f43cbee2a87d
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220509123035]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ae661c08a40bbf8a47368c1570d2cbdb9dbb0865ef587bd35d60f43cbee2a87d]
Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220509123035] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ae661c08a40bbf8a47368c1570d2cbdb9dbb0865ef587bd35d60f43cbee2a87d])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ae661c08a40bbf8a47368c1570d2cbdb9dbb0865ef587bd35d60f43cbee2a87d
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ae661c08a40bbf8a47368c1570d2cbdb9dbb0865ef587bd35d60f43cbee2a87d
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ae661c08a40bbf8a47368c1570d2cbdb9dbb0865ef587bd35d60f43cbee2a87d].
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.
BUILD FAILED in 3h 31m 27s
110 actionable tasks: 73 executed, 33 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/5fdkmmdldigby
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org