You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@beam.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2021/06/22 16:01:41 UTC

Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #5

See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/5/display/redirect?page=changes>

Changes:

[Robert Bradshaw] Support AsMultimap for legacy runners.

[samuelw] [BEAM-12144] Change windmill grpc streams to reevaluate isReady and

[noreply] [BEAM-12514] fixed table_ref parsing when table_reference is a

[noreply] Added changelog for BEAM-12514

[Robert Bradshaw] Better default windowing for zero-input PTransforms.

[noreply] [BEAM-12260] Java - Backport Firestore connector's ramp-up throttling to

[noreply] [BEAM-9547] Add get implementation (#15030)

[Robert Bradshaw] Coder cleanups.


------------------------------------------
[...truncated 63.91 KB...]
The push refers to repository [us.gcr.io/apache-beam-testing/java-postcommit-it/java]
243b98e6c134: Preparing
b0dfc22c00db: Preparing
36ea5e70c85f: Preparing
f7a295bdd20b: Preparing
64f2f83e4536: Preparing
f95d84a918d9: Preparing
cce9eae33460: Preparing
967eb0f7f90d: Preparing
21c01e232bf7: Preparing
31d7dcb5e628: Preparing
68ce8173f970: Preparing
fb141cc0b2db: Preparing
15786a1cf1cb: Preparing
6f770cdc9ebf: Preparing
3fc095fab4a2: Preparing
685934357c89: Preparing
ccb9b68523fd: Preparing
00bcea93703b: Preparing
688e187d6c79: Preparing
685934357c89: Waiting
15786a1cf1cb: Waiting
f95d84a918d9: Waiting
21c01e232bf7: Waiting
68ce8173f970: Waiting
00bcea93703b: Waiting
fb141cc0b2db: Waiting
6f770cdc9ebf: Waiting
31d7dcb5e628: Waiting
3fc095fab4a2: Waiting
cce9eae33460: Waiting
967eb0f7f90d: Waiting
688e187d6c79: Waiting
36ea5e70c85f: Pushed
64f2f83e4536: Pushed
b0dfc22c00db: Pushed
243b98e6c134: Pushed
f95d84a918d9: Pushed
967eb0f7f90d: Pushed
f7a295bdd20b: Pushed
21c01e232bf7: Pushed
15786a1cf1cb: Layer already exists
6f770cdc9ebf: Layer already exists
3fc095fab4a2: Layer already exists
685934357c89: Layer already exists
ccb9b68523fd: Layer already exists
00bcea93703b: Layer already exists
68ce8173f970: Pushed
688e187d6c79: Layer already exists
fb141cc0b2db: Pushed
cce9eae33460: Pushed
31d7dcb5e628: Pushed
20210622124334: digest: sha256:17f453dad5d60b4235e517e1ee57c6b983bc23179cc958653478a32995c0889c size: 4310

> Task :sdks:java:testing:load-tests:run
Jun 22, 2021 12:48:23 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Jun 22, 2021 12:48:23 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 190 files. Enable logging at DEBUG level to see which files will be staged.
Jun 22, 2021 12:48:24 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Jun 22, 2021 12:48:24 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Jun 22, 2021 12:48:26 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Jun 22, 2021 12:48:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <112268 bytes, hash 45417cdd4c83af06952ac92de35eb4ec35dc1feddfa548b5823250db32ace6ec> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-RUF83UyDrwaVKskt41607DXcH-3fpUi1gjJQ2zKs5uw.pb
Jun 22, 2021 12:48:27 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 190 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Jun 22, 2021 12:48:28 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 190 files cached, 0 files newly uploaded in 0 seconds
Jun 22, 2021 12:48:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Jun 22, 2021 12:48:28 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@117d32e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@51d387d3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2370ac7a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@534180a2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6cf0a747, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@21505815, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7e53339, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58fef7f7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@31ddb930, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12eedfee, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c3c4a71, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1352434e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4f9a6c2d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2b6fcb9f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@75de6341, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@74170687, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@68f0f72c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3d96fa9e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3b545206, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77bb48d5]
Jun 22, 2021 12:48:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Jun 22, 2021 12:48:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Jun 22, 2021 12:48:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Jun 22, 2021 12:48:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Jun 22, 2021 12:48:28 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@64aad809, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1f03fba0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@17dd671f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3bbf6abe, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@9679750, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@9b9a327, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@75a0c890, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@358ab600, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@e26af6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@44065156, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3791f50e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@574b7f4a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@44ed0a8f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@32177fa5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@a96d56c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6ab4a5b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2abe9173, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@235d29d6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1fdca564, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43f9dd56]
Jun 22, 2021 12:48:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Jun 22, 2021 12:48:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Jun 22, 2021 12:48:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Jun 22, 2021 12:48:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Jun 22, 2021 12:48:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Jun 22, 2021 12:48:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Jun 22, 2021 12:48:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Jun 22, 2021 12:48:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Jun 22, 2021 12:48:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Jun 22, 2021 12:48:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Jun 22, 2021 12:48:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Jun 22, 2021 12:48:28 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.32.0-SNAPSHOT
Jun 22, 2021 12:48:31 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-06-22_05_48_28-15251107941824902741?project=apache-beam-testing
Jun 22, 2021 12:48:31 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-06-22_05_48_28-15251107941824902741
Jun 22, 2021 12:48:31 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-06-22_05_48_28-15251107941824902741
Jun 22, 2021 12:48:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-06-22T12:48:35.454Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-06-ws9e. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Jun 22, 2021 12:48:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-22T12:48:39.551Z: Worker configuration: n1-standard-2 in us-central1-f.
Jun 22, 2021 12:48:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-22T12:48:40.186Z: Expanding SplittableParDo operations into optimizable parts.
Jun 22, 2021 12:48:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-22T12:48:40.222Z: Expanding CollectionToSingleton operations into optimizable parts.
Jun 22, 2021 12:48:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-22T12:48:40.282Z: Expanding CoGroupByKey operations into optimizable parts.
Jun 22, 2021 12:48:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-22T12:48:40.370Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Jun 22, 2021 12:48:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-22T12:48:40.413Z: Expanding GroupByKey operations into streaming Read/Write steps
Jun 22, 2021 12:48:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-22T12:48:40.478Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jun 22, 2021 12:48:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-22T12:48:40.586Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jun 22, 2021 12:48:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-22T12:48:40.621Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Jun 22, 2021 12:48:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-22T12:48:40.646Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Jun 22, 2021 12:48:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-22T12:48:40.678Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Jun 22, 2021 12:48:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-22T12:48:40.711Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Jun 22, 2021 12:48:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-22T12:48:40.745Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jun 22, 2021 12:48:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-22T12:48:40.778Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jun 22, 2021 12:48:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-22T12:48:40.810Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jun 22, 2021 12:48:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-22T12:48:40.843Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Jun 22, 2021 12:48:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-22T12:48:40.877Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Jun 22, 2021 12:48:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-22T12:48:40.909Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Jun 22, 2021 12:48:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-22T12:48:41.044Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Jun 22, 2021 12:48:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-22T12:48:41.078Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jun 22, 2021 12:48:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-22T12:48:41.112Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jun 22, 2021 12:48:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-22T12:48:41.144Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jun 22, 2021 12:48:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-22T12:48:41.177Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Jun 22, 2021 12:48:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-22T12:48:41.211Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Jun 22, 2021 12:48:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-22T12:48:41.242Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Jun 22, 2021 12:48:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-22T12:48:41.273Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Jun 22, 2021 12:48:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-22T12:48:41.317Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Jun 22, 2021 12:48:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-22T12:48:41.340Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Jun 22, 2021 12:48:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-22T12:48:41.361Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Jun 22, 2021 12:48:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-22T12:48:41.404Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Jun 22, 2021 12:48:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-22T12:48:41.771Z: Starting 5 ****s in us-central1-f...
Jun 22, 2021 12:48:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-22T12:48:59.352Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jun 22, 2021 12:49:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-22T12:49:28.085Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Jun 22, 2021 12:50:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-22T12:50:05.777Z: Workers have started successfully.
Jun 22, 2021 12:50:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-22T12:50:05.809Z: Workers have started successfully.
Jun 22, 2021 4:00:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-22T16:00:30.449Z: Cancel request is committed for workflow job: 2021-06-22_05_48_28-15251107941824902741.
Jun 22, 2021 4:00:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-22T16:00:30.616Z: Cleaning up.
Jun 22, 2021 4:00:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-22T16:00:30.694Z: Stopping **** pool...
Jun 22, 2021 4:00:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-22T16:00:30.748Z: Stopping **** pool...
Jun 22, 2021 4:01:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-22T16:01:28.005Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Jun 22, 2021 4:01:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-22T16:01:28.041Z: Worker pool stopped.
Jun 22, 2021 4:01:36 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-06-22_05_48_28-15251107941824902741 finished with status CANCELLED.
Load test results for test (ID): 0c794e5f-4558-43af-9597-328987c2e244 and timestamp: 2021-06-22T12:48:23.994000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 10113.679
dataflow_v2_java11_total_bytes_count                6.044865E8
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210622124334
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:17f453dad5d60b4235e517e1ee57c6b983bc23179cc958653478a32995c0889c
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:17f453dad5d60b4235e517e1ee57c6b983bc23179cc958653478a32995c0889c
  Associated tags:
 - 20210622124334
Tags:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210622124334
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210622124334].
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:17f453dad5d60b4235e517e1ee57c6b983bc23179cc958653478a32995c0889c].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 18m 19s
104 actionable tasks: 74 executed, 28 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/u2zqretxewdra

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 - Build # 260 - Aborted!

Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 - Build # 260 - Aborted:

Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/260/ to view the results.

Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #259

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/259/display/redirect?page=changes>

Changes:

[rahuliyer573] py: Import beam plugins before starting SdkHarness

[stephen.patel] BEAM-14011 fix s3 filesystem multipart copy

[Valentyn Tymofieiev] Bump numpy bound to include 1.22 and regenerate container deps.

[github-actions] [BEAM-13925] months in date constructor are 0 indexed

[noreply] Merge pull request #16842 from [BEAM-13932][Playground] Container's user

[noreply] Doc updates and blog post for 2.37.0 (#16887)

[noreply] Remove resolved issue in docs + update class path on sample (#17018)

[noreply] [BEAM-14016] Fixed flaky postcommit test (#17009)

[noreply] Remove resolved issue in notebook

[noreply] [BEAM-13947] Add split() and rsplit(), non-deferred column operations on

[noreply] BEAM-14026 - Fixes bug related to Unnesting nested rows in an array


------------------------------------------
[...truncated 772.92 KB...]
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmi
Mar 05, 2022 4:03:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-05T16:03:29.161Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Mar 05, 2022 4:03:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-05T16:03:29.213Z: Worker pool stopped.
Mar 05, 2022 4:03:36 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2022-03-05_04_45_57-13308798815759132861 finished with status CANCELLED.
Load test results for test (ID): 84d8e531-5259-4aed-bd4b-66545d5e14c7 and timestamp: 2022-03-05T12:45:52.717000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11537.986
dataflow_v2_java11_total_bytes_count             3.18738023E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerJavaImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220305124337
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:7044ff27c0dcf836b665d127e6a0044c16176ff18c660f35168eb52d6a447100
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220305124337]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:7044ff27c0dcf836b665d127e6a0044c16176ff18c660f35168eb52d6a447100]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220305124337] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:7044ff27c0dcf836b665d127e6a0044c16176ff18c660f35168eb52d6a447100])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:f4312f1222026043031e5c022ffa83a27d3c88c75dd543fa7bbe0f48ef5aad08
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:f4312f1222026043031e5c022ffa83a27d3c88c75dd543fa7bbe0f48ef5aad08
ERROR: (gcloud.container.images.delete) Not found: response: {'docker-distribution-api-version': 'registry/2.0', 'content-type': 'application/json', 'date': 'Sat, 05 Mar 2022 16:03:42 GMT', 'server': 'Docker Registry', 'cache-control': 'private', 'x-xss-protection': '0', 'x-frame-options': 'SAMEORIGIN', 'transfer-encoding': 'chunked', 'status': '404', 'content-length': '168', '-content-encoding': 'gzip'}
Failed to compute blob liveness for manifest: 'sha256:f4312f1222026043031e5c022ffa83a27d3c88c75dd543fa7bbe0f48ef5aad08': None

> Task :runners:google-cloud-dataflow-java:cleanUpDockerJavaImages FAILED

FAILURE: Build completed with 2 failures.

1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
==============================================================================

2: Task failed with an exception.
-----------
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src/runners/google-cloud-dataflow-java/build.gradle'> line: 297

* What went wrong:
Execution failed for task ':runners:google-cloud-dataflow-java:cleanUpDockerJavaImages'.
> Process 'command './scripts/cleanup_untagged_gcr_images.sh'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
==============================================================================

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings

Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.

BUILD FAILED in 3h 20m 19s
109 actionable tasks: 72 executed, 33 from cache, 4 up-to-date

Publishing build scan...
https://gradle.com/s/e6sdgswnbxmzm

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #258

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/258/display/redirect?page=changes>

Changes:

[stranniknm] [BEAM-13999] playground - support vertical orientation for graph

[noreply] [adhoc] Prepare aws2 ClientConfiguration for json serialization and

[noreply] Merge pull request #16879 from [BEAM-12164] Add javadocs to

[noreply] [Cleanup] Update pre-v2 go package references (#17002)

[noreply] [BEAM-13885] Add unit tests to window package (#16971)

[noreply] Merge pull request #16891 from [BEAM-13872] [Playground] Increase test

[noreply] Merge pull request #16912 from [BEAM-13878] [Playground] Increase test

[noreply] Merge pull request #16946 from [BEAM-13873] [Playground] Increase test

[noreply] [BEAM-13951] Update mass_comment.py list of Run commands (#16889)

[noreply] [BEAM-10652] Allow Clustering without Partition in BigQuery (#16578)

[noreply] [BEAM-13857] Add K:V flags for expansion service jars and addresses to


------------------------------------------
[...truncated 1.11 MB...]
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/windmill_cache_access.cc:40
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/windmill_cache_access.cc:40
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/windmill_cache_access.cc:40
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/windmill_cache_access.cc:40
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/windmill_cache_access.cc:40
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/windmill_cache_access.cc:40
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/windmill_cache_access.cc:40
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/windmill_cache_access.cc:40
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/windmill_cache_access.cc:40
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/windmill_cache_access.cc:40
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/windmill_cache_access.cc:40
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/windmill_cache_access.cc:40
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/windmill_cache_access.cc:40
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/windmill_cache_access.cc:40
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/windmill_cache_access.cc:40
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/windmill_cache_access.cc:40
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item re
Mar 04, 2022 4:03:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-04T16:03:47.907Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Mar 04, 2022 4:03:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-04T16:03:47.945Z: Worker pool stopped.
Mar 04, 2022 4:03:56 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2022-03-04_04_45_49-15483542717043833007 finished with status CANCELLED.
Load test results for test (ID): 7427f78b-52af-48e6-98cf-6ba3094ac1b6 and timestamp: 2022-03-04T12:45:43.646000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11549.654
dataflow_v2_java11_total_bytes_count             3.22753043E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerJavaImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220304124334
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:158baf01a5f8f17310615b14715d3efc87fef6a9e97575436e0f3dec6a8bc8d0
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220304124334]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:158baf01a5f8f17310615b14715d3efc87fef6a9e97575436e0f3dec6a8bc8d0]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220304124334] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:158baf01a5f8f17310615b14715d3efc87fef6a9e97575436e0f3dec6a8bc8d0])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:158baf01a5f8f17310615b14715d3efc87fef6a9e97575436e0f3dec6a8bc8d0
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:158baf01a5f8f17310615b14715d3efc87fef6a9e97575436e0f3dec6a8bc8d0
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:158baf01a5f8f17310615b14715d3efc87fef6a9e97575436e0f3dec6a8bc8d0].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings

Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.

BUILD FAILED in 3h 20m 43s
109 actionable tasks: 72 executed, 33 from cache, 4 up-to-date

Publishing build scan...
https://gradle.com/s/cvj6tr56w27a4

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #257

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/257/display/redirect?page=changes>

Changes:

[Alexey Romanenko] Bump org.mongodb:mongo-java-driver to 3.12.10

[noreply] [BEAM-13973] Link Dataproc Flink master URLs to the InteractiveRunner

[noreply] [BEAM-13925] Turn pr bot on for go prs (#16984)

[Pablo Estrada] Skipping flaky sad-path tests for Spanner changestreams

[noreply] [BEAM-13964] Bump kotlin to 1.6.x (#16882)

[noreply] Merge pull request #16906: [BEAM-13974] Handle idle Storage Api streams

[noreply] Merge pull request #16562 from [BEAM-13051][D] Enable pylint warnings

[noreply] [BEAM-13925] A couple small pr-bot bug fixes (#16996)

[noreply] [BEAM-14029] Add getter, setter for target maven repo (#16995)

[noreply] [BEAM-13903] Improve coverage of metricsx package (#16994)

[noreply] [BEAM-13892] Improve coverage of avroio package (#16990)


------------------------------------------
[...truncated 48.49 KB...]
6d5bdee4481e: Preparing
3ff7f1b89814: Preparing
5feba7d0afd4: Preparing
da814db69f74: Preparing
804bc49f369a: Preparing
d1609e012401: Preparing
e3f84a8cee1f: Preparing
48144a6f44ae: Preparing
26d5108b2cba: Preparing
89fda00479fc: Preparing
ef71ca23d831: Waiting
3ff7f1b89814: Waiting
7d86b0fa5875: Waiting
5feba7d0afd4: Waiting
804bc49f369a: Waiting
da814db69f74: Waiting
d1609e012401: Waiting
48144a6f44ae: Waiting
e3f84a8cee1f: Waiting
d69ddd45c633: Waiting
18654c3dc7ba: Waiting
6d5bdee4481e: Waiting
26d5108b2cba: Waiting
63371a337244: Pushed
3ee33d36611d: Pushed
835b72d70440: Pushed
0af0e0fbfdb1: Pushed
ef71ca23d831: Pushed
55fc7aada96a: Pushed
18654c3dc7ba: Pushed
7bc364698301: Pushed
3ff7f1b89814: Pushed
d69ddd45c633: Pushed
7d86b0fa5875: Pushed
da814db69f74: Layer already exists
804bc49f369a: Layer already exists
d1609e012401: Layer already exists
e3f84a8cee1f: Layer already exists
48144a6f44ae: Layer already exists
26d5108b2cba: Layer already exists
89fda00479fc: Layer already exists
5feba7d0afd4: Pushed
6d5bdee4481e: Pushed
20220303124459: digest: sha256:d37cdf8623963e5234c0de2018c9b92542a7ece319702a285cbaa30459359b96 size: 4520

> Task :sdks:java:testing:load-tests:run
Mar 03, 2022 12:49:12 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Mar 03, 2022 12:49:15 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 206 files. Enable logging at DEBUG level to see which files will be staged.
Mar 03, 2022 12:49:18 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Mar 03, 2022 12:49:18 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Mar 03, 2022 12:49:24 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 206 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Mar 03, 2022 12:49:28 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 206 files cached, 0 files newly uploaded in 4 seconds
Mar 03, 2022 12:49:29 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Mar 03, 2022 12:49:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <114762 bytes, hash 1db0f3546f1bbaa8d1cdd2b82f904391b3a3ebe8e31e671ea204805e6f814f0b> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-HbDzVG8buqjRzdK4L5BDkbOj6-jjHmceogSAXm-BTws.pb
Mar 03, 2022 12:49:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Mar 03, 2022 12:49:32 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5cf3157b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@625dfff3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26350ea2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1e9469b8, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5a08efdc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@57272109, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@59696551, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@648d0e6d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@79e66b2f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@17273273, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5f69e2b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@984169e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43f1bb92, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6d6bbd35, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5c5d6175, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7544ac86, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3b27b497, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@b1534d3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c74aa0d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6c841199]
Mar 03, 2022 12:49:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Mar 03, 2022 12:49:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Mar 03, 2022 12:49:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Mar 03, 2022 12:49:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Mar 03, 2022 12:49:32 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@514de325, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@30c1da48, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43a65cd8, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3f1ef9d6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@17461db, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3fd9e827, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4e682398, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@670b3ca, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@24a86066, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@54402c04, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5b3bb1f7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58d6b7b9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3f1a4795, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6a6f6c7e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6c5ddccd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1dbd580, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6c101cc1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7d0d91a1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7fb48179, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@201c3cda]
Mar 03, 2022 12:49:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Mar 03, 2022 12:49:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Mar 03, 2022 12:49:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Mar 03, 2022 12:49:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Mar 03, 2022 12:49:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Mar 03, 2022 12:49:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Mar 03, 2022 12:49:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Mar 03, 2022 12:49:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Mar 03, 2022 12:49:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Mar 03, 2022 12:49:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Mar 03, 2022 12:49:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Mar 03, 2022 12:49:32 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.38.0-SNAPSHOT
Mar 03, 2022 12:49:33 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-03-03_04_49_33-17973105198643950507?project=apache-beam-testing
Mar 03, 2022 12:49:33 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2022-03-03_04_49_33-17973105198643950507
Mar 03, 2022 12:49:33 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2022-03-03_04_49_33-17973105198643950507
Mar 03, 2022 12:49:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2022-03-03T12:49:42.331Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-03-lwbl. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Mar 03, 2022 12:49:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-03T12:49:48.012Z: Worker configuration: e2-standard-2 in us-central1-b.
Mar 03, 2022 12:49:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-03T12:49:48.875Z: Expanding SplittableParDo operations into optimizable parts.
Mar 03, 2022 12:49:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-03T12:49:48.903Z: Expanding CollectionToSingleton operations into optimizable parts.
Mar 03, 2022 12:49:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-03T12:49:48.971Z: Expanding CoGroupByKey operations into optimizable parts.
Mar 03, 2022 12:49:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-03T12:49:49.042Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Mar 03, 2022 12:49:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-03T12:49:49.076Z: Expanding GroupByKey operations into streaming Read/Write steps
Mar 03, 2022 12:49:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-03T12:49:49.146Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Mar 03, 2022 12:49:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-03T12:49:49.258Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Mar 03, 2022 12:49:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-03T12:49:49.287Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Mar 03, 2022 12:49:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-03T12:49:49.322Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Mar 03, 2022 12:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-03T12:49:49.349Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Mar 03, 2022 12:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-03T12:49:49.386Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Mar 03, 2022 12:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-03T12:49:49.417Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Mar 03, 2022 12:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-03T12:49:49.454Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Mar 03, 2022 12:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-03T12:49:49.479Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Mar 03, 2022 12:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-03T12:49:49.511Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Mar 03, 2022 12:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-03T12:49:49.537Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Mar 03, 2022 12:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-03T12:49:49.607Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Mar 03, 2022 12:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-03T12:49:49.644Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Mar 03, 2022 12:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-03T12:49:49.667Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Mar 03, 2022 12:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-03T12:49:49.695Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Mar 03, 2022 12:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-03T12:49:49.729Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Mar 03, 2022 12:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-03T12:49:49.760Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Mar 03, 2022 12:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-03T12:49:49.792Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Mar 03, 2022 12:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-03T12:49:49.817Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Mar 03, 2022 12:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-03T12:49:49.841Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Mar 03, 2022 12:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-03T12:49:49.862Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Mar 03, 2022 12:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-03T12:49:49.887Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Mar 03, 2022 12:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-03T12:49:49.918Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Mar 03, 2022 12:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-03T12:49:49.940Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Mar 03, 2022 12:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-03T12:49:50.108Z: Running job using Streaming Engine
Mar 03, 2022 12:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-03T12:49:50.365Z: Starting 5 ****s in us-central1-b...
Mar 03, 2022 12:50:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-03T12:50:02.531Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Mar 03, 2022 12:50:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-03T12:50:36.488Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Mar 03, 2022 12:51:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-03T12:51:31.556Z: Workers have started successfully.
Mar 03, 2022 12:51:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-03T12:51:31.608Z: Workers have started successfully.
Mar 03, 2022 4:01:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-03T16:01:02.889Z: Cancel request is committed for workflow job: 2022-03-03_04_49_33-17973105198643950507.
Mar 03, 2022 4:01:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-03T16:01:02.990Z: Cleaning up.
Mar 03, 2022 4:01:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-03T16:01:03.079Z: Stopping **** pool...
Mar 03, 2022 4:01:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-03T16:01:03.158Z: Stopping **** pool...
Mar 03, 2022 4:03:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-03T16:03:21.663Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Mar 03, 2022 4:03:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-03T16:03:21.731Z: Worker pool stopped.
Mar 03, 2022 4:03:28 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2022-03-03_04_49_33-17973105198643950507 finished with status CANCELLED.
Load test results for test (ID): dd23e934-54a7-4127-b2e8-f19371e21ede and timestamp: 2022-03-03T12:49:17.120000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11308.384
dataflow_v2_java11_total_bytes_count             2.62453723E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerJavaImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220303124459
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:d37cdf8623963e5234c0de2018c9b92542a7ece319702a285cbaa30459359b96
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220303124459]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:d37cdf8623963e5234c0de2018c9b92542a7ece319702a285cbaa30459359b96]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220303124459] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:d37cdf8623963e5234c0de2018c9b92542a7ece319702a285cbaa30459359b96])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:d37cdf8623963e5234c0de2018c9b92542a7ece319702a285cbaa30459359b96
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:d37cdf8623963e5234c0de2018c9b92542a7ece319702a285cbaa30459359b96
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:d37cdf8623963e5234c0de2018c9b92542a7ece319702a285cbaa30459359b96].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings

Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.

BUILD FAILED in 3h 19m 41s
109 actionable tasks: 72 executed, 33 from cache, 4 up-to-date

Publishing build scan...
https://gradle.com/s/6mb7mnivhcd4y

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #256

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/256/display/redirect?page=changes>

Changes:

[egalpin] Use default context output rather than outputWithTimestamp for

[stranniknm] Palo Alto case study - fix link

[rogelio.hernandez] [BEAM-12777] Removed current docs version redirect

[noreply] Merge pull request #16850: [BEAM-11205] Upgrade Libraries BOM

[noreply] Merge pull request #16484 from [BEAM-13633] [Playground] Implement

[noreply] Add 2022 events blog post (#16975)

[noreply] Clean up Go formatter suggestions (#16973)

[noreply] [BEAM-14012] Add go fmt to Github Actions (#16978)

[noreply] [BEAM-13911] Add basic tests to Go direct runner. (#16979)

[noreply] [BEAM-13960] Add support for more types when converting from between row


------------------------------------------
[...truncated 593.69 KB...]
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmi
Mar 02, 2022 4:03:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-02T16:03:07.883Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Mar 02, 2022 4:03:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-02T16:03:07.918Z: Worker pool stopped.
Mar 02, 2022 4:03:14 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2022-03-02_04_48_02-8114063887973567786 finished with status CANCELLED.
Load test results for test (ID): b5211415-29b2-43ad-a40b-3845eb907dcf and timestamp: 2022-03-02T12:47:46.150000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11417.075
dataflow_v2_java11_total_bytes_count             2.48113054E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerJavaImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220302124420
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:f1f92d7a674fc4135c4a6f1f52f27796f6a52ac0af8c261a1fed35852a267307
Deleted: sha256:a5736b5024481f8e1bc7254972840da6a9ccee11eed2ffb027f3e8365fb3386e
Deleted: sha256:4138254ee35e6d79fc3f7fdb818edccd7588e9582f3f8fe1cea4d86585357c83
Deleted: sha256:aeb2340e1e813aec3114d76d5ff42d0e357e165cad7074418f0fda92c3de9e6d
Deleted: sha256:e142f8997cca4e6d633298f2deb152b95b2bec9394d3295bdd9815f5a56d9efc
Deleted: sha256:531851ee16f512d501e430ac3a365cb93f8ab672e3a6faa332c247569d47334f
Deleted: sha256:aec91832373af2de627a6e28864934a412d31887030162d9a9acd3430372cb46
Deleted: sha256:ace0e7d55d7a0a2c86c881abc8b18e5f803cc6be53500d39417182abda3fb077
Deleted: sha256:ff2a621d19d1b897bfed273dfdd79802176e335a3d3847e7deb4cd6156ff9062
Deleted: sha256:ec6f0fda68e5a4df70a7dfabe8da30473f10b4d7be5804cbc35f8011577ce131
Deleted: sha256:2fb32fdb89b63481e577828014aa54edf5399ed06904a62c7a559f91b637572d
Deleted: sha256:2b53875223490ae9e104d029e5e355281ec46fa12938d68a89c9a4174b5a33b1
Deleted: sha256:3266ee46048eaca3ea60a5bbb4b01c4cc5b6d2bf4641233a49d443eaf082dd1a
Deleted: sha256:c2807580bea0008baef4e6e51fdea18f5f2ebaa96ce3ed5410f6b24d27a0d112
Deleted: sha256:ba9f93fa191a731a8c08eaeb029eea09d1a77514ecb0cf85b240b7b307607106
Deleted: sha256:db2bc0d10636873d2faece85c3399220db38eb3bd9d99e3f1e9613111b209593
Deleted: sha256:98b7f7cd45d9f8dc3510fd58e7a433282e4491fd3fd6e5a150b125e9a89b2766
Deleted: sha256:2a724a490b831b3393b538695dca9c2b6a70718cd9d30349295f9943cd53f47a
Deleted: sha256:d432e01cdf3fe1037d83058c1f37157f6d3c366042c6da0e6e18b671f220a8a9
Deleted: sha256:24cb3c316e5a83fec0f813cd9695e178da4d325b23fe5cfec04d75d12cacfb8a
Deleted: sha256:08d91b1875b1fcf2e40152968d586a2880774f8294159d3c65a7e5de99ab2d4a
Deleted: sha256:d1128c2ca376bc901f91a9e948d8a333b20edd3193ad88a1ddfe3e18c04caca7
Deleted: sha256:ccf90571aa92375751daa649c5510cbbb3b5c9f3156a25209e88e017605209f8
Deleted: sha256:89781e66168fe766594bde69b0cd04ebd8580b08e45862cd9145a5b5264ab306
Deleted: sha256:d7765a1f1a6adbdaa180595b256a351001493532b2cbdb5e50d12558d58750ec
Deleted: sha256:2b023ba53e31bcb7815f1ff064d5ec5fddd2ee1c94ed94da3f869a5f4100150a
Deleted: sha256:82f687192f5f1aecd4e0ef06aa133e75cb51235e2aab3cfddf1be3cae31c9be6
Deleted: sha256:78e97d4ac63f2ddf115f7ebdbba254de17bf167a5a7cb691bca9c512d3fc5fd1
Deleted: sha256:801adf499529cbda4ab0ee2e011e0a9c09844f4a0e029db5358e93b8181862e4
Deleted: sha256:195ac04207fe714c6a69ea2c3f100b578aad963b0b2560efea16457ba1122c81
Deleted: sha256:4daaf229d0f84521cd9bd08735a1770d40ef6580fcf5c5908b25db29fc97c1fa
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220302124420]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:f1f92d7a674fc4135c4a6f1f52f27796f6a52ac0af8c261a1fed35852a267307]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220302124420] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:f1f92d7a674fc4135c4a6f1f52f27796f6a52ac0af8c261a1fed35852a267307])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:f1f92d7a674fc4135c4a6f1f52f27796f6a52ac0af8c261a1fed35852a267307
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:f1f92d7a674fc4135c4a6f1f52f27796f6a52ac0af8c261a1fed35852a267307
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:f1f92d7a674fc4135c4a6f1f52f27796f6a52ac0af8c261a1fed35852a267307].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings

Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.

BUILD FAILED in 3h 19m 41s
109 actionable tasks: 74 executed, 31 from cache, 4 up-to-date

Publishing build scan...
https://gradle.com/s/qfn2o6rctiy4q

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #255

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/255/display/redirect?page=changes>

Changes:

[noreply] Build wheels for Python 3.9

[noreply] Merge pull request #16892 from [BEAM-13755] [Playground] Scroll the

[noreply] Merge pull request #16880 from [BEAM-13963][Playground] Get bucket name

[noreply] Merge pull request #16870 from [BEAM-13874][Playground] Tag multifile

[noreply] Merge pull request #16910 from [BEAM-13724] [Playground] Get the default

[noreply] [BEAM-14008] Fix incorrect guava import (#16966)

[noreply] Fix ignored exception in BatchSpannerRead. (#16960)

[noreply] [BEAM-13917] Improve coverage of databaseio package (#16956)

[noreply] [BEAM-13925] Add entry files to process new prs and pr updates for PR

[noreply] [BEAM-13899] Improve coverage of debug package (#16951)

[noreply] [BEAM-13907] Improve coverage of textio package (#16937)

[noreply] [BEAM-9150] Fix beam_PostRelease_Python_Candidate (python RC validation


------------------------------------------
[...truncated 556.67 KB...]
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/windmill_cache_access.cc:40
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/windmill_cache_access.cc:40
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/windmill_cache_access.cc:40
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/windmill_cache_access.cc:40
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/windmill_cache_access.cc:40
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/windmill_cache_access.cc:40
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/windmill_cache_access.cc:40
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/windmill_cache_access.
Mar 01, 2022 4:00:47 PM org.apache.beam.sdk.metrics.MetricsEnvironment getCurrentContainer
WARNING: Reporting metrics are not supported in the current execution environment.
Mar 01, 2022 4:00:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-01T16:00:50.388Z: Cancel request is committed for workflow job: 2022-03-01_04_45_40-3174255768133400827.
Mar 01, 2022 4:00:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-01T16:00:50.576Z: Cleaning up.
Mar 01, 2022 4:00:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-01T16:00:50.664Z: Stopping **** pool...
Mar 01, 2022 4:00:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-01T16:00:50.708Z: Stopping **** pool...
Mar 01, 2022 4:03:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-01T16:03:15.690Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Mar 01, 2022 4:03:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-01T16:03:15.792Z: Worker pool stopped.
Mar 01, 2022 4:03:28 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2022-03-01_04_45_40-3174255768133400827 finished with status CANCELLED.
Load test results for test (ID): 26af09ff-74c6-4f0c-b2e6-8344070c924d and timestamp: 2022-03-01T12:45:35.587000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11562.261
dataflow_v2_java11_total_bytes_count             3.74283226E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerJavaImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220301124335
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:a3756776cfb9f7902720f53a08f44dbd46b6019309948e9d30688abc3f35fa14
Deleted: sha256:a267dfb59db9b6a0f6f8659eed4bfe48c3d042d161c39dfc4de87d579ab36c5f
Deleted: sha256:a3c24d76b23ea61b41257ed23879581e8ef6bfd63a6678330f7282cd2f4ac576
Deleted: sha256:3c75eea211d9650a1db4e9813830165eed6b96e4fa06131cb8cb73db14e1d880
Deleted: sha256:16302f0c6f6237114c23d3866f7e9a61b65fd1539e6567ca34ad5ef3023b5c78
Deleted: sha256:50f6da7332b2e71d9088d0d7cf857461a38ddeef093f5076ca6ff0226bd0dcd0
Deleted: sha256:49e797c5e407dc44dc3260ae6f0fc74958b34703260ae6015e043d03238b0e0a
Deleted: sha256:b38339c0fc71a1b3f3a641d58a0e615dcea270510f84ce984975b3aab676274b
Deleted: sha256:aecbc092ae914251253981059fbf2ccb4947289032a1976d9c549da864263085
Deleted: sha256:a7c0efcc6628315fcc6f2b079529d8d9b513a1aeba2ad810222703061bc0487e
Deleted: sha256:f52a97c1ea5d05ce002d3442007ffb87789b8a0decf58b583c7a5f654b4eedb2
Deleted: sha256:7c1616c578950dc2b94cc9f8dc7ef86705c9400a015213b0c9b7e305a98a40b6
Deleted: sha256:cd927f263ccfc829085c79de7874f88cfdbd84c6857940346319a107dfc176d2
Deleted: sha256:1981cf2c859767575ef0d92dc04c235b1f86ae4f3c7398c80c4730f47f63a7d3
Deleted: sha256:732d4ba3db3c16f52c455df8ff9705682f77dcfa4472dcff7402aad782ae93e6
Deleted: sha256:94ecd413d6ef5453e7f097108fae0879b57612417ecb7b44779f3bc6111dc329
Deleted: sha256:4367e54b34849f85db242af2c1dcba0dd4f2e42055aaf4b0192116e9f9498455
Deleted: sha256:54e2c3e3421ea06f98dee2d34f968601ba56d171fc435198eb06ee519ef2a02d
Deleted: sha256:707cdb5dffb767e23b3693fe787b3b70e9ab660cd0c5e5641ba13e5e8e924496
Deleted: sha256:723adb3d55d75cdeecb902ab9163c1dbba9f144bcb71ecea923e8f4b4382016d
Deleted: sha256:cc7ce91685d6f0f96b1c662b373b3aa9ec2a993c8c8e3b1f90f9cc0f8c9f0778
Deleted: sha256:6f374b7250be22611aef0eb2443185cd9b47b50517096974579a8d958a78fa6b
Deleted: sha256:a91ed18ad4468bda35fe324c83712ff2ff677f3aab198bbc088a8042871277b6
Deleted: sha256:fafc84ade5ace43326963faabcb2394e772a863fbd063cba8dee82f57a1bb568
Deleted: sha256:aa84aaf2836214b43e5f98c184d7a5f92a33e30066fc26e69376c1e1990488b8
Deleted: sha256:eb41e638a57ed0ac0198da6f7d986a20986b31d0528d3cf9c0ce2e371ae2318b
Deleted: sha256:3ec2e59137ba6263a1cd07cf80d515bccb8f7827c1fc65a10403f5ef1ba029c9
Deleted: sha256:899b369799c9efd5145cc86c2bc1098778aab24ace7560ef76e02e3fba232778
Deleted: sha256:f5c012a2c2341e9825bed535371caa19fa5f4599b63f2891729bf1297ea1da8f
Deleted: sha256:38fd2cdd86a67ff18476a0057682be3cc53658c5d67dc3124231d73a16bcc9c6
Deleted: sha256:26bd2ed9467e46c500e04242ee6605642fa3f5ecfdcefee46a10bc6038c2bcc3
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220301124335]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:a3756776cfb9f7902720f53a08f44dbd46b6019309948e9d30688abc3f35fa14]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220301124335] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:a3756776cfb9f7902720f53a08f44dbd46b6019309948e9d30688abc3f35fa14])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:a3756776cfb9f7902720f53a08f44dbd46b6019309948e9d30688abc3f35fa14
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:a3756776cfb9f7902720f53a08f44dbd46b6019309948e9d30688abc3f35fa14
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:a3756776cfb9f7902720f53a08f44dbd46b6019309948e9d30688abc3f35fa14].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings

Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.

BUILD FAILED in 3h 20m 15s
109 actionable tasks: 72 executed, 33 from cache, 4 up-to-date

Publishing build scan...
https://gradle.com/s/3d7hyd3dsutbc

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #254

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/254/display/redirect>

Changes:


------------------------------------------
[...truncated 1.22 MB...]
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmi
Feb 28, 2022 4:03:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-28T16:03:25.604Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Feb 28, 2022 4:03:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-28T16:03:25.643Z: Worker pool stopped.
Feb 28, 2022 4:03:32 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2022-02-28_04_45_52-11858468572081011038 finished with status CANCELLED.
Load test results for test (ID): 86409f91-f5b1-411b-bdaf-fcc57df11fb8 and timestamp: 2022-02-28T12:45:47.058000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11569.692
dataflow_v2_java11_total_bytes_count             3.49153289E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerJavaImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220228124338
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:5e911027a3b004ed3985c979619d43b8667f8124268411bc6efc81a17cf55f7e
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220228124338]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:5e911027a3b004ed3985c979619d43b8667f8124268411bc6efc81a17cf55f7e]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220228124338] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:5e911027a3b004ed3985c979619d43b8667f8124268411bc6efc81a17cf55f7e])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:5e911027a3b004ed3985c979619d43b8667f8124268411bc6efc81a17cf55f7e
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:5e911027a3b004ed3985c979619d43b8667f8124268411bc6efc81a17cf55f7e
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:5e911027a3b004ed3985c979619d43b8667f8124268411bc6efc81a17cf55f7e].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings

Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.

BUILD FAILED in 3h 20m 15s
109 actionable tasks: 72 executed, 33 from cache, 4 up-to-date

Publishing build scan...
https://gradle.com/s/dol6avvnm2mwi

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #253

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/253/display/redirect>

Changes:


------------------------------------------
[...truncated 840.58 KB...]
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/windmill_cache_access.cc:40
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/windmill_cache_access.cc:40
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/windmill_cache_access.cc:40
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/windmill_cache_access.cc:40
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/windmill_cache_access.cc:40
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/windmill_cache_access.cc:40
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/windmill_cache_access.cc:40
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/windmill_cache_access.cc:40
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/windmill_cache_access.cc:40
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/windmill_cache_access.cc:40
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/windmill_cache_access.cc:40
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/windmill_cache_access.cc:40
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/windmill_cache_access.cc:40
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the
Feb 27, 2022 4:00:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-27T16:00:35.916Z: Cancel request is committed for workflow job: 2022-02-27_04_45_42-4525283117407589383.
Feb 27, 2022 4:00:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-27T16:00:36.018Z: Cleaning up.
Feb 27, 2022 4:00:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-27T16:00:36.078Z: Stopping **** pool...
Feb 27, 2022 4:00:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-27T16:00:36.120Z: Stopping **** pool...
Feb 27, 2022 4:02:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-27T16:02:52.923Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Feb 27, 2022 4:02:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-27T16:02:52.965Z: Worker pool stopped.
Feb 27, 2022 4:03:00 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2022-02-27_04_45_42-4525283117407589383 finished with status CANCELLED.
Load test results for test (ID): caab9eaf-8109-4d80-a905-c2695a80166d and timestamp: 2022-02-27T12:45:37.243000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11547.894
dataflow_v2_java11_total_bytes_count             3.19826367E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerJavaImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220227124339
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:a1327ed4fd4a06acfbe801b379c7308bb475be2c5ca1938062cd1e8f18d26d02
Deleted: sha256:6710e3b7da000327d1db5c4339465474ba57d60e1db2bae121548153b0d263c8
Deleted: sha256:598c17a70709cab9b3baf541026e7f258d6b57fcaccb34689c995b96ed6a4f45
Deleted: sha256:2cde009e4b7803e35daa881f254ece9bcfc6fb50df0d083e2c5d57eba86d1ad8
Deleted: sha256:e5635e9fd5c92be050cf30f5e08d43864b8cc807aafde84ea2ea6cadafd7b8ed
Deleted: sha256:9f1b1d8c1ac5d63c4fb32ce08b49f59c18dfc5e8eb9416e96efa05f9051125fa
Deleted: sha256:cd30bdcb8dac5023b12dbdc86693b71dfc2eee1d4754b7ba3c716558a5a3f34e
Deleted: sha256:5d0dc58ddce506e1c2f77ab8353f4971f81d6a36bbfe487678805d63c8b5fdf8
Deleted: sha256:037099103f4277dc75cb0e0f9e8ce7718fb5f622c1171fa1c831fd7d3744c3b6
Deleted: sha256:07b78971c9ab10009f0b8b78a8324e317c007fe36435a614c424a0de0d141912
Deleted: sha256:a13577ae76d6b58e41d3e382f68356f06e65db046175a696839ddcc4a582d464
Deleted: sha256:50e3edc9dfda1570baeeffea9e8e4d2dbafb1b0e1c2c273d891e5f39208c9236
Deleted: sha256:b29a000c53e10c0ee3e6db9e343324202e43767f14e55b705bbdd6ec1f5cda3a
Deleted: sha256:68569259a67f97c10f81be06ec0b5a2ad66867f65820382934a6212dff5bcfc3
Deleted: sha256:7241420afd5e5e87d8806b3e95e5a117ac90eebfbccae2d6ba40031708e61be7
Deleted: sha256:e8d59fb05298fd1fba8361e751d9ea04517a73a9575f77f290e573bf046edb14
Deleted: sha256:1b3cd64c7cf6e058f0ae7302df21e1d39498304e57718a974f414ee321b412a9
Deleted: sha256:1162861b6441979a773a1bccd003c024e7140ebf20528b1ac3d168f13aa8b500
Deleted: sha256:382025fb547966f51c506765ae1656f4451bb818ea4c6b2880f2dbd96f27c762
Deleted: sha256:8c902f3f210bada370e5a6103ce545076214a1bc01caf118d0792db83da43ddc
Deleted: sha256:41f7418504cae0d13fd0842bf0815a3657c9aa843d8acd4ad445c06271fefa7a
Deleted: sha256:ba549dd1924af9c87c287fa0db0d741fdf193afa4762c5117cef39f9082d530c
Deleted: sha256:c5eaf11ac52476b7c784db3c0c610cdf982b2e3add3fccc0555c69c3525209e4
Deleted: sha256:0e9ec756de07dc05bf50aa2eb243cab1410c4f5b5b34b53da61c14057cfc3ef5
Deleted: sha256:4957dcb5752414266f018a900a0871b7db39d5b088d6be63f778012637e59d53
Deleted: sha256:873b68cda1ae9f12ca528494d34d9a5e16192b7d712cf24be5d30f38ff7ef548
Deleted: sha256:474505375839d30811ed78c9795bc0a6956fdae47e2d87ca3756dac43da2db62
Deleted: sha256:fd9cfdef5651af22e1704bfeac538349e293fa31fd051d7168f88f137484011b
Deleted: sha256:0ae2fd12e5d673aab19c703917f9b3f73bbac572955af80a2de610c0b10be8e9
Deleted: sha256:d8f633ca169233192eb3d40061c946c6f2e21eafb27be7726f5ed9e951064482
Deleted: sha256:24a75fe045880a8f635c918e150e39b762a83209538e993fb9bb5b01d13ee34e
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220227124339]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:a1327ed4fd4a06acfbe801b379c7308bb475be2c5ca1938062cd1e8f18d26d02]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220227124339] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:a1327ed4fd4a06acfbe801b379c7308bb475be2c5ca1938062cd1e8f18d26d02])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:a1327ed4fd4a06acfbe801b379c7308bb475be2c5ca1938062cd1e8f18d26d02
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:a1327ed4fd4a06acfbe801b379c7308bb475be2c5ca1938062cd1e8f18d26d02
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:a1327ed4fd4a06acfbe801b379c7308bb475be2c5ca1938062cd1e8f18d26d02].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings

Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.

BUILD FAILED in 3h 19m 39s
109 actionable tasks: 72 executed, 33 from cache, 4 up-to-date

Publishing build scan...
https://gradle.com/s/gcsbarxganbe2

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #252

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/252/display/redirect?page=changes>

Changes:

[Ankur Goenka] [BEAM-13952] Sickbaying

[noreply] Memoize some objects for timer processing to reduce overhead. (#16207)

[noreply] [BEAM-13965] Use TypeDeserializer if type information is available to

[noreply] [BEAM-13912] Add more coverage for dataflow.go (#16903)

[noreply] [BEAM-12563] swaplevel general function for dataframe and series

[noreply] [BEAM-14001] Update coder.go unit tests (#16952)

[noreply] [BEAM-13910] Improve coverage of gcsx package (#16942)

[noreply] [BEAM-13015] Use a DirectExecutor for state since we are just completing


------------------------------------------
[...truncated 490.59 KB...]
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmi
Feb 26, 2022 4:03:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-26T16:03:15.527Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Feb 26, 2022 4:03:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-26T16:03:15.572Z: Worker pool stopped.
Feb 26, 2022 4:03:23 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2022-02-26_04_46_21-5289173685615956349 finished with status CANCELLED.
Load test results for test (ID): cdc5142b-2c38-4b4c-b806-6ae6127c4a67 and timestamp: 2022-02-26T12:46:14.901000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                   11525.5
dataflow_v2_java11_total_bytes_count             2.80314284E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerJavaImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220226124336
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ee4350a0dcb60ea9fc38df8d39d4754f20747ab6b0098ef209adccf6f4bc11e2
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220226124336]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ee4350a0dcb60ea9fc38df8d39d4754f20747ab6b0098ef209adccf6f4bc11e2]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220226124336] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ee4350a0dcb60ea9fc38df8d39d4754f20747ab6b0098ef209adccf6f4bc11e2])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ee4350a0dcb60ea9fc38df8d39d4754f20747ab6b0098ef209adccf6f4bc11e2
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ee4350a0dcb60ea9fc38df8d39d4754f20747ab6b0098ef209adccf6f4bc11e2
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ee4350a0dcb60ea9fc38df8d39d4754f20747ab6b0098ef209adccf6f4bc11e2].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings

Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.

BUILD FAILED in 3h 20m 8s
109 actionable tasks: 72 executed, 33 from cache, 4 up-to-date

Publishing build scan...
https://gradle.com/s/j5ktflthhdmwc

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #251

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/251/display/redirect?page=changes>

Changes:

[benjamin.gonzalez] Revert PR#16253 due errors with plugin flaky-test-handler

[noreply] Fix BoundedQueueExecutor and StreamingDataflowWorker to actually limit

[noreply] [BEAM-1857] Add Neo4jIO (#15916)

[noreply] [BEAM-13767] Migrate serveral portable runner tasks to use configuration

[noreply] [BEAM-13996] Removing 'No cluster_manager is associated with the

[noreply] [BEAM-13906] Improve coverage of errors package (#16934)

[noreply] [BEAM-13886] unit tests for trigger package (#16935)

[noreply] [BEAM-4767] Remove beam- prefix from release script tags (#16899)

[noreply] [BEAM-13866] Add small unit tests to errorx, make boolean assignment

[noreply] [BEAM-13925] Add most of the supporting files for the pr management

[noreply] Merge pull request #16846 from [BEAM-12164]: Add sad path tests for


------------------------------------------
[...truncated 69.80 KB...]
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
Feb 25, 2022 12:51:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-02-25T12:51:29.818Z: generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
Feb 25, 2022 3:55:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-02-25T15:55:23.528Z: Staged package animal-sniffer-annotations-1.20-vt1E38otwrj1wIzR1vDgznQJTsZ3gSYJaOA_wOd1Iqw.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/animal-sniffer-annotations-1.20-vt1E38otwrj1wIzR1vDgznQJTsZ3gSYJaOA_wOd1Iqw.jar' is inaccessible.
Feb 25, 2022 3:55:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-02-25T15:55:24.161Z: Staged package beam-runners-core-java-2.38.0-SNAPSHOT-g1gf2aIDSblOWLSzcUcynwLjKg419MgwyCpDR4jXU8Q.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-java-2.38.0-SNAPSHOT-g1gf2aIDSblOWLSzcUcynwLjKg419MgwyCpDR4jXU8Q.jar' is inaccessible.
Feb 25, 2022 3:55:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-02-25T15:55:28.621Z: Staged package opencensus-contrib-http-util-0.28.0-ScPbKinx_bL3OSjL6pab0dQPq3zFu2JzAiur2W96eJs.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/opencensus-contrib-http-util-0.28.0-ScPbKinx_bL3OSjL6pab0dQPq3zFu2JzAiur2W96eJs.jar' is inaccessible.
Feb 25, 2022 3:55:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2022-02-25T15:55:29.444Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Feb 25, 2022 3:58:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2022-02-25T15:58:28.375Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Feb 25, 2022 4:00:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-25T16:00:42.006Z: Cancel request is committed for workflow job: 2022-02-25_04_46_01-8982574014897584564.
Feb 25, 2022 4:00:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-25T16:00:42.035Z: Cleaning up.
Feb 25, 2022 4:00:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-25T16:00:42.109Z: Stopping **** pool...
Feb 25, 2022 4:00:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-25T16:00:42.173Z: Stopping **** pool...
Feb 25, 2022 4:03:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-25T16:03:13.318Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Feb 25, 2022 4:03:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-25T16:03:13.363Z: Worker pool stopped.
Feb 25, 2022 4:03:20 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2022-02-25_04_46_01-8982574014897584564 finished with status CANCELLED.
Load test results for test (ID): c0b28dd9-f76c-4e4f-8e77-4822917b1650 and timestamp: 2022-02-25T12:45:52.531000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11520.807
dataflow_v2_java11_total_bytes_count               2.835687E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerJavaImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220225124344
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:4123850e9a343e91eb97d310664b7b3418c71d9e24c7152c47df9ee75c356982
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220225124344]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:4123850e9a343e91eb97d310664b7b3418c71d9e24c7152c47df9ee75c356982]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220225124344] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:4123850e9a343e91eb97d310664b7b3418c71d9e24c7152c47df9ee75c356982])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:4123850e9a343e91eb97d310664b7b3418c71d9e24c7152c47df9ee75c356982
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:4123850e9a343e91eb97d310664b7b3418c71d9e24c7152c47df9ee75c356982
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:4123850e9a343e91eb97d310664b7b3418c71d9e24c7152c47df9ee75c356982].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings

Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.

BUILD FAILED in 3h 20m 1s
109 actionable tasks: 74 executed, 31 from cache, 4 up-to-date

Publishing build scan...
https://gradle.com/s/n4gpaj57gqr52

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #250

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/250/display/redirect?page=changes>

Changes:

[noreply] [BEAM-12645] Fix code-cov flakes due to monorepo. (#16925)

[noreply] [BEAM-13969] Deprecate stringx package (#16884)

[noreply] Add Go badge to ReadMe (#16897)

[noreply] [BEAM-13980] Re-add method gone missing in af2f8ee6 (#16918)

[noreply] [BEAM-13884] Improve mtime package (#16924)

[noreply] Minor: Update Go API doc links (#16932)

[noreply] [BEAM-13218] Re-enable

[noreply] Merge pull request #16857 from [BEAM-13662] [Playground] Support

[noreply] Merge pull request #16826 from [BEAM-13870] [Playground] Increase test


------------------------------------------
[...truncated 275.65 KB...]
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid
Feb 24, 2022 4:03:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-24T16:03:19.529Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Feb 24, 2022 4:03:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-24T16:03:19.568Z: Worker pool stopped.
Feb 24, 2022 4:03:27 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2022-02-24_04_47_03-2762069967532466111 finished with status CANCELLED.
Load test results for test (ID): 7679e447-256a-4095-a958-a80c1c535077 and timestamp: 2022-02-24T12:46:56.095000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11424.649
dataflow_v2_java11_total_bytes_count             3.76402655E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerJavaImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220224124440
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:9076ce22567ee1b9f39a2d01bc3274e942b9333fd39b43d5b77b5b8ad6d6ad4b
Deleted: sha256:31079c1cd0e8b88d02d173a04a5a251abb6766ad41da3f06d2d763b656d1d2d6
Deleted: sha256:23c50f97613c243ea72cc3c61703d61d4ca3dcf2f78615deb23e5df4c8f6917d
Deleted: sha256:fc7cdd17b8b6bb99f2007b596426e9e3cfd7ed95ce3638445c31c55f5dae6649
Deleted: sha256:f1ce1eef07a834078ca4c6152b54d333e1bcdf93f8104459a2c310f524fae76d
Deleted: sha256:37fbb4edaa5d3ec6750e9ffff7a2df94e3401bedffa06dfa46bcc0898bd7e72d
Deleted: sha256:aac0b18ac0706b91a469d0ecc4b8144fdd66a3105d115cf8e639fe070e91acf5
Deleted: sha256:395ad758dc46cdf3dd3db40cae2afa41fba18da333b51487cef02a6665956054
Deleted: sha256:1217f134588443bf35d55bef44a1e7990fa9962aa7f399075b6bf620d6d76120
Deleted: sha256:f19b3c71f89828591d3936bd9b5c0b9189061218c59deb211d26ee4fcce23eb0
Deleted: sha256:058edbad473d3061365db63dc65a1d822b72e5a1989022a419784bc7a69fe854
Deleted: sha256:a50eb1360d0c6f45a7de1982280e81b2ce3aaaf8a1ee38a926367950101982aa
Deleted: sha256:801ff70683b23c1eed3342d481384d534d4af06c8ede220a0a89932bc23f734a
Deleted: sha256:8d98f5d748b3d72eafc085b1b8037453e222f69a0079a3834fbb67565729e4af
Deleted: sha256:6baf14b6d80b7d68ee35d00af9fb65547c4328bc23ba47f1a235d8375bd783fa
Deleted: sha256:169074b26364b02dc3cec172ee84044cd17cea1c2b652adeeff3abf54968b57f
Deleted: sha256:2a18700acde0efc6886c0f9d37cdd366667343478e1f8a810a4320b7dc982d6d
Deleted: sha256:4cead35f2c8d47bd41539eddbc1b05d7ba5db36a83f01d5fb98433d9d26c8f2d
Deleted: sha256:2b885037c2540f875376662f1f6531e3839992596b1a1af69df2b778e4dc123e
Deleted: sha256:fe2e453174c3417153c7ecbc38d2ab4a681370d09a5885c584eae87222a8efaf
Deleted: sha256:78164900173b7ab463cf8ec5b4296fce19fa7c083966f84eb2a08051c52b305c
Deleted: sha256:5d3df9b5b7ad8d14ea314e3f59d68dee54c85b660ea63817592f73df8417bcf9
Deleted: sha256:592bae978bd39dc5c90c58c7f81bb6c86b71c3ac0e3169473ae9a1de58a4cc58
Deleted: sha256:161e57ae0b490339d88e8c116b4d47b78f03576b32ed8a548f5c3360f7629c86
Deleted: sha256:4ba63a20483625cae0c567abb5f3f640c63e0664daaae6b4fcb8d977a7f5ff01
Deleted: sha256:3969e3218fdfb4a2f73923b934820b0bb403d3d4856d61eeab0d00d8a45944cd
Deleted: sha256:ebd74a1fe3d67fe34b87b25d532565e1531e44c374dce96a2ef91678c74008b6
Deleted: sha256:97b48cee63d1e78a912b774a779bc9188bf4b1def5ff2ac9f62741036f93a6a6
Deleted: sha256:4388cd9d13d809e3eccf6621034bb600c9a88087ff910dbbdf4407eaf77e6282
Deleted: sha256:4dfa9a9d0d69f7387e56db2d0d20e6fa362698bd23b3abcd1f8d642c739fb9eb
Deleted: sha256:3973bc874813844b6c2fa072cd2c36abdf668270e2cb415c2c15df3560940288
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220224124440]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:9076ce22567ee1b9f39a2d01bc3274e942b9333fd39b43d5b77b5b8ad6d6ad4b]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220224124440] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:9076ce22567ee1b9f39a2d01bc3274e942b9333fd39b43d5b77b5b8ad6d6ad4b])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:9076ce22567ee1b9f39a2d01bc3274e942b9333fd39b43d5b77b5b8ad6d6ad4b
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:9076ce22567ee1b9f39a2d01bc3274e942b9333fd39b43d5b77b5b8ad6d6ad4b
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:9076ce22567ee1b9f39a2d01bc3274e942b9333fd39b43d5b77b5b8ad6d6ad4b].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings

Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.

BUILD FAILED in 3h 19m 14s
109 actionable tasks: 72 executed, 33 from cache, 4 up-to-date

Publishing build scan...
https://gradle.com/s/owkktz3mzfauu

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #249

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/249/display/redirect?page=changes>

Changes:

[Kyle Weaver] [BEAM-13796] projection pushdown in BQ IO

[Kyle Weaver] [BEAM-13796] Move test to ReadTest class and correct javadoc for

[Kyle Weaver] [BEAM-13796] Pushdown is not supported on TypedRead#fromQuery.

[noreply] [BEAM-13738] Reenable ignored SQS test after bumping elasticmq for fixed

[noreply] fix build status link (#16907)

[noreply] Merge pull request #16549 from [BEAM-13681][Playground] Refactoring

[noreply] Merge pull request #16732 from [BEAM-13825] [Playground] updated

[noreply] Merge pull request #16683 from [BEAM-13713][Playground] Java graph

[noreply] case study pages - improvements and fixes (#16896)

[noreply] Palo Alto case study (#16915)


------------------------------------------
[...truncated 86.10 KB...]
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
Feb 23, 2022 12:53:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-02-23T12:53:28.071Z: generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
Feb 23, 2022 12:53:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-02-23T12:53:28.936Z: generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
FATAL: command execution failed
java.io.IOException: Backing channel 'apache-beam-jenkins-7' is disconnected.
	at hudson.remoting.RemoteInvocationHandler.channelOrFail(RemoteInvocationHandler.java:216)
	at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:286)
	at com.sun.proxy.$Proxy126.isAlive(Unknown Source)
	at hudson.Launcher$RemoteLauncher$ProcImpl.isAlive(Launcher.java:1213)
	at hudson.Launcher$RemoteLauncher$ProcImpl.join(Launcher.java:1205)
	at hudson.Launcher$ProcStarter.join(Launcher.java:522)
	at hudson.plugins.gradle.Gradle.perform(Gradle.java:317)
	at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
	at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:806)
	at hudson.model.Build$BuildExecution.build(Build.java:198)
	at hudson.model.Build$BuildExecution.doRun(Build.java:163)
	at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:514)
	at hudson.model.Run.execute(Run.java:1888)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
	at hudson.model.ResourceController.execute(ResourceController.java:99)
	at hudson.model.Executor.run(Executor.java:432)
Caused by: java.io.IOException: Unexpected termination of the channel
	at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:75)
Caused by: java.io.EOFException
	at java.io.ObjectInputStream$PeekInputStream.readFully(ObjectInputStream.java:2799)
	at java.io.ObjectInputStream$BlockDataInputStream.readShort(ObjectInputStream.java:3274)
	at java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:934)
	at java.io.ObjectInputStream.<init>(ObjectInputStream.java:396)
	at hudson.remoting.ObjectInputStreamEx.<init>(ObjectInputStreamEx.java:49)
	at hudson.remoting.Command.readFrom(Command.java:142)
	at hudson.remoting.Command.readFrom(Command.java:128)
	at hudson.remoting.AbstractSynchronousByteArrayCommandTransport.read(AbstractSynchronousByteArrayCommandTransport.java:35)
	at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:61)
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
ERROR: apache-beam-jenkins-7 is offline; cannot locate jdk_1.8_latest

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #247

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/247/display/redirect>

Changes:


------------------------------------------
[...truncated 649.46 KB...]
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    ./dist_proc/dax/workflow/****/streaming/windowing_api_delegate.h:67
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    ./dist_proc/dax/workflow/****/streaming/windowing_api_delegate.h:67
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    ./dist_proc/dax/workflow/****/streaming/windowing_api_delegate.h:67
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    ./dist_proc/dax/workflow/****/streaming/windowing_api_delegate.h:67
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    ./dist_proc/dax/workflow/****/streaming/windowing_api_delegate.h:67
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    ./dist_proc/dax/workflow/****/streaming/windowing_api_delegate.h:67
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    ./dist_proc/dax/workflow/****/streaming/windowing_api_delegate.h:67
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    ./dist_proc/dax/workflow/****/streaming/windowing_api_delegate.h:67
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    ./dist_proc/dax/workflow/****/streaming/windowing_api_delegate.h:67
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    ./dist_proc/dax/workflow/****/streaming/windowing_api_delegate.h:67
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    ./dist_proc/dax/workflow/****/streaming/windowing_api_delegate.h:67
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    ./dist_proc/dax/workflow/****/streaming/windowing_api_delegate.h:67
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    ./dist_proc/dax/workflow/****/streaming/windowing_api_delegate.h:67
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    ./dist_proc/dax/workflow/****/streaming/windowing_api_delegate.h:67
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    ./dist_proc/dax/workflow/****/streaming/windowing_api_delegate.h:67
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streamin
FATAL: command execution failed
java.io.IOException: Backing channel 'apache-beam-jenkins-9' is disconnected.
	at hudson.remoting.RemoteInvocationHandler.channelOrFail(RemoteInvocationHandler.java:216)
	at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:286)
	at com.sun.proxy.$Proxy121.isAlive(Unknown Source)
	at hudson.Launcher$RemoteLauncher$ProcImpl.isAlive(Launcher.java:1213)
	at hudson.Launcher$RemoteLauncher$ProcImpl.join(Launcher.java:1205)
	at hudson.Launcher$ProcStarter.join(Launcher.java:522)
	at hudson.plugins.gradle.Gradle.perform(Gradle.java:317)
	at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
	at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:806)
	at hudson.model.Build$BuildExecution.build(Build.java:198)
	at hudson.model.Build$BuildExecution.doRun(Build.java:163)
	at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:514)
	at hudson.model.Run.execute(Run.java:1888)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
	at hudson.model.ResourceController.execute(ResourceController.java:99)
	at hudson.model.Executor.run(Executor.java:432)
Caused by: java.io.IOException: Pipe closed after 0 cycles
	at org.apache.sshd.common.channel.ChannelPipedInputStream.read(ChannelPipedInputStream.java:126)
	at org.apache.sshd.common.channel.ChannelPipedInputStream.read(ChannelPipedInputStream.java:105)
	at hudson.remoting.FlightRecorderInputStream.read(FlightRecorderInputStream.java:93)
	at hudson.remoting.ChunkedInputStream.readHeader(ChunkedInputStream.java:74)
	at hudson.remoting.ChunkedInputStream.readUntilBreak(ChunkedInputStream.java:104)
	at hudson.remoting.ChunkedCommandTransport.readBlock(ChunkedCommandTransport.java:39)
	at hudson.remoting.AbstractSynchronousByteArrayCommandTransport.read(AbstractSynchronousByteArrayCommandTransport.java:34)
	at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:61)
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
ERROR: apache-beam-jenkins-9 is offline; cannot locate jdk_1.8_latest

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #248

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/248/display/redirect>

Changes:


------------------------------------------
[...truncated 552.90 KB...]
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/windmill_cache_access.cc:40
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/windmill_cache_access.cc:40
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/windmill_cache_access.cc:40
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/windmill_cache_access.cc:40
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/windmill_cache_access.cc:40
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/windmill_cache_access.cc:40
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/windmill_cache_access.cc:40
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/windmill_cache_access.cc:40
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/windmill_cache_access.cc:40
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/windmill_cache_access.cc:40
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/windmill_cache_access.cc:40
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/windmill_cache_access.cc:40
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/windmill_cache_access.cc:40
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/windmill/client/streaming_rpc_client.cc:697
==>
    dist_proc/dax/workflow/****/streaming/windmill_cache_access.cc:40
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:230
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during
Build timed out (after 240 minutes). Marking the build as aborted.
Agent went offline during the build
ERROR: Connection was broken: org.apache.sshd.common.channel.exception.SshChannelClosedException: write(ChannelOutputStream[ChannelExec[id=4, recipient=0]-ClientSessionImpl[jenkins@/35.222.180.153:22]] SSH_MSG_CHANNEL_DATA) len=2 - channel already closed
	at org.apache.sshd.common.channel.ChannelOutputStream.write(ChannelOutputStream.java:132)
	at java.io.OutputStream.write(OutputStream.java:75)
	at hudson.remoting.ChunkedOutputStream.sendFrame(ChunkedOutputStream.java:92)
	at hudson.remoting.ChunkedOutputStream.sendBreak(ChunkedOutputStream.java:65)
	at hudson.remoting.ChunkedCommandTransport.writeBlock(ChunkedCommandTransport.java:46)
	at hudson.remoting.AbstractSynchronousByteArrayCommandTransport.write(AbstractSynchronousByteArrayCommandTransport.java:46)
	at hudson.remoting.Channel.send(Channel.java:766)
	at hudson.remoting.Channel.close(Channel.java:1487)
	at hudson.remoting.Channel.close(Channel.java:1454)
	at hudson.slaves.SlaveComputer.closeChannel(SlaveComputer.java:894)
	at hudson.slaves.SlaveComputer.access$100(SlaveComputer.java:108)
	at hudson.slaves.SlaveComputer$2.run(SlaveComputer.java:774)
	at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
	at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:68)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

Build step 'Invoke Gradle script' marked build as failure
ERROR: apache-beam-jenkins-9 is offline; cannot locate jdk_1.8_latest

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #246

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/246/display/redirect?page=changes>

Changes:

[rogelio.hernandez] [BEAM-13051] Pylint misplaced-bare-raise warning enabled

[Jeff Tapper] Update Java LTS roadmap info on website for Java 17

[Pablo Estrada] Simplify README for new users

[Kyle Weaver] [BEAM-13106] Support Flink 1.14.

[Kyle Weaver] [BEAM-13106] Reuse executor instead of shutting it down mid-test.

[Kyle Weaver] [BEAM-13106] Prevent infinite wait in Flink savepoint test.

[Ismaël Mejía] [BEAM-13202] Fix typos on tests names for VarianceFnTest

[Kenneth Knowles] Disable AfterSynchronizedProcessingTime test on Dataflow

[Ismaël Mejía] [BEAM-13202] Add Coder to CountIfFn.Accum

[Ismaël Mejía] [BEAM-13202] Reuse Count transform code since CountIf is a specific case

[Kenneth Knowles] Add test category UsesProcessingTimeTimers

[Kenneth Knowles] Label tests that need UsesProcessingTimeTimers

[Kenneth Knowles] Exclude UsesProcessingTimeTimers from SamzaRunner tests

[Kyle Weaver] [BEAM-13106] A couple additional fixes to FlinkSavepointTest.

[mmack] [adhoc] Migrate KinesisIOIT to use ITEnvironment for Localstack based IT

[rogelio.hernandez] [BEAM-13051] Added descriptions to Kinesis and PortableRunner exceptions

[noreply] [BEAM-13955] Fix pylint breakage from #16836 (#16867)

[relax] Fix TableRow conversion for the case of fields named "f"

[noreply] Bump dataflow.fnapi_container_version (#16874)

[mmack] [BEAM-13563] Introducing common AWS ClientBuilderFactory to unify

[laraschmidt] Fix final allowskew error to properly handle a large allowedSkew

[noreply] Case studies page improvements (#16702)

[noreply] [BEAM-13946] Add get_dummies(), a non-deferred column operation on

[noreply] [release-2.36.0] Fix pickler argument for 2.36 blog (#16774)

[thiagotnunes] fix: fix bug when retrieving either string or json

[noreply] [adhoc] Avoid using SerializablePipelineOptions for testing to minimize

[noreply] [BEAM-13812] Integrate DataprocClusterManager into Interactive

[noreply] [BEAM-12572] Fix failing python examples tests in Dataflow runner

[noreply] Remove build status from PR (#16902)


------------------------------------------
[...truncated 962.91 KB...]
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    ./dist_proc/dax/workflow/****/streaming/windowing_api_delegate.h:67
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    ./dist_proc/dax/workflow/****/streaming/windowing_api_delegate.h:67
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    ./dist_proc/dax/workflow/****/streaming/windowing_api_delegate.h:67
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    ./dist_proc/dax/workflow/****/streaming/windowing_api_delegate.h:67
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    ./dist_proc/dax/workflow/****/streaming/windowing_api_delegate.h:67
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    ./dist_proc/dax/workflow/****/streaming/windowing_api_delegate.h:67
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    ./dist_proc/dax/workflow/****/streaming/windowing_api_delegate.h:67
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    ./dist_proc/dax/workflow/****/streaming/windowing_api_delegate.h:67
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    ./dist_proc/dax/workflow/****/streaming/windowing_api_delegate.h:67
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    ./dist_proc/dax/workflow/****/streaming/windowing_api_delegate.h:67
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    ./dist_proc/dax/workflow/****/streaming/windowing_api_delegate.h:67
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    ./dist_proc/dax/workflow/****/streaming/windowing_api_delegate.h:67
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    ./dist_proc/dax/workflow/****/streaming/windowing_api_delegate.h:67
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/strea
Feb 19, 2022 4:03:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-19T16:03:35.307Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Feb 19, 2022 4:03:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-19T16:03:35.346Z: Worker pool stopped.
Feb 19, 2022 4:03:41 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2022-02-19_04_48_48-3087146533562638637 finished with status CANCELLED.
Load test results for test (ID): b0fe1192-e3fd-4359-a3cd-15e1bd23f94c and timestamp: 2022-02-19T12:48:42.648000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11390.007
dataflow_v2_java11_total_bytes_count             3.63530377E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerJavaImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220219124635
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:0845c54623ae950186c40c626e985c26e6f4208adc845ed37f746edda062b967
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220219124635]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:0845c54623ae950186c40c626e985c26e6f4208adc845ed37f746edda062b967]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220219124635] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:0845c54623ae950186c40c626e985c26e6f4208adc845ed37f746edda062b967])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:0845c54623ae950186c40c626e985c26e6f4208adc845ed37f746edda062b967
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:0845c54623ae950186c40c626e985c26e6f4208adc845ed37f746edda062b967
ERROR: (gcloud.container.images.delete) Not found: response: {'docker-distribution-api-version': 'registry/2.0', 'content-type': 'application/json', 'date': 'Sat, 19 Feb 2022 16:03:48 GMT', 'server': 'Docker Registry', 'cache-control': 'private', 'x-xss-protection': '0', 'x-frame-options': 'SAMEORIGIN', 'transfer-encoding': 'chunked', 'status': '404', 'content-length': '168', '-content-encoding': 'gzip'}
Failed to compute blob liveness for manifest: 'sha256:0845c54623ae950186c40c626e985c26e6f4208adc845ed37f746edda062b967': None

> Task :runners:google-cloud-dataflow-java:cleanUpDockerJavaImages FAILED

FAILURE: Build completed with 2 failures.

1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
==============================================================================

2: Task failed with an exception.
-----------
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src/runners/google-cloud-dataflow-java/build.gradle'> line: 297

* What went wrong:
Execution failed for task ':runners:google-cloud-dataflow-java:cleanUpDockerJavaImages'.
> Process 'command './scripts/cleanup_untagged_gcr_images.sh'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
==============================================================================

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings

Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.

BUILD FAILED in 3h 17m 28s
109 actionable tasks: 72 executed, 33 from cache, 4 up-to-date

Publishing build scan...
https://gradle.com/s/uzgnwtb3xlkeg

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #243

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/243/display/redirect?page=changes>

Changes:

[Kyle Weaver] [BEAM-12712] Spark: Exclude looping timer tests.

[Kyle Weaver] [BEAM-13919] Annotate PerKeyOrderingTest with UsesStatefulParDo.

[noreply] Update 2.36.0 blog post to mention ARM64 support

[stranniknm] [BEAM-13785] playground - enable scio sdk

[noreply] Minor: Disable checker framework in nightly snapshot (#16829)

[artur.khanin] Updated example link

[noreply] [BEAM-13860] Make `DoFn.infer_output_type` return element type (#16788)

[noreply] [BEAM-13894] Unit test utilities in the ptest package (#16830)

[Kenneth Knowles] Add test for processing time continuation trigger

[noreply] [BEAM-13922] [Coverage] Make boot.go more testable and add tests

[noreply] Exclude SpannerChangeStream IT from Dataflow V1 postcommit (#16851)

[noreply] [BEAM-13930] Address StateSpec consistency issue between Runner and Fn

[mattcasters] [BEAM-13854] Document casting trick for Avro value serializer in KafkaIO

[noreply] Merge pull request #16838 from [BEAM-13931] - make sure large rows cause

[noreply] Seznam Case Study (#16825)

[noreply] [Website] Apache Hop Case Study (#16824)

[noreply] [BEAM-13694] Force hadoop-hdfs-client in hadoopVersion tests for hdfs

[noreply] [Website] Ricardo - added case study feedback (#16807)

[noreply] Merge pull request #16735 from [BEAM-13827] - fix medium file size

[noreply] Merge pull request #16753 from [BEAM-13837] [Playground] show graph on


------------------------------------------
[...truncated 1.18 MB...]
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server
Feb 16, 2022 4:00:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-16T16:00:51.140Z: Cancel request is committed for workflow job: 2022-02-16_04_45_28-8394044303879218212.
Feb 16, 2022 4:00:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-16T16:00:51.179Z: Cleaning up.
Feb 16, 2022 4:00:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-16T16:00:51.249Z: Stopping **** pool...
Feb 16, 2022 4:00:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-16T16:00:51.291Z: Stopping **** pool...
Feb 16, 2022 4:03:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-16T16:03:10.773Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Feb 16, 2022 4:03:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-16T16:03:10.807Z: Worker pool stopped.
Feb 16, 2022 4:03:16 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2022-02-16_04_45_28-8394044303879218212 finished with status CANCELLED.
Load test results for test (ID): 8256999b-a4d6-417c-976c-89b173a2d457 and timestamp: 2022-02-16T12:45:22.430000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11578.599
dataflow_v2_java11_total_bytes_count             2.90091086E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerJavaImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220216124334
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:f13939611991a56d92a65b5c4f5ee7d772fbe526008063aefa99282e90c6ce59
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220216124334]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:f13939611991a56d92a65b5c4f5ee7d772fbe526008063aefa99282e90c6ce59]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220216124334] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:f13939611991a56d92a65b5c4f5ee7d772fbe526008063aefa99282e90c6ce59])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:f13939611991a56d92a65b5c4f5ee7d772fbe526008063aefa99282e90c6ce59
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:f13939611991a56d92a65b5c4f5ee7d772fbe526008063aefa99282e90c6ce59
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:f13939611991a56d92a65b5c4f5ee7d772fbe526008063aefa99282e90c6ce59].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings

Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.

BUILD FAILED in 3h 20m 2s
109 actionable tasks: 72 executed, 33 from cache, 4 up-to-date

Publishing build scan...
https://gradle.com/s/4aphwbydqjxc2

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #241

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/241/display/redirect?page=changes>

Changes:

[akustov] fix name project id from secreton scio deploy action

[alexander.zhuravlev] [BEAM-13775] Fixed bug with run button

[ihr] [BEAM-13836] Fix the answers placeholders locations in the Python katas

[noreply] Merge pull request #16703 from [BEAM-13804][Playground][Bugfix] Add

[noreply] Merge pull request #16611 from [BEAM-13712][Playground] Add graph for

[noreply] Merge pull request #16757 from [BEAM-13655] [Playground] Persist the


------------------------------------------
[...truncated 323.39 KB...]
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server
FATAL: command execution failed
java.io.IOException: Backing channel 'apache-beam-jenkins-4' is disconnected.
	at hudson.remoting.RemoteInvocationHandler.channelOrFail(RemoteInvocationHandler.java:216)
	at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:286)
	at com.sun.proxy.$Proxy135.isAlive(Unknown Source)
	at hudson.Launcher$RemoteLauncher$ProcImpl.isAlive(Launcher.java:1211)
	at hudson.Launcher$RemoteLauncher$ProcImpl.join(Launcher.java:1203)
	at hudson.Launcher$ProcStarter.join(Launcher.java:523)
	at hudson.plugins.gradle.Gradle.perform(Gradle.java:317)
	at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
	at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:803)
	at hudson.model.Build$BuildExecution.build(Build.java:197)
	at hudson.model.Build$BuildExecution.doRun(Build.java:163)
	at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:513)
	at hudson.model.Run.execute(Run.java:1906)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
	at hudson.model.ResourceController.execute(ResourceController.java:97)
	at hudson.model.Executor.run(Executor.java:429)
Caused by: java.io.IOException: Pipe closed after 0 cycles
	at org.apache.sshd.common.channel.ChannelPipedInputStream.read(ChannelPipedInputStream.java:118)
	at org.apache.sshd.common.channel.ChannelPipedInputStream.read(ChannelPipedInputStream.java:101)
	at hudson.remoting.FlightRecorderInputStream.read(FlightRecorderInputStream.java:93)
	at hudson.remoting.ChunkedInputStream.readHeader(ChunkedInputStream.java:74)
	at hudson.remoting.ChunkedInputStream.readUntilBreak(ChunkedInputStream.java:104)
	at hudson.remoting.ChunkedCommandTransport.readBlock(ChunkedCommandTransport.java:39)
	at hudson.remoting.AbstractSynchronousByteArrayCommandTransport.read(AbstractSynchronousByteArrayCommandTransport.java:34)
	at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:61)
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
ERROR: apache-beam-jenkins-4 is offline; cannot locate jdk_1.8_latest

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #242

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/242/display/redirect?page=changes>

Changes:

[Kyle Weaver] [BEAM-12712] Spark: Exclude looping timer tests.

[Kyle Weaver] [BEAM-13919] Annotate PerKeyOrderingTest with UsesStatefulParDo.

[noreply] Update 2.36.0 blog post to mention ARM64 support

[noreply] Minor: Disable checker framework in nightly snapshot (#16829)

[artur.khanin] Updated example link

[noreply] [BEAM-13860] Make `DoFn.infer_output_type` return element type (#16788)

[noreply] [BEAM-13894] Unit test utilities in the ptest package (#16830)

[Kenneth Knowles] Add test for processing time continuation trigger

[noreply] [BEAM-13922] [Coverage] Make boot.go more testable and add tests

[noreply] Exclude SpannerChangeStream IT from Dataflow V1 postcommit (#16851)

[noreply] [BEAM-13930] Address StateSpec consistency issue between Runner and Fn


------------------------------------------
[...truncated 1.06 MB...]
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    ./dist_proc/dax/workflow/****/streaming/windowing_api_delegate.h:67
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    ./dist_proc/dax/workflow/****/streaming/windowing_api_delegate.h:67
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    ./dist_proc/dax/workflow/****/streaming/windowing_api_delegate.h:67
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    ./dist_proc/dax/workflow/****/streaming/windowing_api_delegate.h:67
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    ./dist_proc/dax/workflow/****/streaming/windowing_api_delegate.h:67
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    ./dist_proc/dax/workflow/****/streaming/windowing_api_delegate.h:67
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    ./dist_proc/dax/workflow/****/streaming/windowing_api_delegate.h:67
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    ./dist_proc/dax/workflow/****/streaming/windowing_api_delegate.h:67
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    ./dist_proc/dax/workflow/****/streaming/windowing_api_delegate.h:67
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
FATAL: command execution failed
java.io.IOException: Backing channel 'apache-beam-jenkins-8' is disconnected.
	at hudson.remoting.RemoteInvocationHandler.channelOrFail(RemoteInvocationHandler.java:216)
	at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:286)
	at com.sun.proxy.$Proxy139.isAlive(Unknown Source)
	at hudson.Launcher$RemoteLauncher$ProcImpl.isAlive(Launcher.java:1211)
	at hudson.Launcher$RemoteLauncher$ProcImpl.join(Launcher.java:1203)
	at hudson.Launcher$ProcStarter.join(Launcher.java:523)
	at hudson.plugins.gradle.Gradle.perform(Gradle.java:317)
	at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
	at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:803)
	at hudson.model.Build$BuildExecution.build(Build.java:197)
	at hudson.model.Build$BuildExecution.doRun(Build.java:163)
	at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:513)
	at hudson.model.Run.execute(Run.java:1906)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
	at hudson.model.ResourceController.execute(ResourceController.java:97)
	at hudson.model.Executor.run(Executor.java:429)
Caused by: java.io.IOException: Pipe closed after 0 cycles
	at org.apache.sshd.common.channel.ChannelPipedInputStream.read(ChannelPipedInputStream.java:118)
	at org.apache.sshd.common.channel.ChannelPipedInputStream.read(ChannelPipedInputStream.java:101)
	at hudson.remoting.FlightRecorderInputStream.read(FlightRecorderInputStream.java:93)
	at hudson.remoting.ChunkedInputStream.readHeader(ChunkedInputStream.java:74)
	at hudson.remoting.ChunkedInputStream.readUntilBreak(ChunkedInputStream.java:104)
	at hudson.remoting.ChunkedCommandTransport.readBlock(ChunkedCommandTransport.java:39)
	at hudson.remoting.AbstractSynchronousByteArrayCommandTransport.read(AbstractSynchronousByteArrayCommandTransport.java:34)
	at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:61)
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
ERROR: apache-beam-jenkins-8 is offline; cannot locate jdk_1.8_latest

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #240

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/240/display/redirect?page=changes>

Changes:

[randomstep] [BEAM-9195] Bump org.testcontainers to 1.16.3


------------------------------------------
[...truncated 1.37 MB...]
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    ./dist_proc/dax/workflow/****/streaming/windowing_api_delegate.h:67
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    ./dist_proc/dax/workflow/****/streaming/windowing_api_delegate.h:67
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    ./dist_proc/dax/workflow/****/streaming/windowing_api_delegate.h:67
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    ./dist_proc/dax/workflow/****/streaming/windowing_api_delegate.h:67
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    ./dist_proc/dax/workflow/****/streaming/windowing_api_delegate.h:67
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    ./dist_proc/dax/workflow/****/streaming/windowing_api_delegate.h:67
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    ./dist_proc/dax/workflow/****/streaming/windowing_api_delegate.h:67
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passe
Feb 13, 2022 4:03:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-13T16:03:36.711Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Feb 13, 2022 4:03:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-13T16:03:36.772Z: Worker pool stopped.
Feb 13, 2022 4:03:44 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2022-02-13_04_45_29-16751563456799598367 finished with status CANCELLED.
Load test results for test (ID): a0c63359-f6d7-41a1-8822-bfafbefab0e2 and timestamp: 2022-02-13T12:45:24.086000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11570.501
dataflow_v2_java11_total_bytes_count             2.86910428E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerJavaImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220213124334
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:300c8c379ee0ed0c206591cafa4153221622ad1f4b2e630d136e5d5450998b1f
Deleted: sha256:0efaf401e4dfd45c8b6610e69abe5a32cd18e429fb3b7f240556e841f563a532
Deleted: sha256:01e5dd960b2a93b18f7ada1d70ec9164064bb8b7fbda1133010259154634412d
Deleted: sha256:04e6ceb2bf217fe3be73caac295c796fc3b9d7b90271acdf57b480ad6d55f00e
Deleted: sha256:c9269396ebcb4100532180e9d75704880f3f0140a2f3287e5161024fa8e3ef55
Deleted: sha256:76c9c071c9957f427e216522d3384612bf6b8502fe0df003f65bc7a015712d86
Deleted: sha256:d5dbc69339b77fe4dc5d404f4bb6207c16a016bf2929e23a05497639b68ceaf7
Deleted: sha256:5cb8bb9f69d384cbf9fd9d666cb8fd465919565f78216ede0d8371e7dea5cdeb
Deleted: sha256:a58537abc1af3d96aae3548f63fc142da1044f812647c7d1dfd46c0fe9d531f1
Deleted: sha256:93152afc2418154443dd9533105b918b1a17d3cab810dd7e310b557e066032af
Deleted: sha256:0a9615f262d8984f29d2eef4f8550785e0857528d30cab69014ce323c8ce4287
Deleted: sha256:e8072a51e81a2d17e25442a873510626379a0d3d7d48375df02a634f8313daf4
Deleted: sha256:7eed13aa46eee45114b44de0318d658d1b85625fd86bc0fe2114faf4834bc991
Deleted: sha256:923cac7c3bdbfc5ac53bed6f46eef1a46b9d8b8c3a730e94b203154f1df900a5
Deleted: sha256:efb875a7a7f0883126c5cbf7d6412bfba82c070104e1e7299ead13d92060dc28
Deleted: sha256:bf06d700b94112c0d896d12369e5d2ebefcc7d68447d2b56854adc2ebf3348ee
Deleted: sha256:28d3c70bf2ba013f14c3fdddf037a627d136ba36964ed5dc274727a8073d9896
Deleted: sha256:c8308f79a6b9f4fb781ef874e6b7a744ff4ec799afd089e819c380fdcea0e62d
Deleted: sha256:4447d4ae46e5241ecb14a886138def92d65711a1acebc3f089680017caccee6b
Deleted: sha256:61d53d8b46135cf6b2791881129560027d158c32097554118b673625a3088cf6
Deleted: sha256:e587c0c24f8c0e1b410b1dd9eaa20464a4d24cf589b77324abcb33c492d5e3dc
Deleted: sha256:ffcd66ed87283be0fc6fe081ec6a6b08bdbc91f2adaab2bd24c29dc32dc0276d
Deleted: sha256:546320789d12dc1cbd3ede0559f1f5a74fb49ad9c65b7b4b87b43c068008aad5
Deleted: sha256:e8db8686d1ae283643c94751f3d7a519cdd2efa8fb0951ba51e0bc4cb44141c1
Deleted: sha256:d8506ed0b04fcf9dba5d6c924d08fa68f0886da320a040263bc7882156c1ffcb
Deleted: sha256:0afcb583a9b0b79e5b726e68bd838b1c9eef66a811715e9b37983beb714d09d6
Deleted: sha256:1df4f5096ed24e1c1943b2e6e40ddf6776db90787a31f2f52a9085bc7b0a7ddb
Deleted: sha256:86041402d03c5cbdb14922e72b33e51f83291b9ca319142ff1b5eb98b8ec3275
Deleted: sha256:da813778abb691c2cc907faa4b9809c66b061ebbef58af9aa3273e44f556040d
Deleted: sha256:5366fe34eec601661fd760e5c729aed13e6b9f76a29799c42de7119b932f3e06
Deleted: sha256:fbedd8cd30751c134eda9ee4a9f50b35bbc592af444530fd60613aa2485598c7
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220213124334]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:300c8c379ee0ed0c206591cafa4153221622ad1f4b2e630d136e5d5450998b1f]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220213124334] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:300c8c379ee0ed0c206591cafa4153221622ad1f4b2e630d136e5d5450998b1f])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:300c8c379ee0ed0c206591cafa4153221622ad1f4b2e630d136e5d5450998b1f
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:300c8c379ee0ed0c206591cafa4153221622ad1f4b2e630d136e5d5450998b1f
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:300c8c379ee0ed0c206591cafa4153221622ad1f4b2e630d136e5d5450998b1f].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings

Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.

BUILD FAILED in 3h 20m 29s
109 actionable tasks: 72 executed, 33 from cache, 4 up-to-date

Publishing build scan...
https://gradle.com/s/znyxjhfqxgdrq

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #239

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/239/display/redirect?page=changes>

Changes:

[Carl Yeksigian] Cache bucket matcher regex in GcsPath

[benjamin.gonzalez] [BEAM-12672] Retry flaky tests

[benjamin.gonzalez] [BEAM-12672] Fix spotlessApply

[laraschmidt] Fixing the log line to properly handle a large allowed skew.

[n] BEAM-13159 Update embedded-redis dependency

[n] address comments

[noreply] Minor: Add 2.38.0 section to CHANGES.md (#16804)

[noreply] [BEAM-12000] Fix typo in portable Python job definition (#16812)

[noreply] [BEAM-12164]: Fixes SpannerChangeStreamIT (#16806)

[noreply] [BEAM-12572] Fix failures in python examples tests (#16781)

[noreply] [BEAM-13921] filter out debeziumIO test for spark runner (#16815)

[noreply] [BEAM-13855] Skip SpannerChangeStreamOrderedWithinKeyIT and

[noreply] [BEAM-13679] playground - move quick start category to the top (#16808)

[noreply] Update license_script.sh (#16789)

[noreply] [BEAM-13908] [Coverage] Better testing coverage for gcpopts (#16816)

[noreply] Merge pull request #16809 from [BEAM-12164] Added integration test for

[noreply] [BEAM-4032]Support staging binary distributions of dependency packages

[noreply] [BEAM-13834] Increase influxDB persistent storage. (#16817)

[noreply] Minor: Fix link to nexmark benchmarks (#16803)

[noreply] Regenerate python container base_image_requirements.txt (#16832)


------------------------------------------
[...truncated 948.86 KB...]
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server
Feb 12, 2022 4:03:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-12T16:03:27.649Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Feb 12, 2022 4:03:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-12T16:03:27.697Z: Worker pool stopped.
Feb 12, 2022 4:03:34 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2022-02-12_04_46_18-10455573273840819178 finished with status CANCELLED.
Load test results for test (ID): 23971631-0aa7-432a-b4f9-07099db99af6 and timestamp: 2022-02-12T12:46:12.894000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                  11519.96
dataflow_v2_java11_total_bytes_count             3.17940905E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerJavaImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220212124336
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:0ecd0439cbc9d1355e6b88471f3f8d594bf81564d33d33ad7a83291e0f93743e
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220212124336]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:0ecd0439cbc9d1355e6b88471f3f8d594bf81564d33d33ad7a83291e0f93743e]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220212124336] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:0ecd0439cbc9d1355e6b88471f3f8d594bf81564d33d33ad7a83291e0f93743e])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:0ecd0439cbc9d1355e6b88471f3f8d594bf81564d33d33ad7a83291e0f93743e
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:0ecd0439cbc9d1355e6b88471f3f8d594bf81564d33d33ad7a83291e0f93743e
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:0ecd0439cbc9d1355e6b88471f3f8d594bf81564d33d33ad7a83291e0f93743e].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:f340e1832ea427685bbba3c2dc9be10588ba8f60eff487e62c21f0f4547a1a58
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:f340e1832ea427685bbba3c2dc9be10588ba8f60eff487e62c21f0f4547a1a58
ERROR: (gcloud.container.images.delete) Not found: response: {'docker-distribution-api-version': 'registry/2.0', 'content-type': 'application/json', 'date': 'Sat, 12 Feb 2022 16:03:42 GMT', 'server': 'Docker Registry', 'cache-control': 'private', 'x-xss-protection': '0', 'x-frame-options': 'SAMEORIGIN', 'transfer-encoding': 'chunked', 'status': '404', 'content-length': '168', '-content-encoding': 'gzip'}
Failed to compute blob liveness for manifest: 'sha256:f340e1832ea427685bbba3c2dc9be10588ba8f60eff487e62c21f0f4547a1a58': None

> Task :runners:google-cloud-dataflow-java:cleanUpDockerJavaImages FAILED

FAILURE: Build completed with 2 failures.

1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
==============================================================================

2: Task failed with an exception.
-----------
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src/runners/google-cloud-dataflow-java/build.gradle'> line: 297

* What went wrong:
Execution failed for task ':runners:google-cloud-dataflow-java:cleanUpDockerJavaImages'.
> Process 'command './scripts/cleanup_untagged_gcr_images.sh'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
==============================================================================

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings

Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.

BUILD FAILED in 3h 20m 20s
109 actionable tasks: 72 executed, 33 from cache, 4 up-to-date

Publishing build scan...
https://gradle.com/s/bojdml7mqtlia

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #238

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/238/display/redirect?page=changes>

Changes:

[Carl Yeksigian] Cache bucket matcher regex in GcsPath

[benjamin.gonzalez] [BEAM-12672] Retry flaky tests

[benjamin.gonzalez] [BEAM-12672] Fix spotlessApply

[laraschmidt] Fixing the log line to properly handle a large allowed skew.

[n] BEAM-13159 Update embedded-redis dependency

[n] address comments

[noreply] Minor: Add 2.38.0 section to CHANGES.md (#16804)

[noreply] [BEAM-12000] Fix typo in portable Python job definition (#16812)

[noreply] [BEAM-12164]: Fixes SpannerChangeStreamIT (#16806)

[noreply] [BEAM-12572] Fix failures in python examples tests (#16781)

[noreply] [BEAM-13921] filter out debeziumIO test for spark runner (#16815)

[noreply] [BEAM-13855] Skip SpannerChangeStreamOrderedWithinKeyIT and

[noreply] [BEAM-13679] playground - move quick start category to the top (#16808)

[noreply] Update license_script.sh (#16789)

[noreply] [BEAM-13908] [Coverage] Better testing coverage for gcpopts (#16816)

[noreply] Merge pull request #16809 from [BEAM-12164] Added integration test for

[noreply] [BEAM-4032]Support staging binary distributions of dependency packages


------------------------------------------
[...truncated 63.70 KB...]
Feb 11, 2022 6:16:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-11T18:16:40.713Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Feb 11, 2022 6:16:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-11T18:16:40.744Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Feb 11, 2022 6:16:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-11T18:16:40.776Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Feb 11, 2022 6:16:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-11T18:16:40.810Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Feb 11, 2022 6:16:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-11T18:16:40.846Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Feb 11, 2022 6:16:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-11T18:16:40.881Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Feb 11, 2022 6:16:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-11T18:16:40.907Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Feb 11, 2022 6:16:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-11T18:16:40.935Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Feb 11, 2022 6:16:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-11T18:16:40.970Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Feb 11, 2022 6:16:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-11T18:16:41.005Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Feb 11, 2022 6:16:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-11T18:16:41.031Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Feb 11, 2022 6:16:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-11T18:16:41.054Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Feb 11, 2022 6:16:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-11T18:16:41.084Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Feb 11, 2022 6:16:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-11T18:16:41.118Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Feb 11, 2022 6:16:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-11T18:16:41.138Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Feb 11, 2022 6:16:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-11T18:16:41.165Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Feb 11, 2022 6:16:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-11T18:16:41.194Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Feb 11, 2022 6:16:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-11T18:16:41.228Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Feb 11, 2022 6:16:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-11T18:16:41.259Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Feb 11, 2022 6:16:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-11T18:16:41.285Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Feb 11, 2022 6:16:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-11T18:16:41.313Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Feb 11, 2022 6:16:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-11T18:16:41.340Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Feb 11, 2022 6:16:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-11T18:16:41.761Z: Starting 5 ****s in us-central1-b...
Feb 11, 2022 6:16:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-11T18:16:51.294Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Feb 11, 2022 6:17:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-11T18:17:28.020Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Feb 11, 2022 6:18:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-11T18:18:26.835Z: Workers have started successfully.
Feb 11, 2022 6:18:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-11T18:18:26.870Z: Workers have started successfully.
Feb 11, 2022 6:19:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-02-11T18:19:56.995Z: generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
Feb 11, 2022 6:19:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-02-11T18:19:57.060Z: generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
Feb 11, 2022 6:19:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-02-11T18:19:57.345Z: generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
Feb 11, 2022 6:19:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-02-11T18:19:58.170Z: generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
==>
    dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:214
Build timed out (after 240 minutes). Marking the build as aborted.
FATAL: command execution failed
hudson.remoting.ChannelClosedException: Channel "hudson.remoting.Channel@365b4851:apache-beam-jenkins-15": Remote call on apache-beam-jenkins-15 failed. The channel is closing down or has closed down
	at hudson.remoting.Channel.call(Channel.java:994)
	at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:286)
	at com.sun.proxy.$Proxy128.isAlive(Unknown Source)
	at hudson.Launcher$RemoteLauncher$ProcImpl.isAlive(Launcher.java:1211)
	at hudson.Launcher$RemoteLauncher$ProcImpl.join(Launcher.java:1203)
	at hudson.Launcher$ProcStarter.join(Launcher.java:523)
	at hudson.plugins.gradle.Gradle.perform(Gradle.java:317)
	at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
	at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:803)
	at hudson.model.Build$BuildExecution.build(Build.java:197)
	at hudson.model.Build$BuildExecution.doRun(Build.java:163)
	at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:513)
	at hudson.model.Run.execute(Run.java:1906)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
	at hudson.model.ResourceController.execute(ResourceController.java:97)
	at hudson.model.Executor.run(Executor.java:429)
Caused by: java.io.IOException
	at hudson.remoting.Channel.close(Channel.java:1499)
	at hudson.remoting.Channel.close(Channel.java:1455)
	at hudson.slaves.SlaveComputer.closeChannel(SlaveComputer.java:884)
	at hudson.slaves.SlaveComputer.access$100(SlaveComputer.java:110)
	at hudson.slaves.SlaveComputer$2.run(SlaveComputer.java:765)
	at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
	at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:68)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
ERROR: apache-beam-jenkins-15 is offline; cannot locate jdk_1.8_latest

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #237

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/237/display/redirect?page=changes>

Changes:

[david.prieto.rivera] Missing contribution

[noreply] [BEAM-13803] Add support for native iterable side inputs to the Go SDK

[noreply] [BEAM-11095] Better error handling for illegal emit functions (#16776)

[noreply] Merge pull request #16613 from Supporting JdbcIO driver in classpath for

[noreply] Merge pull request #15848 from [BEAM-13835] An any-type implementation

[Valentyn Tymofieiev] [BEAM-12920] Assume that bare generators types define simple generators.

[Valentyn Tymofieiev] Add a container for Python 3.9.

[Valentyn Tymofieiev] Allow job submission with Python 3.9 on Dataflow runner

[Valentyn Tymofieiev] Add Python 3.9 test suites. Keep Dataflow V1 suites unchanged for now.

[Valentyn Tymofieiev] Add py3.9 Github actions suites.

[Valentyn Tymofieiev] Py39 Doc updates.

[Valentyn Tymofieiev] [BEAM-9980] Simplify run_validates_container.sh to avoid branching.

[Valentyn Tymofieiev] Update Cython to a new version that has py39 wheels.

[Valentyn Tymofieiev] [BEAM-13845] Fix comparison with potentially incomparable default

[Valentyn Tymofieiev] [BEAM-12920] Assume that bare generators types define simple generators.

[Valentyn Tymofieiev] Mark Python 3.9 as supported version.

[noreply] [release-2.36.0][website] Fix github release notes script, header for

[noreply] Use shell to run python for setupVirtualenv (#16796)

[Daniel Oliveira] [BEAM-13830] Properly shut down Debezium expansion service in IT script.

[noreply] Merge pull request #16659 from [BEAM-13774][Playground] Add user to

[Valentyn Tymofieiev] [BEAM-13868] Remove gsutil dep from hdfs IT test.

[noreply] [BEAM-13776][Playground] (#16731)

[noreply] [BEAM-13867] Drop NaNs returned by nlargest in flight_delays example

[noreply] Announce Python 3.9 in CHANGES.md (#16802)

[Brian Hulette] Moving to 2.38.0-SNAPSHOT on master branch.

[noreply] [BEAM-11095] Better error handling for iter/reiter/multimap (#16794)


------------------------------------------
[...truncated 47.39 KB...]
57aec383ac7b: Pushed
b4164e5f025d: Pushed
535d88b6378e: Pushed
8dda956c1426: Pushed
75f72f6b56b5: Pushed
a0603f3a02d3: Pushed
f3e8e87a4b44: Pushed
a1445b7ad2a8: Pushed
d695f0110876: Pushed
efb3f834d1ce: Pushed
0aa3674558b5: Layer already exists
bf1de93fcdde: Pushed
7c072cee6a29: Layer already exists
1e5fdc3d671c: Layer already exists
bed676ceab7a: Layer already exists
613ab28cf833: Layer already exists
6398d5cccd2c: Layer already exists
0b0f2f2f5279: Layer already exists
edb67dc046f7: Pushed
dd2cb0231f4d: Pushed
20220210124333: digest: sha256:d70eb28f103e3451da3491cfddc45351966ba703159dbc8e38f8a3e6a9e05426 size: 4520

> Task :sdks:java:testing:load-tests:run
Feb 10, 2022 12:45:34 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Feb 10, 2022 12:45:34 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 204 files. Enable logging at DEBUG level to see which files will be staged.
Feb 10, 2022 12:45:35 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Feb 10, 2022 12:45:35 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Feb 10, 2022 12:45:37 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 204 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Feb 10, 2022 12:45:37 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 204 files cached, 0 files newly uploaded in 0 seconds
Feb 10, 2022 12:45:37 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Feb 10, 2022 12:45:37 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <114077 bytes, hash e6bd569948c953f638f65e779a71f8956f2f2cd1860191aaf2b885e327c45633> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-5r1WmUjJU_Y49l53mnH4lW8vLNGGAZGq8riF4yfEVjM.pb
Feb 10, 2022 12:45:39 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Feb 10, 2022 12:45:39 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3dd31157, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@31c628e7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3240b2a4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58434b19, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7d3fb0ef, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7dbe2ebf, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4adc663e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@885e7ff, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@8bd86c8, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4fa9ab6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2d3ef181, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@a2341c6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6e4c0d8c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3e3315d9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@64db4967, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@74e6094b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7a485a36, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5cf3157b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@625dfff3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26350ea2]
Feb 10, 2022 12:45:39 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Feb 10, 2022 12:45:39 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Feb 10, 2022 12:45:39 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Feb 10, 2022 12:45:39 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Feb 10, 2022 12:45:39 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@806996, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@78b612c6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@257e0827, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@22752544, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@21ba2445, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@69d23296, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c3820bb, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@376c7d7d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4784efd9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3fba233d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@427ae189, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@16a9eb2e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@76332405, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@187e5235, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@d1d8e1a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5434e40c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3b48e183, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@514de325, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@30c1da48, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43a65cd8]
Feb 10, 2022 12:45:39 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Feb 10, 2022 12:45:39 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Feb 10, 2022 12:45:39 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Feb 10, 2022 12:45:39 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Feb 10, 2022 12:45:39 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Feb 10, 2022 12:45:39 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Feb 10, 2022 12:45:39 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Feb 10, 2022 12:45:39 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Feb 10, 2022 12:45:39 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Feb 10, 2022 12:45:39 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Feb 10, 2022 12:45:39 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Feb 10, 2022 12:45:39 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.38.0-SNAPSHOT
Feb 10, 2022 12:45:40 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-02-10_04_45_39-2704250670955367490?project=apache-beam-testing
Feb 10, 2022 12:45:40 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2022-02-10_04_45_39-2704250670955367490
Feb 10, 2022 12:45:40 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2022-02-10_04_45_39-2704250670955367490
Feb 10, 2022 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2022-02-10T12:45:50.733Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-02-6o4n. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Feb 10, 2022 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-10T12:45:56.036Z: Worker configuration: e2-standard-2 in us-central1-b.
Feb 10, 2022 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-10T12:45:56.896Z: Expanding SplittableParDo operations into optimizable parts.
Feb 10, 2022 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-10T12:45:56.925Z: Expanding CollectionToSingleton operations into optimizable parts.
Feb 10, 2022 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-10T12:45:56.997Z: Expanding CoGroupByKey operations into optimizable parts.
Feb 10, 2022 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-10T12:45:57.066Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Feb 10, 2022 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-10T12:45:57.095Z: Expanding GroupByKey operations into streaming Read/Write steps
Feb 10, 2022 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-10T12:45:57.149Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Feb 10, 2022 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-10T12:45:57.251Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Feb 10, 2022 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-10T12:45:57.279Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Feb 10, 2022 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-10T12:45:57.306Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Feb 10, 2022 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-10T12:45:57.339Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Feb 10, 2022 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-10T12:45:57.373Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Feb 10, 2022 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-10T12:45:57.409Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Feb 10, 2022 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-10T12:45:57.431Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Feb 10, 2022 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-10T12:45:57.477Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Feb 10, 2022 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-10T12:45:57.510Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Feb 10, 2022 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-10T12:45:57.542Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Feb 10, 2022 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-10T12:45:57.579Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Feb 10, 2022 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-10T12:45:57.614Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Feb 10, 2022 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-10T12:45:57.648Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Feb 10, 2022 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-10T12:45:57.669Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Feb 10, 2022 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-10T12:45:57.694Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Feb 10, 2022 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-10T12:45:57.726Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Feb 10, 2022 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-10T12:45:57.758Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Feb 10, 2022 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-10T12:45:57.803Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Feb 10, 2022 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-10T12:45:57.827Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Feb 10, 2022 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-10T12:45:57.862Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Feb 10, 2022 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-10T12:45:57.887Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Feb 10, 2022 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-10T12:45:57.907Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Feb 10, 2022 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-10T12:45:57.935Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Feb 10, 2022 12:46:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-10T12:45:58.287Z: Starting 5 ****s in us-central1-b...
Feb 10, 2022 12:46:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-10T12:46:02.315Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Feb 10, 2022 12:46:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-10T12:46:38.629Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Feb 10, 2022 12:47:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-10T12:47:40.557Z: Workers have started successfully.
Feb 10, 2022 12:47:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-10T12:47:40.593Z: Workers have started successfully.
Feb 10, 2022 4:00:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-10T16:00:36.045Z: Cancel request is committed for workflow job: 2022-02-10_04_45_39-2704250670955367490.
Feb 10, 2022 4:00:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-10T16:00:36.110Z: Cleaning up.
Feb 10, 2022 4:00:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-10T16:00:36.186Z: Stopping **** pool...
Feb 10, 2022 4:00:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-10T16:00:36.239Z: Stopping **** pool...
Feb 10, 2022 4:02:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-10T16:02:55.035Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Feb 10, 2022 4:02:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-10T16:02:55.077Z: Worker pool stopped.
Feb 10, 2022 4:03:03 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2022-02-10_04_45_39-2704250670955367490 finished with status CANCELLED.
Load test results for test (ID): 11464781-aed8-45cf-91b2-a0e767eaa5bb and timestamp: 2022-02-10T12:45:34.858000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11526.713
dataflow_v2_java11_total_bytes_count             3.00254704E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerJavaImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220210124333
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:d70eb28f103e3451da3491cfddc45351966ba703159dbc8e38f8a3e6a9e05426
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220210124333]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:d70eb28f103e3451da3491cfddc45351966ba703159dbc8e38f8a3e6a9e05426]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220210124333] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:d70eb28f103e3451da3491cfddc45351966ba703159dbc8e38f8a3e6a9e05426])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:d70eb28f103e3451da3491cfddc45351966ba703159dbc8e38f8a3e6a9e05426
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:d70eb28f103e3451da3491cfddc45351966ba703159dbc8e38f8a3e6a9e05426
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:d70eb28f103e3451da3491cfddc45351966ba703159dbc8e38f8a3e6a9e05426].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:9ad812c02dd245785430c732fa9769e7eaac5982fbeb695b53b4928449dfd98f
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:9ad812c02dd245785430c732fa9769e7eaac5982fbeb695b53b4928449dfd98f
ERROR: (gcloud.container.images.delete) Not found: response: {'docker-distribution-api-version': 'registry/2.0', 'content-type': 'application/json', 'date': 'Thu, 10 Feb 2022 16:03:13 GMT', 'server': 'Docker Registry', 'cache-control': 'private', 'x-xss-protection': '0', 'x-frame-options': 'SAMEORIGIN', 'transfer-encoding': 'chunked', 'status': '404', 'content-length': '168', '-content-encoding': 'gzip'}
Failed to compute blob liveness for manifest: 'sha256:9ad812c02dd245785430c732fa9769e7eaac5982fbeb695b53b4928449dfd98f': None

> Task :runners:google-cloud-dataflow-java:cleanUpDockerJavaImages FAILED

FAILURE: Build completed with 2 failures.

1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
==============================================================================

2: Task failed with an exception.
-----------
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src/runners/google-cloud-dataflow-java/build.gradle'> line: 297

* What went wrong:
Execution failed for task ':runners:google-cloud-dataflow-java:cleanUpDockerJavaImages'.
> Process 'command './scripts/cleanup_untagged_gcr_images.sh'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
==============================================================================

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings

Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.

BUILD FAILED in 3h 19m 53s
109 actionable tasks: 74 executed, 31 from cache, 4 up-to-date

Publishing build scan...
https://gradle.com/s/2ogl5mpme33i4

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #236

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/236/display/redirect?page=changes>

Changes:

[noreply] Update README.md

[marco.robles] Update README with latest PreCommit Jobs

[marco.robles] Update Postcommit jobs with latest jobs

[marco.robles] Update Performace job tests in readme

[marco.robles] update load job tests with latest updates

[marco.robles] update other jobs test with latest updates

[marco.robles] mismatch links fix

[marco.robles] update trigger phrase for some postCommit jobs

[marco.robles] correct trigger phrases in readme

[marco.robles] add pending jobs to readme

[noreply] Update README.md

[mmack] [BEAM-13246] Add support for S3 Bucket Key at the object level (AWS Sdk

[Pablo Estrada] Output successful rows from BQ Streaming Inserts

[schapman] BEAM-13439 Type annotation for ptransform_fn

[noreply] [BEAM-13606] Fail bundles with failed BigTable mutations (#16751)

[mmack] [adhoc] Remove remaining usage of Powermock from aws2.

[marco.robles] fix broken links in jobs & remove the invalid ones

[Kyle Weaver] Update Dataflow Python dev container images.

[Kiley Sok] Add java 17 to changes

[noreply] [BEAM-12914] Add missing 3.9 opcodes to type inference. (#16761)

[noreply] [BEAM-13321] Initial BigQueryIO externalization. (#16489)

[noreply] [BEAM-13193] Enable process bundle response elements embedding in Java

[noreply] [BEAM-13830] added a debeziumio_expansion_addr flag to GoSDK (#16780)

[noreply] Apply spotless. (#16783)

[Daniel Oliveira] [BEAM-13732] Switch x-lang BigQueryIO expansion service to GCP one.

[noreply] [BEAM-13858] Fix broken github action on :sdks:go:examples:wordCount

[Kiley Sok] add jira for runner v2

[noreply] [BEAM-13732] Go SDK BigQuery IO wrapper. Initial implementation.

[noreply] [BEAM-13732] Add example for Go BigQuery IO wrapper. (#16786)

[noreply] Update CHANGES.md with Go SDK milestones. (#16787)

[noreply] [BEAM-13193] Allow BeamFnDataOutboundObserver to flush elements.


------------------------------------------
[...truncated 49.28 KB...]
58c06f1c539d: Preparing
1d91539a970d: Preparing
67ac77c9cbe8: Preparing
2607d1e76597: Preparing
0aa3674558b5: Preparing
7c072cee6a29: Preparing
1e5fdc3d671c: Preparing
613ab28cf833: Preparing
bed676ceab7a: Preparing
d77b999a6aa0: Waiting
6398d5cccd2c: Preparing
0b0f2f2f5279: Preparing
5fc56bb17504: Waiting
0aa3674558b5: Waiting
1d91539a970d: Waiting
2607d1e76597: Waiting
7c072cee6a29: Waiting
67ac77c9cbe8: Waiting
a37adf383347: Waiting
869cf694e13b: Waiting
613ab28cf833: Waiting
0b0f2f2f5279: Waiting
6398d5cccd2c: Waiting
58c06f1c539d: Waiting
bed676ceab7a: Waiting
e66cec84260a: Pushed
30502d728975: Pushed
378d28703ba4: Pushed
038dbb835d07: Pushed
5fc56bb17504: Pushed
214a57736d20: Pushed
a37adf383347: Pushed
67ac77c9cbe8: Pushed
58c06f1c539d: Pushed
0aa3674558b5: Layer already exists
d77b999a6aa0: Pushed
869cf694e13b: Pushed
7c072cee6a29: Layer already exists
613ab28cf833: Layer already exists
1e5fdc3d671c: Layer already exists
bed676ceab7a: Layer already exists
0b0f2f2f5279: Layer already exists
6398d5cccd2c: Layer already exists
2607d1e76597: Pushed
1d91539a970d: Pushed
20220209124334: digest: sha256:223356628f1d4c13d8950fd4844c957a25297f6233684232482b34529bf6676f size: 4520

> Task :sdks:java:testing:load-tests:run
Feb 09, 2022 12:45:22 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Feb 09, 2022 12:45:22 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 204 files. Enable logging at DEBUG level to see which files will be staged.
Feb 09, 2022 12:45:23 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Feb 09, 2022 12:45:23 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Feb 09, 2022 12:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 204 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Feb 09, 2022 12:45:26 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 204 files cached, 0 files newly uploaded in 1 seconds
Feb 09, 2022 12:45:26 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Feb 09, 2022 12:45:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <114077 bytes, hash dcaa666cf308bd25bd7794484787973f598ff3e67f5cab79a9e4a1ad5b859154> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-3KpmbPMIvSW9d5RIR4eXP1mP8-Z_XKt5qeShrVuFkVQ.pb
Feb 09, 2022 12:45:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Feb 09, 2022 12:45:28 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3dd31157, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@31c628e7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3240b2a4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58434b19, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7d3fb0ef, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7dbe2ebf, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4adc663e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@885e7ff, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@8bd86c8, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4fa9ab6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2d3ef181, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@a2341c6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6e4c0d8c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3e3315d9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@64db4967, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@74e6094b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7a485a36, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5cf3157b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@625dfff3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26350ea2]
Feb 09, 2022 12:45:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Feb 09, 2022 12:45:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Feb 09, 2022 12:45:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Feb 09, 2022 12:45:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Feb 09, 2022 12:45:28 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@806996, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@78b612c6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@257e0827, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@22752544, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@21ba2445, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@69d23296, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c3820bb, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@376c7d7d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4784efd9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3fba233d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@427ae189, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@16a9eb2e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@76332405, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@187e5235, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@d1d8e1a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5434e40c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3b48e183, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@514de325, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@30c1da48, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43a65cd8]
Feb 09, 2022 12:45:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Feb 09, 2022 12:45:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Feb 09, 2022 12:45:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Feb 09, 2022 12:45:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Feb 09, 2022 12:45:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Feb 09, 2022 12:45:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Feb 09, 2022 12:45:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Feb 09, 2022 12:45:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Feb 09, 2022 12:45:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Feb 09, 2022 12:45:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Feb 09, 2022 12:45:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Feb 09, 2022 12:45:28 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.37.0-SNAPSHOT
Feb 09, 2022 12:45:31 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-02-09_04_45_28-12291619623864716731?project=apache-beam-testing
Feb 09, 2022 12:45:31 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2022-02-09_04_45_28-12291619623864716731
Feb 09, 2022 12:45:31 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2022-02-09_04_45_28-12291619623864716731
Feb 09, 2022 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2022-02-09T12:45:38.543Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-02-mkes. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Feb 09, 2022 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-09T12:45:44.030Z: Worker configuration: e2-standard-2 in us-central1-b.
Feb 09, 2022 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-09T12:45:44.706Z: Expanding SplittableParDo operations into optimizable parts.
Feb 09, 2022 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-09T12:45:44.744Z: Expanding CollectionToSingleton operations into optimizable parts.
Feb 09, 2022 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-09T12:45:44.810Z: Expanding CoGroupByKey operations into optimizable parts.
Feb 09, 2022 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-09T12:45:44.873Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Feb 09, 2022 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-09T12:45:44.894Z: Expanding GroupByKey operations into streaming Read/Write steps
Feb 09, 2022 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-09T12:45:44.964Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Feb 09, 2022 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-09T12:45:45.070Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Feb 09, 2022 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-09T12:45:45.100Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Feb 09, 2022 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-09T12:45:45.136Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Feb 09, 2022 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-09T12:45:45.165Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Feb 09, 2022 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-09T12:45:45.190Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Feb 09, 2022 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-09T12:45:45.267Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Feb 09, 2022 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-09T12:45:45.299Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Feb 09, 2022 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-09T12:45:45.331Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Feb 09, 2022 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-09T12:45:45.361Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Feb 09, 2022 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-09T12:45:45.388Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Feb 09, 2022 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-09T12:45:45.419Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Feb 09, 2022 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-09T12:45:45.464Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Feb 09, 2022 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-09T12:45:45.523Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Feb 09, 2022 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-09T12:45:45.567Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Feb 09, 2022 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-09T12:45:45.606Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Feb 09, 2022 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-09T12:45:45.628Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Feb 09, 2022 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-09T12:45:45.665Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Feb 09, 2022 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-09T12:45:45.698Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Feb 09, 2022 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-09T12:45:45.732Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Feb 09, 2022 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-09T12:45:45.755Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Feb 09, 2022 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-09T12:45:45.797Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Feb 09, 2022 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-09T12:45:45.839Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Feb 09, 2022 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-09T12:45:45.878Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Feb 09, 2022 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-09T12:45:46.295Z: Starting 5 ****s in us-central1-b...
Feb 09, 2022 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-09T12:45:53.961Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Feb 09, 2022 12:46:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-09T12:46:28.357Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Feb 09, 2022 12:47:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-09T12:47:31.571Z: Workers have started successfully.
Feb 09, 2022 12:47:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-09T12:47:31.602Z: Workers have started successfully.
Feb 09, 2022 4:00:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-09T16:00:41.127Z: Cancel request is committed for workflow job: 2022-02-09_04_45_28-12291619623864716731.
Feb 09, 2022 4:00:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-09T16:00:41.214Z: Cleaning up.
Feb 09, 2022 4:00:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-09T16:00:41.289Z: Stopping **** pool...
Feb 09, 2022 4:00:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-09T16:00:41.336Z: Stopping **** pool...
Feb 09, 2022 4:03:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-09T16:03:07.241Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Feb 09, 2022 4:03:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-09T16:03:07.272Z: Worker pool stopped.
Feb 09, 2022 4:03:13 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2022-02-09_04_45_28-12291619623864716731 finished with status CANCELLED.
Load test results for test (ID): 10e3dad5-0b89-41b6-b29a-e409ab737c1e and timestamp: 2022-02-09T12:45:22.814000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11560.618
dataflow_v2_java11_total_bytes_count             2.20859091E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerJavaImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220209124334
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:223356628f1d4c13d8950fd4844c957a25297f6233684232482b34529bf6676f
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220209124334]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:223356628f1d4c13d8950fd4844c957a25297f6233684232482b34529bf6676f]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220209124334] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:223356628f1d4c13d8950fd4844c957a25297f6233684232482b34529bf6676f])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:223356628f1d4c13d8950fd4844c957a25297f6233684232482b34529bf6676f
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:223356628f1d4c13d8950fd4844c957a25297f6233684232482b34529bf6676f
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:223356628f1d4c13d8950fd4844c957a25297f6233684232482b34529bf6676f].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings

Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.

BUILD FAILED in 3h 19m 59s
109 actionable tasks: 72 executed, 33 from cache, 4 up-to-date

Publishing build scan...
https://gradle.com/s/2nwxap4lptptq

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #235

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/235/display/redirect?page=changes>

Changes:

[Kyle Weaver] [BEAM-12976] Log projection pushdown optimizations.

[benjamin.gonzalez] [BEAM-12572] Change jobs to run as cron jobs

[Ismaël Mejía] [BEAM-13839] Upgrade zstd-jni to version 1.5.2-1

[mmack] [BEAM-13840] Fix usage of legacy rawtypes in AWS modules

[alexander.zhuravlev] [BEAM-13820] Changed color of delete icon in pipeline options dropdown,

[noreply] [BEAM-11971] Revert "Fix timer consistency in direct runner" (#16748)

[noreply] [BEAM-13193] Aggregates fn api outbound data/timers of different

[noreply] [BEAM-13767] Migrate a bundle of grade tasks to use configuration

[noreply] Merge pull request #16653 from [BEAM-12164]: Add integration tests for

[noreply] Merge pull request #16728 from [BEAM-13823] Update docs for SnowflakeIO

[noreply] Merge pull request #16660 from [BEAM-13771][Playground] Send multifile

[noreply] Merge pull request #16646 from [BEAM-13643][Playground] Setup running

[noreply] [BEAM-13015] Add state caching benchmark and move benchmarks to their

[noreply] [BEAM-13419] Check for initialization in dataflow runner (#16765)

[noreply] Merge pull request #16701 from [BEAM-13786] [Playground] [Bugfix] Update

[noreply] Merge pull request #16754 from [BEAM-13838][Playground] Add logs in case

[noreply] [BEAM-13293] consistent naming for expansion service address and flag

[noreply] Merge pull request #16700 from [BEAM-13790][Playground] Change logic of

[noreply] [BEAM-13830] update dependency for debeziumio expansion service (#16743)

[noreply] [BEAM-13761] consistent namings for expansion address in Debezium IO

[noreply] [BEAM-13806] Shutting down SchemaIO expansion services from Go VR

[noreply] [release-2.36.0] Update website/changelog for release 2.36.0 (#16627)

[noreply] [BEAM-13848] Update numpy intersphinx link (#16767)

[noreply] [release-23.6.0] Fix JIRA link for 2.36 blog (#16771)

[noreply] [BEAM-13647] Use role for Go worker binary. (#16729)


------------------------------------------
[...truncated 49.38 KB...]
0aa3674558b5: Layer already exists
7c072cee6a29: Layer already exists
1e5fdc3d671c: Layer already exists
613ab28cf833: Layer already exists
bed676ceab7a: Layer already exists
6398d5cccd2c: Layer already exists
0b0f2f2f5279: Layer already exists
3f5fa4e217b9: Pushed
79e4afa916f0: Pushed
d0a88de6d715: Pushed
d95d966c943e: Pushed
20220208125609: digest: sha256:de0d91a58578225b2f7d9c60b619a573956dfb912c73dec95c2f349d7da1826a size: 4520

> Task :sdks:java:testing:load-tests:run
Feb 08, 2022 12:59:53 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Feb 08, 2022 12:59:53 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 204 files. Enable logging at DEBUG level to see which files will be staged.
Feb 08, 2022 12:59:55 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Feb 08, 2022 12:59:55 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Feb 08, 2022 12:59:59 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 204 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Feb 08, 2022 1:00:00 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 204 files cached, 0 files newly uploaded in 1 seconds
Feb 08, 2022 1:00:00 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Feb 08, 2022 1:00:01 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <114077 bytes, hash 91e84a6f8b002deb2d1f044b1a9892dc5e43e3665547133f439fbe112505c373> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-kehKb4sALestHwRLGpiS3F5D42ZVRxM_Q5--ESUFw3M.pb
Feb 08, 2022 1:00:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Feb 08, 2022 1:00:03 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3240b2a4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58434b19, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7d3fb0ef, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7dbe2ebf, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4adc663e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@885e7ff, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@8bd86c8, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4fa9ab6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2d3ef181, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@a2341c6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6e4c0d8c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3e3315d9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@64db4967, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@74e6094b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7a485a36, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5cf3157b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@625dfff3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26350ea2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1e9469b8, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5a08efdc]
Feb 08, 2022 1:00:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Feb 08, 2022 1:00:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Feb 08, 2022 1:00:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Feb 08, 2022 1:00:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Feb 08, 2022 1:00:03 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@257e0827, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@22752544, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@21ba2445, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@69d23296, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c3820bb, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@376c7d7d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4784efd9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3fba233d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@427ae189, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@16a9eb2e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@76332405, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@187e5235, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@d1d8e1a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5434e40c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3b48e183, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@514de325, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@30c1da48, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43a65cd8, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3f1ef9d6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@17461db]
Feb 08, 2022 1:00:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Feb 08, 2022 1:00:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Feb 08, 2022 1:00:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Feb 08, 2022 1:00:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Feb 08, 2022 1:00:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Feb 08, 2022 1:00:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Feb 08, 2022 1:00:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Feb 08, 2022 1:00:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Feb 08, 2022 1:00:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Feb 08, 2022 1:00:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Feb 08, 2022 1:00:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Feb 08, 2022 1:00:03 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.37.0-SNAPSHOT
Feb 08, 2022 1:00:04 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-02-08_05_00_03-14329417766500327917?project=apache-beam-testing
Feb 08, 2022 1:00:04 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2022-02-08_05_00_03-14329417766500327917
Feb 08, 2022 1:00:04 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2022-02-08_05_00_03-14329417766500327917
Feb 08, 2022 1:00:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2022-02-08T13:00:11.656Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-02-fq8i. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Feb 08, 2022 1:00:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-08T13:00:16.190Z: Worker configuration: e2-standard-2 in us-central1-b.
Feb 08, 2022 1:00:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-08T13:00:16.905Z: Expanding SplittableParDo operations into optimizable parts.
Feb 08, 2022 1:00:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-08T13:00:16.937Z: Expanding CollectionToSingleton operations into optimizable parts.
Feb 08, 2022 1:00:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-08T13:00:17.010Z: Expanding CoGroupByKey operations into optimizable parts.
Feb 08, 2022 1:00:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-08T13:00:17.079Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Feb 08, 2022 1:00:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-08T13:00:17.106Z: Expanding GroupByKey operations into streaming Read/Write steps
Feb 08, 2022 1:00:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-08T13:00:17.165Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Feb 08, 2022 1:00:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-08T13:00:17.390Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Feb 08, 2022 1:00:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-08T13:00:17.454Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Feb 08, 2022 1:00:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-08T13:00:17.496Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Feb 08, 2022 1:00:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-08T13:00:17.530Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Feb 08, 2022 1:00:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-08T13:00:17.565Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Feb 08, 2022 1:00:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-08T13:00:17.600Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Feb 08, 2022 1:00:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-08T13:00:17.636Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Feb 08, 2022 1:00:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-08T13:00:17.681Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Feb 08, 2022 1:00:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-08T13:00:17.720Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Feb 08, 2022 1:00:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-08T13:00:17.751Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Feb 08, 2022 1:00:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-08T13:00:17.783Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Feb 08, 2022 1:00:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-08T13:00:17.816Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Feb 08, 2022 1:00:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-08T13:00:17.850Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Feb 08, 2022 1:00:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-08T13:00:17.880Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Feb 08, 2022 1:00:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-08T13:00:17.913Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Feb 08, 2022 1:00:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-08T13:00:17.943Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Feb 08, 2022 1:00:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-08T13:00:17.978Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Feb 08, 2022 1:00:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-08T13:00:18.011Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Feb 08, 2022 1:00:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-08T13:00:18.059Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Feb 08, 2022 1:00:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-08T13:00:18.110Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Feb 08, 2022 1:00:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-08T13:00:18.133Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Feb 08, 2022 1:00:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-08T13:00:18.172Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Feb 08, 2022 1:00:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-08T13:00:18.204Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Feb 08, 2022 1:00:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-08T13:00:18.567Z: Starting 5 ****s in us-central1-b...
Feb 08, 2022 1:00:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-08T13:00:50.250Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Feb 08, 2022 1:00:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-08T13:00:58.016Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Feb 08, 2022 1:01:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-08T13:01:58.443Z: Workers have started successfully.
Feb 08, 2022 1:01:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-08T13:01:58.523Z: Workers have started successfully.
Feb 08, 2022 4:00:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-08T16:00:43.652Z: Cancel request is committed for workflow job: 2022-02-08_05_00_03-14329417766500327917.
Feb 08, 2022 4:00:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-08T16:00:43.754Z: Cleaning up.
Feb 08, 2022 4:00:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-08T16:00:43.812Z: Stopping **** pool...
Feb 08, 2022 4:00:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-08T16:00:43.853Z: Stopping **** pool...
Feb 08, 2022 4:03:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-08T16:03:03.218Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Feb 08, 2022 4:03:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-08T16:03:03.257Z: Worker pool stopped.
Feb 08, 2022 4:03:09 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2022-02-08_05_00_03-14329417766500327917 finished with status CANCELLED.
Load test results for test (ID): 98f52b9f-7fc8-43e3-afd4-0eb01ceb8593 and timestamp: 2022-02-08T12:59:54.393000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 10693.906
dataflow_v2_java11_total_bytes_count             1.46058293E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerJavaImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220208125609
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:de0d91a58578225b2f7d9c60b619a573956dfb912c73dec95c2f349d7da1826a
Deleted: sha256:9966169d97db3b794e3d00b2b4c4052e8f20d6fb790b6379b22db2daf94df1db
Deleted: sha256:3a4bf4ac56d605e7e35f46073c4adcdf0d0f1447cb1cd7c317f0f2f5bd3d81f0
Deleted: sha256:2761c5c5bc2905f7874125019b6f9d800bc1ea901f98db23974e969234688ff3
Deleted: sha256:0eb90dcde92badd20aefce84345e738e4b495440653e1932eb81386e3629cf23
Deleted: sha256:baad742a2ce89bcc1f0d172c089d25d4f4296b2c7cb07371e627920a2783b14c
Deleted: sha256:ad2cad60ef842f597982851df6f4e5671b98aff7b2a11b390fee7799a0dc4db5
Deleted: sha256:76f9c8a34f7b68a18e0b47a0a671c62b4765cc6eb4c19abec8d6f3266d7e3cbc
Deleted: sha256:17c1408e765a1791cac7127b68c3f81f6df369887b8be29e39ffc7477f187a93
Deleted: sha256:b67f7a26d1bffdefa895fad0f6fc5cb69338f499da0ba2973869c0b5faa9a554
Deleted: sha256:c9a917211b827ec2e069c5d3c63e8a761617d148482d17f1622120d65d2de059
Deleted: sha256:7010b557bead148ba239d672c9cf37bb8e257c3ccbf0133a140f7c9aa57a05bd
Deleted: sha256:5ed1065b58c77e7a4d5deff7ee9944e655de4f52ab3b5ec5a480c8b6eaa851ea
Deleted: sha256:6acef123fc6956883728d2fc3be46cb1c5fa146efc8e221eb0fe1cf2525d4156
Deleted: sha256:2a39641026428cb92e175272450ce9e59475b2fc04243c84879cbbcc1741a19e
Deleted: sha256:7fc5d389490a742166f166980214c363b004e94eed53988e786fc53324ff03fd
Deleted: sha256:8ae52dab523527bd7e4435dbc1125db6a18eefc11f28eb0493098bdcd926a4cf
Deleted: sha256:e16a3ee959c56bf6c48ca4c33362ab3989829d2985cff762f934cbbcc9527ee0
Deleted: sha256:6db5826c29e293d615feb426ec25c629f2c4b15a6a75efd7bcc38950c9eb87a2
Deleted: sha256:dbeaa5bbd78d2d7cd7a146b230edfd6094a79266abcce02e32bdbcd8591e6c78
Deleted: sha256:218be1b3302c00a7d2f27bea0cb5ace01ef14ec3e33a61588573bf3c25aa52f0
Deleted: sha256:21f2880dc0e11796b416621158f54de5e19bbd5c5501e6883e6dda616838d535
Deleted: sha256:777905852eaa0fc0a0b26ee69d783c2fe759510f9c01b1ba9632fa37e748a5e3
Deleted: sha256:6202ca84ef19abdd5d9b8bc272216b887892095bfe1d13959383bcb98595d770
Deleted: sha256:bb195afe5443b3e7b773c8cda77cdbadaf75861822f4370614842c40b9215660
Deleted: sha256:139a03ee025a9ff0e1bc4e64f9ba9e22c8c3a7e22e9b2f01adbb57c7f119e96c
Deleted: sha256:7da729d1b31e582ef2c20b6b30e10464921db83e3c7fd2ba5db65174f2c18f07
Deleted: sha256:7a306a9a8889a6528461b65cbde8b8aea5c89e3998fd78fdfea97df5b4efafd5
Deleted: sha256:291ff9501a3ca7f30ca57867af10b2a3f9469aca237d29ccd593539fd4f7730d
Deleted: sha256:d05d2b606c8c876a43ac272fb618a39feb5a426be84900b94f31cef7289764c2
Deleted: sha256:5639912a3f0df986498f5aa9d83715487c92f7b975fdd655bccd28e496cfaad9
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220208125609]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:de0d91a58578225b2f7d9c60b619a573956dfb912c73dec95c2f349d7da1826a]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220208125609] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:de0d91a58578225b2f7d9c60b619a573956dfb912c73dec95c2f349d7da1826a])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:de0d91a58578225b2f7d9c60b619a573956dfb912c73dec95c2f349d7da1826a
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:de0d91a58578225b2f7d9c60b619a573956dfb912c73dec95c2f349d7da1826a
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:de0d91a58578225b2f7d9c60b619a573956dfb912c73dec95c2f349d7da1826a].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:30c96bbc415702dedc0923f914f0ac2a7fda1f51fb0c397ee3c621eaadc6341c
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:30c96bbc415702dedc0923f914f0ac2a7fda1f51fb0c397ee3c621eaadc6341c
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:30c96bbc415702dedc0923f914f0ac2a7fda1f51fb0c397ee3c621eaadc6341c].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings

Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.

BUILD FAILED in 3h 7m 42s
109 actionable tasks: 72 executed, 33 from cache, 4 up-to-date

Publishing build scan...
https://gradle.com/s/op23ssz7lggey

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 - Build # 234 - Aborted!

Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 - Build # 234 - Aborted:

Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/234/ to view the results.

Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #233

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/233/display/redirect?page=changes>

Changes:

[mmack] [BEAM-13663] Remove unused duplicate option for AWS client configuration

[mmack] [BEAM-13203] Deprecate SnsIO.writeAsync for AWS Sdk v2 due to risk of

[noreply] [BEAM-13828] Fix stale bot (#16734)

[noreply] Merge pull request #16364 from [BEAM-13182]  Add diagrams to backend

[noreply] [BEAM-13811] Fix save_main_session arg in tests examples (#16709)

[Kiley Sok] Update beam-master version

[noreply] [BEAM-13015] Calculate exception for closing BeamFnDataInboundObserver2

[noreply] Minor doc tweaks for validating vendoring. (#16747)

[noreply] [BEAM-13686] OOM while logging a large pipeline even when logging level

[noreply] [BEAM-13629] Update URL artifact type for Dataflow Go (#16490)

[noreply] [BEAM-13832] Add automated expansion service start-up to JDBCio (#16739)

[noreply] [BEAM-13831] Add automated expansion service infra into Debezium Read()

[noreply] [BEAM-13821] Add automated expansion service start-up to KafkaIO

[noreply] [BEAM-13799] Created a Dataproc cluster manager for Interactive Beam

[noreply] Merge pull request #16727: [BEAM-11971] remove unsafe Concurrent data


------------------------------------------
[...truncated 49.78 KB...]
bd18a1a09476: Preparing
6f78efdc0a6b: Preparing
352660b137e6: Preparing
4590f8c89770: Preparing
ca33502d2cac: Preparing
0aa3674558b5: Preparing
7c072cee6a29: Preparing
1e5fdc3d671c: Preparing
613ab28cf833: Preparing
bed676ceab7a: Preparing
6398d5cccd2c: Preparing
84954f958ede: Waiting
0aa3674558b5: Waiting
2a5e893f830d: Waiting
bd18a1a09476: Waiting
352660b137e6: Waiting
7c072cee6a29: Waiting
ca33502d2cac: Waiting
4590f8c89770: Waiting
6f78efdc0a6b: Waiting
1e5fdc3d671c: Waiting
0b0f2f2f5279: Preparing
69984deac861: Waiting
6398d5cccd2c: Waiting
0b0f2f2f5279: Waiting
c0c80369275f: Pushed
ea7e0dd8d579: Pushed
eafcc0a37937: Pushed
0bc31fcf22e4: Pushed
84954f958ede: Pushed
73c3bea82b73: Pushed
2a5e893f830d: Pushed
6f78efdc0a6b: Pushed
bd18a1a09476: Pushed
0aa3674558b5: Layer already exists
7c072cee6a29: Layer already exists
4590f8c89770: Pushed
1e5fdc3d671c: Layer already exists
613ab28cf833: Layer already exists
bed676ceab7a: Layer already exists
69984deac861: Pushed
6398d5cccd2c: Layer already exists
0b0f2f2f5279: Layer already exists
ca33502d2cac: Pushed
352660b137e6: Pushed
20220205125145: digest: sha256:055c05273d506a6e8284e92fc3428bb39a7bb498d9237f4ba1aa6b63ac7d017b size: 4520

> Task :sdks:java:testing:load-tests:run
Feb 05, 2022 12:54:32 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Feb 05, 2022 12:54:33 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 204 files. Enable logging at DEBUG level to see which files will be staged.
Feb 05, 2022 12:54:34 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Feb 05, 2022 12:54:34 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Feb 05, 2022 12:54:37 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 204 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Feb 05, 2022 12:54:39 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 204 files cached, 0 files newly uploaded in 1 seconds
Feb 05, 2022 12:54:39 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Feb 05, 2022 12:54:39 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <114077 bytes, hash da1da67ee3a7a67940df1af68529f9f579d6853ec42a1c3e292b590edd29193b> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-2h2mfuOnpnlA3xr2hSn59XnWhT7EKhw-KStZDt0pGTs.pb
Feb 05, 2022 12:54:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Feb 05, 2022 12:54:41 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@574a89e2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1e1e9ef3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3dd31157, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@31c628e7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3240b2a4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58434b19, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7d3fb0ef, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7dbe2ebf, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4adc663e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@885e7ff, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@8bd86c8, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4fa9ab6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2d3ef181, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@a2341c6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6e4c0d8c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3e3315d9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@64db4967, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@74e6094b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7a485a36, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5cf3157b]
Feb 05, 2022 12:54:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Feb 05, 2022 12:54:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Feb 05, 2022 12:54:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Feb 05, 2022 12:54:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Feb 05, 2022 12:54:41 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4c5228e7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@38e7ed69, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@806996, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@78b612c6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@257e0827, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@22752544, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@21ba2445, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@69d23296, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c3820bb, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@376c7d7d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4784efd9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3fba233d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@427ae189, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@16a9eb2e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@76332405, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@187e5235, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@d1d8e1a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5434e40c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3b48e183, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@514de325]
Feb 05, 2022 12:54:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Feb 05, 2022 12:54:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Feb 05, 2022 12:54:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Feb 05, 2022 12:54:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Feb 05, 2022 12:54:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Feb 05, 2022 12:54:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Feb 05, 2022 12:54:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Feb 05, 2022 12:54:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Feb 05, 2022 12:54:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Feb 05, 2022 12:54:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Feb 05, 2022 12:54:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Feb 05, 2022 12:54:41 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.37.0-SNAPSHOT
Feb 05, 2022 12:54:42 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-02-05_04_54_41-13484680452619416034?project=apache-beam-testing
Feb 05, 2022 12:54:42 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2022-02-05_04_54_41-13484680452619416034
Feb 05, 2022 12:54:42 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2022-02-05_04_54_41-13484680452619416034
Feb 05, 2022 12:54:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2022-02-05T12:54:49.223Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-02-9qkg. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Feb 05, 2022 12:54:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-05T12:54:53.857Z: Worker configuration: e2-standard-2 in us-central1-b.
Feb 05, 2022 12:54:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-05T12:54:54.395Z: Expanding SplittableParDo operations into optimizable parts.
Feb 05, 2022 12:54:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-05T12:54:54.442Z: Expanding CollectionToSingleton operations into optimizable parts.
Feb 05, 2022 12:54:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-05T12:54:54.520Z: Expanding CoGroupByKey operations into optimizable parts.
Feb 05, 2022 12:54:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-05T12:54:54.636Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Feb 05, 2022 12:54:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-05T12:54:54.713Z: Expanding GroupByKey operations into streaming Read/Write steps
Feb 05, 2022 12:54:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-05T12:54:54.759Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Feb 05, 2022 12:54:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-05T12:54:54.869Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Feb 05, 2022 12:54:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-05T12:54:54.908Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Feb 05, 2022 12:54:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-05T12:54:54.973Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Feb 05, 2022 12:54:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-05T12:54:55.004Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Feb 05, 2022 12:54:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-05T12:54:55.037Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Feb 05, 2022 12:54:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-05T12:54:55.079Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Feb 05, 2022 12:54:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-05T12:54:55.120Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Feb 05, 2022 12:54:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-05T12:54:55.166Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Feb 05, 2022 12:54:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-05T12:54:55.194Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Feb 05, 2022 12:54:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-05T12:54:55.228Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Feb 05, 2022 12:54:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-05T12:54:55.249Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Feb 05, 2022 12:54:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-05T12:54:55.276Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Feb 05, 2022 12:54:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-05T12:54:55.303Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Feb 05, 2022 12:54:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-05T12:54:55.336Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Feb 05, 2022 12:54:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-05T12:54:55.367Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Feb 05, 2022 12:54:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-05T12:54:55.402Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Feb 05, 2022 12:54:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-05T12:54:55.437Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Feb 05, 2022 12:54:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-05T12:54:55.474Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Feb 05, 2022 12:54:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-05T12:54:55.504Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Feb 05, 2022 12:54:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-05T12:54:55.539Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Feb 05, 2022 12:54:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-05T12:54:55.573Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Feb 05, 2022 12:54:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-05T12:54:55.608Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Feb 05, 2022 12:54:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-05T12:54:55.643Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Feb 05, 2022 12:54:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-05T12:54:56.146Z: Starting 5 ****s in us-central1-b...
Feb 05, 2022 12:55:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-05T12:55:06.874Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Feb 05, 2022 12:55:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-05T12:55:40.608Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Feb 05, 2022 12:56:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-05T12:56:41.210Z: Workers have started successfully.
Feb 05, 2022 12:56:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-05T12:56:41.244Z: Workers have started successfully.
Feb 05, 2022 4:00:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-05T16:00:45.291Z: Cancel request is committed for workflow job: 2022-02-05_04_54_41-13484680452619416034.
Feb 05, 2022 4:00:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-05T16:00:45.365Z: Cleaning up.
Feb 05, 2022 4:00:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-05T16:00:45.423Z: Stopping **** pool...
Feb 05, 2022 4:00:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-05T16:00:45.465Z: Stopping **** pool...
Feb 05, 2022 4:03:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-05T16:03:14.878Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Feb 05, 2022 4:03:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-05T16:03:15.058Z: Worker pool stopped.
Feb 05, 2022 4:03:23 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2022-02-05_04_54_41-13484680452619416034 finished with status CANCELLED.
Load test results for test (ID): 6a3f8a7b-51ed-4375-acb8-a03a87edb427 and timestamp: 2022-02-05T12:54:33.571000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11018.873
dataflow_v2_java11_total_bytes_count             1.31740493E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerJavaImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220205125145
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:055c05273d506a6e8284e92fc3428bb39a7bb498d9237f4ba1aa6b63ac7d017b
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220205125145]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:055c05273d506a6e8284e92fc3428bb39a7bb498d9237f4ba1aa6b63ac7d017b]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220205125145] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:055c05273d506a6e8284e92fc3428bb39a7bb498d9237f4ba1aa6b63ac7d017b])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:055c05273d506a6e8284e92fc3428bb39a7bb498d9237f4ba1aa6b63ac7d017b
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:055c05273d506a6e8284e92fc3428bb39a7bb498d9237f4ba1aa6b63ac7d017b
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:055c05273d506a6e8284e92fc3428bb39a7bb498d9237f4ba1aa6b63ac7d017b].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings

Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.

BUILD FAILED in 3h 12m 5s
109 actionable tasks: 72 executed, 33 from cache, 4 up-to-date

Publishing build scan...
https://gradle.com/s/hegfl4wykygb2

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #232

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/232/display/redirect?page=changes>

Changes:

[Kiley Sok] Allow Java 17 to be used in SDK

[Kiley Sok] add testing support

[Kiley Sok] Add more testing support for java 17

[Kiley Sok] workaround for jamm

[Kiley Sok] Add Jenkins test for Java 17

[Kiley Sok] Fix jvm hex and skip errorprone

[Kiley Sok] Fix display data for anonymous classes

[Kiley Sok] fix jpms tests

[Kiley Sok] skip zetasql

[Kiley Sok] spotless

[Kiley Sok] spotless

[Kiley Sok] Fix trigger

[Kiley Sok] skip checker framework

[Kiley Sok] fix app name

[Kiley Sok] remove duplicate property check

[Heejong Lee] [BEAM-13813] Add support for URL artifact to extractStagingToPath

[avilovpavel6] Remove Python SQL Test example from catalog

[relax] Fix timer consistency in direct runner

[noreply] [BEAM-13757] adds pane observation in DoFn (#16629)

[Jan Lukavský] Change links to Books from Amazon to Publisher

[noreply] [BEAM-13605] Add support for pandas 1.4.0 (#16590)

[noreply] [BEAM-13761] adds Debezium IO wrapper for Go SDK (#16642)

[noreply] [BEAM-13024] Unify PipelineOptions behavior (#16719)

[noreply] Update sdks/go/pkg/beam/artifact/materialize_test.go

[noreply] Merge pull request #16605 from [BEAM-13634][Playground] Create a

[noreply] Merge pull request #16593 from [BEAM-13725][Playground] Add graph to the

[noreply] Merge pull request #16699 from [BEAM-13789][Playground] Change logic of

[alexander.chermenin] Fixed CSS for Case study page


------------------------------------------
[...truncated 49.87 KB...]
c19bd81b1efb: Waiting
4b8d37f582e3: Preparing
0aa3674558b5: Preparing
7c072cee6a29: Preparing
1e5fdc3d671c: Preparing
613ab28cf833: Preparing
8b589c1403fa: Waiting
bed676ceab7a: Preparing
6398d5cccd2c: Preparing
0b0f2f2f5279: Preparing
0aa3674558b5: Waiting
7c072cee6a29: Waiting
0b0f2f2f5279: Waiting
bed676ceab7a: Waiting
6398d5cccd2c: Waiting
1e5fdc3d671c: Waiting
613ab28cf833: Waiting
e80698b4686c: Waiting
4b8d37f582e3: Waiting
43ca28afbb62: Waiting
bdc334ec4ad3: Waiting
713d88b5bc37: Pushed
f2f97eb50931: Pushed
b672ffe8299a: Pushed
6d802e23ba91: Pushed
891460082312: Pushed
583e128fea73: Pushed
8faf192789a7: Pushed
e80698b4686c: Pushed
8b589c1403fa: Pushed
0aa3674558b5: Layer already exists
7c072cee6a29: Layer already exists
bdc334ec4ad3: Pushed
1e5fdc3d671c: Layer already exists
613ab28cf833: Layer already exists
bed676ceab7a: Layer already exists
c19bd81b1efb: Pushed
6398d5cccd2c: Layer already exists
0b0f2f2f5279: Layer already exists
4b8d37f582e3: Pushed
43ca28afbb62: Pushed
20220204124342: digest: sha256:3ee374fec2e9f88c91a044b0d0ff9cdeae20b4dd8e7b92ec1240d584c61368cc size: 4520

> Task :sdks:java:testing:load-tests:run
Feb 04, 2022 12:46:41 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Feb 04, 2022 12:46:42 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 204 files. Enable logging at DEBUG level to see which files will be staged.
Feb 04, 2022 12:46:43 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Feb 04, 2022 12:46:43 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Feb 04, 2022 12:46:46 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 204 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Feb 04, 2022 12:46:47 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 204 files cached, 0 files newly uploaded in 0 seconds
Feb 04, 2022 12:46:47 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Feb 04, 2022 12:46:47 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <114077 bytes, hash 595adb75bf06a4399b57cbd9a7a9bc641ea4e3cac2d64784a40e96a902df8ccc> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-WVrbdb8GpDmbV8vZp6m8ZB6k48rC1keEpA6WqQLfjMw.pb
Feb 04, 2022 12:46:49 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Feb 04, 2022 12:46:49 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1d805aa1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@176f7f3b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@30ca0779, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58740366, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@47be0f9b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2bc426f0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4bd51d3e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@33425811, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4b74b35, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@e4e1ef5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6d11ceef, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4cb2918c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@72e295cc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@c2584d3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6fa0450e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@37468787, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@51ec2856, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@714f3da4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1caa9eb6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1f53481b]
Feb 04, 2022 12:46:49 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Feb 04, 2022 12:46:49 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Feb 04, 2022 12:46:49 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Feb 04, 2022 12:46:49 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Feb 04, 2022 12:46:49 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@56399b9e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@34b9eb03, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43fda8d9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@49d831c2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@53a7a60c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@28bdbe88, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6a87026, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@ef60710, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@600f5704, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2503ec73, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@606f81b5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5e1fc42f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@44b21f9f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7ee8130e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6296474f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4288d98e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@169268a7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@285c6918, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@78a0ff63, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7c601d50]
Feb 04, 2022 12:46:49 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Feb 04, 2022 12:46:49 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Feb 04, 2022 12:46:49 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Feb 04, 2022 12:46:49 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Feb 04, 2022 12:46:49 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Feb 04, 2022 12:46:49 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Feb 04, 2022 12:46:49 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Feb 04, 2022 12:46:49 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Feb 04, 2022 12:46:49 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Feb 04, 2022 12:46:49 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Feb 04, 2022 12:46:49 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Feb 04, 2022 12:46:49 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.37.0-SNAPSHOT
Feb 04, 2022 12:46:50 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-02-04_04_46_49-1907528646063558349?project=apache-beam-testing
Feb 04, 2022 12:46:50 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2022-02-04_04_46_49-1907528646063558349
Feb 04, 2022 12:46:50 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2022-02-04_04_46_49-1907528646063558349
Feb 04, 2022 12:47:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2022-02-04T12:46:58.149Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-02-4grb. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Feb 04, 2022 12:47:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-04T12:47:08.245Z: Worker configuration: e2-standard-2 in us-central1-b.
Feb 04, 2022 12:47:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-04T12:47:08.933Z: Expanding SplittableParDo operations into optimizable parts.
Feb 04, 2022 12:47:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-04T12:47:08.998Z: Expanding CollectionToSingleton operations into optimizable parts.
Feb 04, 2022 12:47:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-04T12:47:09.052Z: Expanding CoGroupByKey operations into optimizable parts.
Feb 04, 2022 12:47:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-04T12:47:09.117Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Feb 04, 2022 12:47:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-04T12:47:09.156Z: Expanding GroupByKey operations into streaming Read/Write steps
Feb 04, 2022 12:47:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-04T12:47:09.216Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Feb 04, 2022 12:47:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-04T12:47:09.322Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Feb 04, 2022 12:47:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-04T12:47:09.358Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Feb 04, 2022 12:47:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-04T12:47:09.382Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Feb 04, 2022 12:47:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-04T12:47:09.406Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Feb 04, 2022 12:47:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-04T12:47:09.428Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Feb 04, 2022 12:47:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-04T12:47:09.452Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Feb 04, 2022 12:47:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-04T12:47:09.477Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Feb 04, 2022 12:47:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-04T12:47:09.502Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Feb 04, 2022 12:47:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-04T12:47:09.528Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Feb 04, 2022 12:47:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-04T12:47:09.549Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Feb 04, 2022 12:47:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-04T12:47:09.576Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Feb 04, 2022 12:47:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-04T12:47:09.601Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Feb 04, 2022 12:47:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-04T12:47:09.634Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Feb 04, 2022 12:47:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-04T12:47:09.669Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Feb 04, 2022 12:47:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-04T12:47:09.702Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Feb 04, 2022 12:47:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-04T12:47:09.735Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Feb 04, 2022 12:47:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-04T12:47:09.770Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Feb 04, 2022 12:47:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-04T12:47:09.824Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Feb 04, 2022 12:47:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-04T12:47:09.858Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Feb 04, 2022 12:47:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-04T12:47:09.892Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Feb 04, 2022 12:47:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-04T12:47:09.924Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Feb 04, 2022 12:47:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-04T12:47:09.979Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Feb 04, 2022 12:47:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-04T12:47:10.008Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Feb 04, 2022 12:47:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-04T12:47:10.469Z: Starting 5 ****s in us-central1-b...
Feb 04, 2022 12:47:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-04T12:47:34.446Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Feb 04, 2022 12:48:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-04T12:48:00.452Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Feb 04, 2022 12:49:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-04T12:49:02.670Z: Workers have started successfully.
Feb 04, 2022 12:49:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-04T12:49:02.722Z: Workers have started successfully.
Feb 04, 2022 4:00:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-04T16:00:37.024Z: Cancel request is committed for workflow job: 2022-02-04_04_46_49-1907528646063558349.
Feb 04, 2022 4:00:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-04T16:00:37.089Z: Cleaning up.
Feb 04, 2022 4:00:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-04T16:00:37.203Z: Stopping **** pool...
Feb 04, 2022 4:00:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-04T16:00:37.300Z: Stopping **** pool...
Feb 04, 2022 4:02:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-04T16:02:54.125Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Feb 04, 2022 4:02:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-04T16:02:54.177Z: Worker pool stopped.
Feb 04, 2022 4:03:00 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2022-02-04_04_46_49-1907528646063558349 finished with status CANCELLED.
Load test results for test (ID): 1a161861-7766-4ff7-9498-738bfba79e90 and timestamp: 2022-02-04T12:46:42.816000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11338.712
dataflow_v2_java11_total_bytes_count             1.95515622E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerJavaImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220204124342
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:3ee374fec2e9f88c91a044b0d0ff9cdeae20b4dd8e7b92ec1240d584c61368cc
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220204124342]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:3ee374fec2e9f88c91a044b0d0ff9cdeae20b4dd8e7b92ec1240d584c61368cc]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220204124342] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:3ee374fec2e9f88c91a044b0d0ff9cdeae20b4dd8e7b92ec1240d584c61368cc])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:3ee374fec2e9f88c91a044b0d0ff9cdeae20b4dd8e7b92ec1240d584c61368cc
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:3ee374fec2e9f88c91a044b0d0ff9cdeae20b4dd8e7b92ec1240d584c61368cc
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:3ee374fec2e9f88c91a044b0d0ff9cdeae20b4dd8e7b92ec1240d584c61368cc].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:0035bdbcee488f123ccaa7ee9d663dfccf8e289224ecbc1989320d0e222f2efa
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:0035bdbcee488f123ccaa7ee9d663dfccf8e289224ecbc1989320d0e222f2efa
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:0035bdbcee488f123ccaa7ee9d663dfccf8e289224ecbc1989320d0e222f2efa].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings

Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.

BUILD FAILED in 3h 19m 43s
109 actionable tasks: 72 executed, 33 from cache, 4 up-to-date

Publishing build scan...
https://gradle.com/s/i7s2cmzcqyfbk

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 - Build # 231 - Aborted!

Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 - Build # 231 - Aborted:

Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/231/ to view the results.

Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #230

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/230/display/redirect?page=changes>

Changes:

[career] [BEAM-13734] Support cache directories that use GCS buckets

[noreply] Merge pull request #16655 from [BEAM-12164]: Add retry protection to

[noreply] Merge pull request #16586 from [BEAM-13731] FhirIO: Add support for

[noreply] [BEAM-13011] Adds a link to Multi-language Pipelines Tips wiki page

[noreply] [BEAM-12572] Run python examples on multiple runners (#16154)

[noreply] [BEAM-13574] Large Wordcount (#16455)

[noreply] [BEAM-13293] Refactor JDBC IO Go Wrapper (#16686)

[noreply] Edit license script for Java, add manual licenses for xz (#16692)


------------------------------------------
[...truncated 50.96 KB...]
a89e28714e65: Pushed
0afb37a61eef: Pushed
a63956e42953: Pushed
eaae80668d8b: Pushed
120d0635c2e7: Pushed
1a0d8be7fecb: Layer already exists
7c072cee6a29: Layer already exists
1e5fdc3d671c: Layer already exists
613ab28cf833: Layer already exists
ab7303d1380c: Pushed
bed676ceab7a: Layer already exists
6398d5cccd2c: Layer already exists
0b0f2f2f5279: Layer already exists
5f89de82cb39: Pushed
1627f9a0564a: Pushed
20220202124338: digest: sha256:d6539f1dff0ea28028603f51cab425bd32b970be6af0edcbdda5d4c8cc90543f size: 4520

> Task :sdks:java:testing:load-tests:run
Feb 02, 2022 12:46:05 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Feb 02, 2022 12:46:09 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 204 files. Enable logging at DEBUG level to see which files will be staged.
Feb 02, 2022 12:46:10 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Feb 02, 2022 12:46:10 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Feb 02, 2022 12:46:13 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 204 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Feb 02, 2022 12:46:13 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 204 files cached, 0 files newly uploaded in 0 seconds
Feb 02, 2022 12:46:14 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Feb 02, 2022 12:46:14 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <114077 bytes, hash 451f758bb1d6ad78711bd4be17be988bbca00f3199f0131dd3d03e3e5f3fbae1> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-RR91i7HWrXhxG9S-F76Yi7ygDzGZ8BMd09A-Pl8_uuE.pb
Feb 02, 2022 12:46:15 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Feb 02, 2022 12:46:15 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2100d047, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4af45442, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1d805aa1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@176f7f3b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@30ca0779, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58740366, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@47be0f9b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2bc426f0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4bd51d3e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@33425811, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4b74b35, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@e4e1ef5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6d11ceef, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4cb2918c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@72e295cc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@c2584d3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6fa0450e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@37468787, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@51ec2856, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@714f3da4]
Feb 02, 2022 12:46:15 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Feb 02, 2022 12:46:15 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Feb 02, 2022 12:46:15 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Feb 02, 2022 12:46:16 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Feb 02, 2022 12:46:16 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6242ae3b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@65ddee5a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@56399b9e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@34b9eb03, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43fda8d9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@49d831c2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@53a7a60c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@28bdbe88, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6a87026, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@ef60710, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@600f5704, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2503ec73, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@606f81b5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5e1fc42f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@44b21f9f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7ee8130e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6296474f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4288d98e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@169268a7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@285c6918]
Feb 02, 2022 12:46:16 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Feb 02, 2022 12:46:16 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Feb 02, 2022 12:46:16 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Feb 02, 2022 12:46:16 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Feb 02, 2022 12:46:16 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Feb 02, 2022 12:46:16 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Feb 02, 2022 12:46:16 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Feb 02, 2022 12:46:16 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Feb 02, 2022 12:46:16 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Feb 02, 2022 12:46:16 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Feb 02, 2022 12:46:16 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Feb 02, 2022 12:46:16 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.37.0-SNAPSHOT
Feb 02, 2022 12:46:17 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-02-02_04_46_16-18133998314817617901?project=apache-beam-testing
Feb 02, 2022 12:46:17 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2022-02-02_04_46_16-18133998314817617901
Feb 02, 2022 12:46:17 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2022-02-02_04_46_16-18133998314817617901
Feb 02, 2022 12:46:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2022-02-02T12:46:23.607Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-02-yoet. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Feb 02, 2022 12:46:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-02T12:46:29.105Z: Worker configuration: e2-standard-2 in us-central1-b.
Feb 02, 2022 12:46:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-02T12:46:29.775Z: Expanding SplittableParDo operations into optimizable parts.
Feb 02, 2022 12:46:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-02T12:46:29.811Z: Expanding CollectionToSingleton operations into optimizable parts.
Feb 02, 2022 12:46:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-02T12:46:29.900Z: Expanding CoGroupByKey operations into optimizable parts.
Feb 02, 2022 12:46:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-02T12:46:29.968Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Feb 02, 2022 12:46:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-02T12:46:30.000Z: Expanding GroupByKey operations into streaming Read/Write steps
Feb 02, 2022 12:46:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-02T12:46:30.056Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Feb 02, 2022 12:46:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-02T12:46:30.160Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Feb 02, 2022 12:46:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-02T12:46:30.197Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Feb 02, 2022 12:46:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-02T12:46:30.229Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Feb 02, 2022 12:46:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-02T12:46:30.261Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Feb 02, 2022 12:46:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-02T12:46:30.284Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Feb 02, 2022 12:46:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-02T12:46:30.317Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Feb 02, 2022 12:46:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-02T12:46:30.338Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Feb 02, 2022 12:46:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-02T12:46:30.360Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Feb 02, 2022 12:46:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-02T12:46:30.418Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Feb 02, 2022 12:46:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-02T12:46:30.454Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Feb 02, 2022 12:46:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-02T12:46:30.485Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Feb 02, 2022 12:46:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-02T12:46:30.528Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Feb 02, 2022 12:46:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-02T12:46:30.563Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Feb 02, 2022 12:46:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-02T12:46:30.597Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Feb 02, 2022 12:46:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-02T12:46:30.637Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Feb 02, 2022 12:46:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-02T12:46:30.669Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Feb 02, 2022 12:46:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-02T12:46:30.692Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Feb 02, 2022 12:46:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-02T12:46:30.725Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Feb 02, 2022 12:46:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-02T12:46:30.748Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Feb 02, 2022 12:46:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-02T12:46:30.782Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Feb 02, 2022 12:46:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-02T12:46:30.818Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Feb 02, 2022 12:46:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-02T12:46:30.861Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Feb 02, 2022 12:46:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-02T12:46:30.885Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Feb 02, 2022 12:46:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-02T12:46:31.214Z: Starting 5 ****s in us-central1-b...
Feb 02, 2022 12:47:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-02T12:47:00.651Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Feb 02, 2022 12:47:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-02T12:47:15.595Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Feb 02, 2022 12:48:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-02T12:48:17.631Z: Workers have started successfully.
Feb 02, 2022 12:48:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-02T12:48:17.661Z: Workers have started successfully.
Feb 02, 2022 4:00:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-02T16:00:36.272Z: Cancel request is committed for workflow job: 2022-02-02_04_46_16-18133998314817617901.
Feb 02, 2022 4:00:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-02T16:00:36.354Z: Cleaning up.
Feb 02, 2022 4:00:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-02T16:00:36.436Z: Stopping **** pool...
Feb 02, 2022 4:00:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-02T16:00:36.494Z: Stopping **** pool...
Feb 02, 2022 4:02:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-02T16:02:54.502Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Feb 02, 2022 4:02:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-02T16:02:54.547Z: Worker pool stopped.
Feb 02, 2022 4:03:01 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2022-02-02_04_46_16-18133998314817617901 finished with status CANCELLED.
Load test results for test (ID): 2b66bcb4-e5c7-4d30-ae5d-c1d3bf260d18 and timestamp: 2022-02-02T12:46:09.836000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11510.768
dataflow_v2_java11_total_bytes_count             2.21004923E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerJavaImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220202124338
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:d6539f1dff0ea28028603f51cab425bd32b970be6af0edcbdda5d4c8cc90543f
Deleted: sha256:123eaad56948b21de901271c8288dda548b1d1c337870c6d0a6b34d51ddb81a7
Deleted: sha256:e2980f17f1d47b5b1cc1b003ad784c26eb689fca3db3c7f442ae9cf6c1201f6d
Deleted: sha256:730733f9045c510868de9ed4aedfa46484628a275f8d98b26e21b2ec131ef5c4
Deleted: sha256:b18d377527fa5e1646de41810fb19b9bef72fc949b7e0abe60adf6dfe0178908
Deleted: sha256:2e479672d36f9c63bc3d3a5aa20fc17546f982ce55ca319b125a1daa989cc21b
Deleted: sha256:fcba682443676300048dc2ce59410286fd4222f0ebcfee142523fb32d4d8cb1b
Deleted: sha256:8f5c5b69f9865365f7cc0ed03d4d56e089fb158ea1811a11c924482d1f23905a
Deleted: sha256:40fe6748cb47fd0de85890888dc1a3c78df6b612c54dcc6460ef135f96c805cd
Deleted: sha256:e8852104807b3242f1e85fbf74c3425da5044dedfe45896885e1b9e8f421f303
Deleted: sha256:b29192f6950f8cb8cb11a0db44cfe0229da2b32c82480114ba7ca005628c8075
Deleted: sha256:a73875698268c2f344f16db92548e71055668cb8f5085db62cb1aeb7447cbb68
Deleted: sha256:e84fcfa3dadb190d2b1b9ad8f67617b13b9fa9f0fb29c0b97f080558de701003
Deleted: sha256:d6b0b9197faa7c1532864171a822953245fefe042e56cfe9cfea9e612f60cd2c
Deleted: sha256:fd3e77cf34325ee83c545989c2697d70d2c62da0bad2eed2d1d73c11a9d7b633
Deleted: sha256:fca7aba841a7d8f8c3e2b594d9cb885e548f130f50e79db1f7841c37cd8c33a7
Deleted: sha256:4dd2f552cddf2f1281ec8eca6e4205afdaf6db7c4f30c045fed692984b6070c3
Deleted: sha256:dc5ab14744238929d3c6c5060a67b3d77d248b4a09ba71818870e18ae3f157bd
Deleted: sha256:fa3f27991d3fc074b8c9db6ea261f5810d6bfe022f04218e886e3e125ab299a4
Deleted: sha256:da4d986c718ae5951f69826f84f3301eed70f469d650aff76a842fdb38a318fe
Deleted: sha256:3848d096414abddcf8e63df38197ef168a3c3b5ef712c92168773b9fae558f54
Deleted: sha256:ff8508d2ab472eaee54b8f7d18e46f87a920d50b17ee299a23152dea5c84f40a
Deleted: sha256:84e16c5471ab9455dfbe6611aa9dc4d8b4ace162083803e7bb615b6d186cc665
Deleted: sha256:cf9ce6b88e5d9b03732ce3468dfefd51dacd3241d09d715785d03a1ca4a9be40
Deleted: sha256:f92c02afe0d6dc19b3af79f00a6150a340d5b24816c1fb2334e3dedbbc780c0a
Deleted: sha256:902e329add528333e47e8bce53d9532f0cacdf972486fb0a761e3b1a93bbdc08
Deleted: sha256:bb0e41ddd4ed447d9ca3c3ae6a77ba8c32ecc999039caa0018cc7fe69de89748
Deleted: sha256:d7da683a33932f4056c2ed1acbff035abdba829b17d6464510ec87a88eefe8eb
Deleted: sha256:3a64b0bf9d343c643b4dcd44b770775692691814da26a3e1dc1faced836badfc
Deleted: sha256:952a00834a4f1310976619f2dcea04baac636f483b6c8d1edbe3a602aea1b93f
Deleted: sha256:1663d22179d31f6ccc3681af813c70396a6e74f2eb28a68d723b230030b4502b
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220202124338]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:d6539f1dff0ea28028603f51cab425bd32b970be6af0edcbdda5d4c8cc90543f]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220202124338] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:d6539f1dff0ea28028603f51cab425bd32b970be6af0edcbdda5d4c8cc90543f])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:d6539f1dff0ea28028603f51cab425bd32b970be6af0edcbdda5d4c8cc90543f
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:d6539f1dff0ea28028603f51cab425bd32b970be6af0edcbdda5d4c8cc90543f
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:d6539f1dff0ea28028603f51cab425bd32b970be6af0edcbdda5d4c8cc90543f].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings

Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.

BUILD FAILED in 3h 19m 45s
109 actionable tasks: 72 executed, 33 from cache, 4 up-to-date

Publishing build scan...
https://gradle.com/s/hb36ptj3lj7wa

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #229

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/229/display/redirect?page=changes>

Changes:

[daria.malkova] Support SCIO SDK via sbt projects

[samuelw] [BEAM-11648] Share thread pool across RetryManager instances.

[Pablo Estrada] Exclude per-key order tests on Twister2 runner

[Heejong Lee] Fix Java SDK container image name for load-tests and nexmark

[daria.malkova] Change executable name fo go tests

[avilovpavel6] Fix java test

[noreply] [BEAM-13769] Skip test_main_session_not_staged_when_using_cloudpickle

[noreply] [BEAM-6744] Support implicitly setting project id in Go Dataflow runner

[noreply] Merge pull request #16493 from [BEAM-13632][Playground] Save catalog

[noreply] Exclude jul-to-slf4j from Spark runner in quickstart POM templates

[noreply] [BEAM-11936] Enable a few errorprone checks that were broken by pinned

[noreply] [BEAM-13780] Add CONTRIBUTING.md pointing to main guide (#16666)

[noreply] [BEAM-13777] Accept cache capacity as input parameter instead of default

[noreply] [BEAM-13051][A] Enable pylint warnings

[noreply] [BEAM-13779] Fix pr labeling (#16665)

[noreply] Merge pull request #16581 from [BEAM-12164]: Add

[noreply] Fix labeler trigger (#16674)

[noreply] [BEAM-13781] Exclude grpc-netty-shaded from gax-grpc's dependency

[noreply] [BEAM-13051] Fixed pylint warnings : raising-non-exception (E0710),

[noreply] [BEAM-13740] Correctly install go before running tests (#16673)

[noreply] [BEAM-12830] Update local Docker env Go version. (#16670)

[noreply] [BEAM-13051][B] Enable pylint warnings

[noreply] [BEAM-13430] Revert Spark libraries in Spark runner to provided (#16675)

[noreply] [BEAM-12240] Add Java 17 support (#16568)

[noreply] [BEAM-13760] Add random component to default python dataflow job name


------------------------------------------
[...truncated 96.01 KB...]
    return self.do_open(http.client.HTTPConnection, req)
  File "/usr/lib/python3.8/urllib/request.py", line 1357, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error [Errno 101] Network is unreachable>
ERROR:root:Invalid url for checkstyle-8.23: http://www.gnu.org/licenses/old-licenses/lgpl-2.1.txt. Retrying...
Traceback (most recent call last):
  File "/usr/lib/python3.8/urllib/request.py", line 1354, in do_open
    h.request(req.get_method(), req.selector, req.data, headers,
  File "/usr/lib/python3.8/http/client.py", line 1256, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/lib/python3.8/http/client.py", line 1302, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.8/http/client.py", line 1251, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.8/http/client.py", line 1011, in _send_output
    self.send(msg)
  File "/usr/lib/python3.8/http/client.py", line 951, in send
    self.connect()
  File "/usr/lib/python3.8/http/client.py", line 1418, in connect
    super().connect()
  File "/usr/lib/python3.8/http/client.py", line 922, in connect
    self.sock = self._create_connection(
  File "/usr/lib/python3.8/socket.py", line 808, in create_connection
    raise err
  File "/usr/lib/python3.8/socket.py", line 796, in create_connection
    sock.connect(sa)
OSError: [Errno 101] Network is unreachable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src/sdks/java/container/license_scripts/pull_licenses_java.py",> line 53, in pull_from_url
    url_read = urlopen(url)
  File "/usr/lib/python3.8/urllib/request.py", line 222, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib/python3.8/urllib/request.py", line 525, in open
    response = self._open(req, data)
  File "/usr/lib/python3.8/urllib/request.py", line 542, in _open
    result = self._call_chain(self.handle_open, protocol, protocol +
  File "/usr/lib/python3.8/urllib/request.py", line 502, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.8/urllib/request.py", line 1397, in https_open
    return self.do_open(http.client.HTTPSConnection, req,
  File "/usr/lib/python3.8/urllib/request.py", line 1357, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error [Errno 101] Network is unreachable>
ERROR:root:Invalid url for spotbugs-annotations-4.0.6: https://www.gnu.org/licenses/old-licenses/lgpl-2.1.en.html. Retrying...
Traceback (most recent call last):
  File "/usr/lib/python3.8/urllib/request.py", line 1354, in do_open
    h.request(req.get_method(), req.selector, req.data, headers,
  File "/usr/lib/python3.8/http/client.py", line 1256, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/lib/python3.8/http/client.py", line 1302, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.8/http/client.py", line 1251, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.8/http/client.py", line 1011, in _send_output
    self.send(msg)
  File "/usr/lib/python3.8/http/client.py", line 951, in send
    self.connect()
  File "/usr/lib/python3.8/http/client.py", line 922, in connect
    self.sock = self._create_connection(
  File "/usr/lib/python3.8/socket.py", line 808, in create_connection
    raise err
  File "/usr/lib/python3.8/socket.py", line 796, in create_connection
    sock.connect(sa)
OSError: [Errno 101] Network is unreachable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src/sdks/java/container/license_scripts/pull_licenses_java.py",> line 53, in pull_from_url
    url_read = urlopen(url)
  File "/usr/lib/python3.8/urllib/request.py", line 222, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib/python3.8/urllib/request.py", line 525, in open
    response = self._open(req, data)
  File "/usr/lib/python3.8/urllib/request.py", line 542, in _open
    result = self._call_chain(self.handle_open, protocol, protocol +
  File "/usr/lib/python3.8/urllib/request.py", line 502, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.8/urllib/request.py", line 1383, in http_open
    return self.do_open(http.client.HTTPConnection, req)
  File "/usr/lib/python3.8/urllib/request.py", line 1357, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error [Errno 101] Network is unreachable>
ERROR:root:Invalid url for jFormatString-3.0.0: http://www.gnu.org/licenses/lgpl.html. Retrying...
Traceback (most recent call last):
  File "/usr/lib/python3.8/urllib/request.py", line 1354, in do_open
    h.request(req.get_method(), req.selector, req.data, headers,
  File "/usr/lib/python3.8/http/client.py", line 1256, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/lib/python3.8/http/client.py", line 1302, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.8/http/client.py", line 1251, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.8/http/client.py", line 1011, in _send_output
    self.send(msg)
  File "/usr/lib/python3.8/http/client.py", line 951, in send
    self.connect()
  File "/usr/lib/python3.8/http/client.py", line 1418, in connect
    super().connect()
  File "/usr/lib/python3.8/http/client.py", line 922, in connect
    self.sock = self._create_connection(
  File "/usr/lib/python3.8/socket.py", line 808, in create_connection
    raise err
  File "/usr/lib/python3.8/socket.py", line 796, in create_connection
    sock.connect(sa)
OSError: [Errno 101] Network is unreachable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src/sdks/java/container/license_scripts/pull_licenses_java.py",> line 53, in pull_from_url
    url_read = urlopen(url)
  File "/usr/lib/python3.8/urllib/request.py", line 222, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib/python3.8/urllib/request.py", line 525, in open
    response = self._open(req, data)
  File "/usr/lib/python3.8/urllib/request.py", line 542, in _open
    result = self._call_chain(self.handle_open, protocol, protocol +
  File "/usr/lib/python3.8/urllib/request.py", line 502, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.8/urllib/request.py", line 1397, in https_open
    return self.do_open(http.client.HTTPSConnection, req,
  File "/usr/lib/python3.8/urllib/request.py", line 1357, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error [Errno 101] Network is unreachable>
ERROR:root:Invalid url for spotbugs-annotations-4.0.6: https://www.gnu.org/licenses/old-licenses/lgpl-2.1.en.html after 9 retries.
Traceback (most recent call last):
  File "/usr/lib/python3.8/urllib/request.py", line 1354, in do_open
    h.request(req.get_method(), req.selector, req.data, headers,
  File "/usr/lib/python3.8/http/client.py", line 1256, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/lib/python3.8/http/client.py", line 1302, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.8/http/client.py", line 1251, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.8/http/client.py", line 1011, in _send_output
    self.send(msg)
  File "/usr/lib/python3.8/http/client.py", line 951, in send
    self.connect()
  File "/usr/lib/python3.8/http/client.py", line 922, in connect
    self.sock = self._create_connection(
  File "/usr/lib/python3.8/socket.py", line 808, in create_connection
    raise err
  File "/usr/lib/python3.8/socket.py", line 796, in create_connection
    sock.connect(sa)
OSError: [Errno 101] Network is unreachable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src/sdks/java/container/license_scripts/pull_licenses_java.py",> line 53, in pull_from_url
    url_read = urlopen(url)
  File "/usr/lib/python3.8/urllib/request.py", line 222, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib/python3.8/urllib/request.py", line 525, in open
    response = self._open(req, data)
  File "/usr/lib/python3.8/urllib/request.py", line 542, in _open
    result = self._call_chain(self.handle_open, protocol, protocol +
  File "/usr/lib/python3.8/urllib/request.py", line 502, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.8/urllib/request.py", line 1383, in http_open
    return self.do_open(http.client.HTTPConnection, req)
  File "/usr/lib/python3.8/urllib/request.py", line 1357, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error [Errno 101] Network is unreachable>
ERROR:root:Invalid url for jFormatString-3.0.0: http://www.gnu.org/licenses/lgpl.html after 9 retries.
Traceback (most recent call last):
  File "/usr/lib/python3.8/urllib/request.py", line 1354, in do_open
    h.request(req.get_method(), req.selector, req.data, headers,
  File "/usr/lib/python3.8/http/client.py", line 1256, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/lib/python3.8/http/client.py", line 1302, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.8/http/client.py", line 1251, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.8/http/client.py", line 1011, in _send_output
    self.send(msg)
  File "/usr/lib/python3.8/http/client.py", line 951, in send
    self.connect()
  File "/usr/lib/python3.8/http/client.py", line 922, in connect
    self.sock = self._create_connection(
  File "/usr/lib/python3.8/socket.py", line 808, in create_connection
    raise err
  File "/usr/lib/python3.8/socket.py", line 796, in create_connection
    sock.connect(sa)
OSError: [Errno 101] Network is unreachable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src/sdks/java/container/license_scripts/pull_licenses_java.py",> line 53, in pull_from_url
    url_read = urlopen(url)
  File "/usr/lib/python3.8/urllib/request.py", line 222, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib/python3.8/urllib/request.py", line 525, in open
    response = self._open(req, data)
  File "/usr/lib/python3.8/urllib/request.py", line 542, in _open
    result = self._call_chain(self.handle_open, protocol, protocol +
  File "/usr/lib/python3.8/urllib/request.py", line 502, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.8/urllib/request.py", line 1383, in http_open
    return self.do_open(http.client.HTTPConnection, req)
  File "/usr/lib/python3.8/urllib/request.py", line 1357, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error [Errno 101] Network is unreachable>
ERROR:root:Invalid url for checkstyle-8.23: http://www.gnu.org/licenses/old-licenses/lgpl-2.1.txt after 9 retries.
ERROR:root:['spotbugs-annotations-4.0.6', 'jFormatString-3.0.0', 'checkstyle-8.23']
ERROR:root:**************************************** Licenses were not able to be pulled automatically for some dependencies. Please search source code of the dependencies on the internet and add "license" and "notice" (if available) field to <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src/sdks/java/container/license_scripts/dep_urls_java.yaml> for each missing license. Dependency List: [checkstyle-8.23,jFormatString-3.0.0,spotbugs-annotations-4.0.6]
INFO:root:pull_licenses_java.py failed. It took 1226.370358 seconds with 16 threads.
Traceback (most recent call last):
  File "<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src/sdks/java/container/license_scripts/pull_licenses_java.py",> line 314, in <module>
    raise RuntimeError('{n} error(s) occurred.'.format(n=len(error_msg)),
RuntimeError: ('1 error(s) occurred.', ['**************************************** Licenses were not able to be pulled automatically for some dependencies. Please search source code of the dependencies on the internet and add "license" and "notice" (if available) field to <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src/sdks/java/container/license_scripts/dep_urls_java.yaml> for each missing license. Dependency List: [checkstyle-8.23,jFormatString-3.0.0,spotbugs-annotations-4.0.6]'])

> Task :sdks:java:container:pullLicenses FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:container:pullLicenses'.
> Process 'command './license_scripts/license_script.sh'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings

Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.

BUILD FAILED in 21m 12s
103 actionable tasks: 66 executed, 33 from cache, 4 up-to-date

Publishing build scan...
https://gradle.com/s/uajqb2awvk6yk

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #228

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/228/display/redirect?page=changes>

Changes:

[mrudary] Generalize S3FileSystem to support multiple URI schemes.


------------------------------------------
Started by timer
Running as SYSTEM
[EnvInject] - Loading node environment variables.
Building remotely on apache-beam-jenkins-2 (beam) in workspace <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/>
The recommended git tool is: NONE
No credentials specified
Wiping out workspace first.
Cloning the remote Git repository
Cloning repository https://github.com/apache/beam.git
 > git init <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src> # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
 > git --version # timeout=10
 > git --version # 'git version 2.25.1'
 > git fetch --tags --force --progress -- https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* # timeout=10
 > git config remote.origin.url https://github.com/apache/beam.git # timeout=10
 > git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
 > git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
 > git fetch --tags --force --progress -- https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/* # timeout=10
 > git rev-parse origin/master^{commit} # timeout=10
Checking out Revision e638c1183407999bca3e8e4987119c0d9158d00d (origin/master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f e638c1183407999bca3e8e4987119c0d9158d00d # timeout=10
Commit message: "Merge pull request #16607: [BEAM-13245] Generalize S3FileSystem to support multiple URI schemes."
 > git rev-list --no-walk 6c9c208197d3d74b1c3643d22716ad3b00213506 # timeout=10
No emails were triggered.
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content 
SPARK_LOCAL_IP=127.0.0.1
SETUPTOOLS_USE_DISTUTILS=stdlib

[EnvInject] - Variables injected successfully.
[beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11] $ /bin/bash -xe /tmp/jenkins4689547643951386454.sh
+ echo '*** Load test: CoGBK 2GB 100  byte records - single key ***'
*** Load test: CoGBK 2GB 100  byte records - single key ***
[Gradle] - Launching build.
[src] $ <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src/gradlew> -PloadTest.mainClass=org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest -Prunner=:runners:google-cloud-dataflow-java '-PloadTest.args=--project=apache-beam-testing --region=us-central1 --appName=load_tests_Java11_Dataflow_V2_streaming_CoGBK_1 --tempLocation=gs://temp-storage-for-perf-tests/loadtests --influxMeasurement=java_streaming_cogbk_1 --influxTags={"runnerVersion":"v2","jdk":"java11"} --publishToInfluxDB=true --sourceOptions={"numRecords":20000000,"keySizeBytes":10,"valueSizeBytes":90,"numHotKeys":1} --coSourceOptions={"numRecords":2000000,"keySizeBytes":10,"valueSizeBytes":90,"numHotKeys":1000} --iterations=1 --numWorkers=5 --autoscalingAlgorithm=NONE --streaming=true --influxDatabase=beam_test_metrics --influxHost=http://10.128.0.96:8086 --inputWindowDurationSec=1200 --coInputWindowDurationSec=1200 --runner=DataflowRunner' -Prunner.version=V2 -PcompileAndRunTestsWithJava11 -Pjava11Home=/usr/lib/jvm/java-11-openjdk-amd64 --continue --max-****s=12 -Dorg.gradle.jvmargs=-Xms2g -Dorg.gradle.jvmargs=-Xmx4g -Dorg.gradle.vfs.watch=false -Pdocker-pull-licenses :sdks:java:testing:load-tests:run
Starting a Gradle Daemon, 3 busy Daemons could not be reused, use --status for details
Configuration on demand is an incubating feature.
> Task :buildSrc:compileJava NO-SOURCE
> Task :buildSrc:compileGroovy FROM-CACHE
> Task :buildSrc:pluginDescriptors
> Task :buildSrc:processResources
> Task :buildSrc:classes
> Task :buildSrc:jar
> Task :buildSrc:assemble
> Task :buildSrc:spotlessGroovy FROM-CACHE
> Task :buildSrc:spotlessGroovyCheck UP-TO-DATE
> Task :buildSrc:spotlessGroovyGradle FROM-CACHE
> Task :buildSrc:spotlessGroovyGradleCheck UP-TO-DATE
> Task :buildSrc:spotlessCheck UP-TO-DATE
> Task :buildSrc:pluginUnderTestMetadata
> Task :buildSrc:compileTestJava NO-SOURCE
> Task :buildSrc:compileTestGroovy NO-SOURCE
> Task :buildSrc:processTestResources NO-SOURCE
> Task :buildSrc:testClasses UP-TO-DATE
> Task :buildSrc:test NO-SOURCE
> Task :buildSrc:validatePlugins FROM-CACHE
> Task :buildSrc:check UP-TO-DATE
> Task :buildSrc:build

> Configure project :sdks:go:test
System Go installation: /snap/bin/go is go version go1.16.13 linux/amd64; Preparing to use /home/jenkins/go/bin/go1.16.12
go1.16.12: already downloaded in /home/jenkins/sdk/go1.16.12
GOCMD=/home/jenkins/go/bin/go1.16.12

FAILURE: Build failed with an exception.

* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src/sdks/java/testing/load-tests/build.gradle'> line: 113

* What went wrong:
A problem occurred evaluating project ':sdks:java:testing:load-tests'.
> Could not get unknown property 'dockerImageName' for project ':runners:google-cloud-dataflow-java' of type org.gradle.api.Project.

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 18s
10 actionable tasks: 4 executed, 4 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/4ixgjqo44yyqm

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #227

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/227/display/redirect>

Changes:


------------------------------------------
Started by timer
Running as SYSTEM
[EnvInject] - Loading node environment variables.
Building remotely on apache-beam-jenkins-6 (beam) in workspace <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/>
The recommended git tool is: NONE
No credentials specified
Wiping out workspace first.
Cloning the remote Git repository
Cloning repository https://github.com/apache/beam.git
 > git init <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src> # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
 > git --version # timeout=10
 > git --version # 'git version 2.25.1'
 > git fetch --tags --force --progress -- https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* # timeout=10
 > git config remote.origin.url https://github.com/apache/beam.git # timeout=10
 > git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
 > git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
 > git fetch --tags --force --progress -- https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/* # timeout=10
 > git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 6c9c208197d3d74b1c3643d22716ad3b00213506 (origin/master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 6c9c208197d3d74b1c3643d22716ad3b00213506 # timeout=10
Commit message: "[BEAM-10206] Add Go Vet to Github Actions (#16612)"
 > git rev-list --no-walk 6c9c208197d3d74b1c3643d22716ad3b00213506 # timeout=10
No emails were triggered.
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content 
SPARK_LOCAL_IP=127.0.0.1
SETUPTOOLS_USE_DISTUTILS=stdlib

[EnvInject] - Variables injected successfully.
[beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11] $ /bin/bash -xe /tmp/jenkins415292196793121541.sh
+ echo '*** Load test: CoGBK 2GB 100  byte records - single key ***'
*** Load test: CoGBK 2GB 100  byte records - single key ***
[Gradle] - Launching build.
[src] $ <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src/gradlew> -PloadTest.mainClass=org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest -Prunner=:runners:google-cloud-dataflow-java '-PloadTest.args=--project=apache-beam-testing --region=us-central1 --appName=load_tests_Java11_Dataflow_V2_streaming_CoGBK_1 --tempLocation=gs://temp-storage-for-perf-tests/loadtests --influxMeasurement=java_streaming_cogbk_1 --influxTags={"runnerVersion":"v2","jdk":"java11"} --publishToInfluxDB=true --sourceOptions={"numRecords":20000000,"keySizeBytes":10,"valueSizeBytes":90,"numHotKeys":1} --coSourceOptions={"numRecords":2000000,"keySizeBytes":10,"valueSizeBytes":90,"numHotKeys":1000} --iterations=1 --numWorkers=5 --autoscalingAlgorithm=NONE --streaming=true --influxDatabase=beam_test_metrics --influxHost=http://10.128.0.96:8086 --inputWindowDurationSec=1200 --coInputWindowDurationSec=1200 --runner=DataflowRunner' -Prunner.version=V2 -PcompileAndRunTestsWithJava11 -Pjava11Home=/usr/lib/jvm/java-11-openjdk-amd64 --continue --max-****s=12 -Dorg.gradle.jvmargs=-Xms2g -Dorg.gradle.jvmargs=-Xmx4g -Dorg.gradle.vfs.watch=false -Pdocker-pull-licenses :sdks:java:testing:load-tests:run
Starting a Gradle Daemon, 3 busy Daemons could not be reused, use --status for details
Configuration on demand is an incubating feature.
> Task :buildSrc:compileJava NO-SOURCE
> Task :buildSrc:compileGroovy FROM-CACHE
> Task :buildSrc:pluginDescriptors
> Task :buildSrc:processResources
> Task :buildSrc:classes
> Task :buildSrc:jar
> Task :buildSrc:assemble
> Task :buildSrc:spotlessGroovy FROM-CACHE
> Task :buildSrc:spotlessGroovyCheck UP-TO-DATE
> Task :buildSrc:spotlessGroovyGradle FROM-CACHE
> Task :buildSrc:spotlessGroovyGradleCheck UP-TO-DATE
> Task :buildSrc:spotlessCheck UP-TO-DATE
> Task :buildSrc:pluginUnderTestMetadata
> Task :buildSrc:compileTestJava NO-SOURCE
> Task :buildSrc:compileTestGroovy NO-SOURCE
> Task :buildSrc:processTestResources NO-SOURCE
> Task :buildSrc:testClasses UP-TO-DATE
> Task :buildSrc:test NO-SOURCE
> Task :buildSrc:validatePlugins FROM-CACHE
> Task :buildSrc:check UP-TO-DATE
> Task :buildSrc:build

> Configure project :sdks:go:test
System Go installation: /snap/bin/go is go version go1.16.13 linux/amd64; Preparing to use /home/jenkins/go/bin/go1.16.12
go1.16.12: already downloaded in /home/jenkins/sdk/go1.16.12
GOCMD=/home/jenkins/go/bin/go1.16.12

FAILURE: Build failed with an exception.

* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src/sdks/java/testing/load-tests/build.gradle'> line: 113

* What went wrong:
A problem occurred evaluating project ':sdks:java:testing:load-tests'.
> Could not get unknown property 'dockerImageName' for project ':runners:google-cloud-dataflow-java' of type org.gradle.api.Project.

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 13s
10 actionable tasks: 4 executed, 4 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/44darudcfdm3c

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #226

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/226/display/redirect?page=changes>

Changes:

[Kyle Weaver] [BEAM-13751] Don't block on gcloud when attempting to get default GCP

[Kyle Weaver] [BEAM-13751] Parameterize wait timeout so test doesn't waste 2s.

[Kyle Weaver] [BEAM-13751] Add comment explaining sleep.

[noreply] Update Python SDK beam-master tags (#16630)

[noreply] Merge pull request #16592 from [BEAM-13722][Playground] Add precompiling

[noreply] Merge pull request #16505 from [BEAM-13527] [Playground] Pipeline

[noreply] [BEAM-13293] XLang Jdbc IO for Go SDK (#16111)

[noreply] [BEAM-10206] Add Go Vet to Github Actions (#16612)


------------------------------------------
Started by timer
Running as SYSTEM
[EnvInject] - Loading node environment variables.
Building remotely on apache-beam-jenkins-7 (beam) in workspace <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/>
The recommended git tool is: NONE
No credentials specified
Wiping out workspace first.
Cloning the remote Git repository
Cloning repository https://github.com/apache/beam.git
 > git init <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src> # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
 > git --version # timeout=10
 > git --version # 'git version 2.25.1'
 > git fetch --tags --force --progress -- https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* # timeout=10
 > git config remote.origin.url https://github.com/apache/beam.git # timeout=10
 > git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
 > git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
 > git fetch --tags --force --progress -- https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/* # timeout=10
 > git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 6c9c208197d3d74b1c3643d22716ad3b00213506 (origin/master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 6c9c208197d3d74b1c3643d22716ad3b00213506 # timeout=10
Commit message: "[BEAM-10206] Add Go Vet to Github Actions (#16612)"
 > git rev-list --no-walk 178cb7b65401d860b63fc7415f02f1dea2c4582f # timeout=10
No emails were triggered.
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content 
SPARK_LOCAL_IP=127.0.0.1
SETUPTOOLS_USE_DISTUTILS=stdlib

[EnvInject] - Variables injected successfully.
[beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11] $ /bin/bash -xe /tmp/jenkins1855643352193347518.sh
+ echo '*** Load test: CoGBK 2GB 100  byte records - single key ***'
*** Load test: CoGBK 2GB 100  byte records - single key ***
[Gradle] - Launching build.
[src] $ <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src/gradlew> -PloadTest.mainClass=org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest -Prunner=:runners:google-cloud-dataflow-java '-PloadTest.args=--project=apache-beam-testing --region=us-central1 --appName=load_tests_Java11_Dataflow_V2_streaming_CoGBK_1 --tempLocation=gs://temp-storage-for-perf-tests/loadtests --influxMeasurement=java_streaming_cogbk_1 --influxTags={"runnerVersion":"v2","jdk":"java11"} --publishToInfluxDB=true --sourceOptions={"numRecords":20000000,"keySizeBytes":10,"valueSizeBytes":90,"numHotKeys":1} --coSourceOptions={"numRecords":2000000,"keySizeBytes":10,"valueSizeBytes":90,"numHotKeys":1000} --iterations=1 --numWorkers=5 --autoscalingAlgorithm=NONE --streaming=true --influxDatabase=beam_test_metrics --influxHost=http://10.128.0.96:8086 --inputWindowDurationSec=1200 --coInputWindowDurationSec=1200 --runner=DataflowRunner' -Prunner.version=V2 -PcompileAndRunTestsWithJava11 -Pjava11Home=/usr/lib/jvm/java-11-openjdk-amd64 --continue --max-****s=12 -Dorg.gradle.jvmargs=-Xms2g -Dorg.gradle.jvmargs=-Xmx4g -Dorg.gradle.vfs.watch=false -Pdocker-pull-licenses :sdks:java:testing:load-tests:run
Starting a Gradle Daemon, 3 busy Daemons could not be reused, use --status for details
Configuration on demand is an incubating feature.
> Task :buildSrc:compileJava NO-SOURCE
> Task :buildSrc:compileGroovy FROM-CACHE
> Task :buildSrc:pluginDescriptors
> Task :buildSrc:processResources
> Task :buildSrc:classes
> Task :buildSrc:jar
> Task :buildSrc:assemble
> Task :buildSrc:spotlessGroovy FROM-CACHE
> Task :buildSrc:spotlessGroovyCheck UP-TO-DATE
> Task :buildSrc:spotlessGroovyGradle FROM-CACHE
> Task :buildSrc:spotlessGroovyGradleCheck UP-TO-DATE
> Task :buildSrc:spotlessCheck UP-TO-DATE
> Task :buildSrc:pluginUnderTestMetadata
> Task :buildSrc:compileTestJava NO-SOURCE
> Task :buildSrc:compileTestGroovy NO-SOURCE
> Task :buildSrc:processTestResources NO-SOURCE
> Task :buildSrc:testClasses UP-TO-DATE
> Task :buildSrc:test NO-SOURCE
> Task :buildSrc:validatePlugins FROM-CACHE
> Task :buildSrc:check UP-TO-DATE
> Task :buildSrc:build

> Configure project :sdks:go:test
System Go installation: /snap/bin/go is go version go1.16.13 linux/amd64; Preparing to use /home/jenkins/go/bin/go1.16.12
go1.16.12: already downloaded in /home/jenkins/sdk/go1.16.12
GOCMD=/home/jenkins/go/bin/go1.16.12

FAILURE: Build failed with an exception.

* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src/sdks/java/testing/load-tests/build.gradle'> line: 113

* What went wrong:
A problem occurred evaluating project ':sdks:java:testing:load-tests'.
> Could not get unknown property 'dockerImageName' for project ':runners:google-cloud-dataflow-java' of type org.gradle.api.Project.

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 14s
10 actionable tasks: 4 executed, 4 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/oigweayvbqq66

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #225

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/225/display/redirect?page=changes>

Changes:

[marcin.kuthan] Get rid of unnessecary logs for BigQuery streaming writes in

[dhuntsperger] added GitHub example references to Python multilang quickstart

[mmack] [adhoc] Test S3Options and AwsOptions for Sdk v2

[noreply] [BEAM-13537] Fix NPE in kafkatopubsub example (#16625)

[noreply] [BEAM-13740] update java_tests.yml to remove setup-go, which is

[Heejong Lee] Fix google3 import error

[noreply] [BEAM-12976] Implement Java projection pushdown optimizer. (#16513)

[noreply] Merge pull request #16579 from Revert "Revert "Merge pull request #15863

[noreply] Merge pull request #16606 from [BEAM-13247] [Playground] Embedding


------------------------------------------
Started by timer
Running as SYSTEM
[EnvInject] - Loading node environment variables.
Building remotely on apache-beam-jenkins-15 (beam) in workspace <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/>
The recommended git tool is: NONE
No credentials specified
Wiping out workspace first.
Cloning the remote Git repository
Cloning repository https://github.com/apache/beam.git
 > git init <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src> # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
 > git --version # timeout=10
 > git --version # 'git version 2.7.4'
 > git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* # timeout=10
 > git config remote.origin.url https://github.com/apache/beam.git # timeout=10
 > git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
 > git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
 > git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/* # timeout=10
 > git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 178cb7b65401d860b63fc7415f02f1dea2c4582f (origin/master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 178cb7b65401d860b63fc7415f02f1dea2c4582f # timeout=10
Commit message: "Merge pull request #16606 from [BEAM-13247] [Playground] Embedding iframe"
 > git rev-list --no-walk f687ece82d3623062fb1ab0f7b3e1366638ad867 # timeout=10
No emails were triggered.
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content 
SPARK_LOCAL_IP=127.0.0.1
SETUPTOOLS_USE_DISTUTILS=stdlib

[EnvInject] - Variables injected successfully.
[beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11] $ /bin/bash -xe /tmp/jenkins321842524113068946.sh
+ echo '*** Load test: CoGBK 2GB 100  byte records - single key ***'
*** Load test: CoGBK 2GB 100  byte records - single key ***
[Gradle] - Launching build.
[src] $ <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src/gradlew> -PloadTest.mainClass=org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest -Prunner=:runners:google-cloud-dataflow-java '-PloadTest.args=--project=apache-beam-testing --region=us-central1 --appName=load_tests_Java11_Dataflow_V2_streaming_CoGBK_1 --tempLocation=gs://temp-storage-for-perf-tests/loadtests --influxMeasurement=java_streaming_cogbk_1 --influxTags={"runnerVersion":"v2","jdk":"java11"} --publishToInfluxDB=true --sourceOptions={"numRecords":20000000,"keySizeBytes":10,"valueSizeBytes":90,"numHotKeys":1} --coSourceOptions={"numRecords":2000000,"keySizeBytes":10,"valueSizeBytes":90,"numHotKeys":1000} --iterations=1 --numWorkers=5 --autoscalingAlgorithm=NONE --streaming=true --influxDatabase=beam_test_metrics --influxHost=http://10.128.0.96:8086 --inputWindowDurationSec=1200 --coInputWindowDurationSec=1200 --runner=DataflowRunner' -Prunner.version=V2 -PcompileAndRunTestsWithJava11 -Pjava11Home=/usr/lib/jvm/java-11-openjdk-amd64 --continue --max-****s=12 -Dorg.gradle.jvmargs=-Xms2g -Dorg.gradle.jvmargs=-Xmx4g -Dorg.gradle.vfs.watch=false -Pdocker-pull-licenses :sdks:java:testing:load-tests:run
Starting a Gradle Daemon, 3 busy Daemons could not be reused, use --status for details
Configuration on demand is an incubating feature.
> Task :buildSrc:compileJava NO-SOURCE
> Task :buildSrc:compileGroovy FROM-CACHE
> Task :buildSrc:pluginDescriptors
> Task :buildSrc:processResources
> Task :buildSrc:classes
> Task :buildSrc:jar
> Task :buildSrc:assemble
> Task :buildSrc:spotlessGroovy FROM-CACHE
> Task :buildSrc:spotlessGroovyCheck UP-TO-DATE
> Task :buildSrc:spotlessGroovyGradle FROM-CACHE
> Task :buildSrc:spotlessGroovyGradleCheck UP-TO-DATE
> Task :buildSrc:spotlessCheck UP-TO-DATE
> Task :buildSrc:pluginUnderTestMetadata
> Task :buildSrc:compileTestJava NO-SOURCE
> Task :buildSrc:compileTestGroovy NO-SOURCE
> Task :buildSrc:processTestResources NO-SOURCE
> Task :buildSrc:testClasses UP-TO-DATE
> Task :buildSrc:test NO-SOURCE
> Task :buildSrc:validatePlugins FROM-CACHE
> Task :buildSrc:check UP-TO-DATE
> Task :buildSrc:build

> Configure project :sdks:go:test
System Go installation: /usr/bin/go is go version go1.16.13 linux/amd64; Preparing to use /home/jenkins/go/bin/go1.16.12
go1.16.12: already downloaded in /home/jenkins/sdk/go1.16.12
GOCMD=/home/jenkins/go/bin/go1.16.12

FAILURE: Build failed with an exception.

* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src/sdks/java/testing/load-tests/build.gradle'> line: 113

* What went wrong:
A problem occurred evaluating project ':sdks:java:testing:load-tests'.
> Could not get unknown property 'dockerImageName' for project ':runners:google-cloud-dataflow-java' of type org.gradle.api.Project.

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 15s
10 actionable tasks: 4 executed, 4 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/gaqq5xefmbvpi

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #224

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/224/display/redirect?page=changes>

Changes:

[Heejong Lee] [BEAM-13093] Enable JavaUsingPython CrossLanguageValidateRunner test for

[mmack] [BEAM-13746] Fix deserialization of SSECustomerKey for AWS Sdk v2

[noreply] [BEAM-7928] Allow users to specify worker disk type for Dataflow runner

[noreply] Merge pull request #16534 from [BEAM-13671][Playground] Add backend

[noreply] [BEAM-13271] Bump errorprone to 2.10.0 (#16231)

[noreply] [BEAM-13595] Don't load main session when cloudpickle is used. (#16589)

[Heejong Lee] Update readme for XVR tests


------------------------------------------
Started by timer
Running as SYSTEM
[EnvInject] - Loading node environment variables.
Building remotely on apache-beam-jenkins-5 (beam) in workspace <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/>
The recommended git tool is: NONE
No credentials specified
Wiping out workspace first.
Cloning the remote Git repository
Cloning repository https://github.com/apache/beam.git
 > git init <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src> # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
 > git --version # timeout=10
 > git --version # 'git version 2.25.1'
 > git fetch --tags --force --progress -- https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* # timeout=10
 > git config remote.origin.url https://github.com/apache/beam.git # timeout=10
 > git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
 > git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
 > git fetch --tags --force --progress -- https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/* # timeout=10
 > git rev-parse origin/master^{commit} # timeout=10
Checking out Revision f687ece82d3623062fb1ab0f7b3e1366638ad867 (origin/master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f f687ece82d3623062fb1ab0f7b3e1366638ad867 # timeout=10
Commit message: "Merge pull request #16626 from ihji/update_readme"
 > git rev-list --no-walk 41d585f82b10195f758d14e3a54076ea1f05aa75 # timeout=10
No emails were triggered.
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content 
SPARK_LOCAL_IP=127.0.0.1
SETUPTOOLS_USE_DISTUTILS=stdlib

[EnvInject] - Variables injected successfully.
[beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11] $ /bin/bash -xe /tmp/jenkins3078908429813086756.sh
+ echo '*** Load test: CoGBK 2GB 100  byte records - single key ***'
*** Load test: CoGBK 2GB 100  byte records - single key ***
[Gradle] - Launching build.
[src] $ <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src/gradlew> -PloadTest.mainClass=org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest -Prunner=:runners:google-cloud-dataflow-java '-PloadTest.args=--project=apache-beam-testing --region=us-central1 --appName=load_tests_Java11_Dataflow_V2_streaming_CoGBK_1 --tempLocation=gs://temp-storage-for-perf-tests/loadtests --influxMeasurement=java_streaming_cogbk_1 --influxTags={"runnerVersion":"v2","jdk":"java11"} --publishToInfluxDB=true --sourceOptions={"numRecords":20000000,"keySizeBytes":10,"valueSizeBytes":90,"numHotKeys":1} --coSourceOptions={"numRecords":2000000,"keySizeBytes":10,"valueSizeBytes":90,"numHotKeys":1000} --iterations=1 --numWorkers=5 --autoscalingAlgorithm=NONE --streaming=true --influxDatabase=beam_test_metrics --influxHost=http://10.128.0.96:8086 --inputWindowDurationSec=1200 --coInputWindowDurationSec=1200 --runner=DataflowRunner' -Prunner.version=V2 -PcompileAndRunTestsWithJava11 -Pjava11Home=/usr/lib/jvm/java-11-openjdk-amd64 --continue --max-****s=12 -Dorg.gradle.jvmargs=-Xms2g -Dorg.gradle.jvmargs=-Xmx4g -Dorg.gradle.vfs.watch=false -Pdocker-pull-licenses :sdks:java:testing:load-tests:run
Starting a Gradle Daemon, 3 busy Daemons could not be reused, use --status for details
Configuration on demand is an incubating feature.
> Task :buildSrc:compileJava NO-SOURCE
> Task :buildSrc:compileGroovy FROM-CACHE
> Task :buildSrc:pluginDescriptors
> Task :buildSrc:processResources
> Task :buildSrc:classes
> Task :buildSrc:jar
> Task :buildSrc:assemble
> Task :buildSrc:spotlessGroovy FROM-CACHE
> Task :buildSrc:spotlessGroovyCheck UP-TO-DATE
> Task :buildSrc:spotlessGroovyGradle FROM-CACHE
> Task :buildSrc:spotlessGroovyGradleCheck UP-TO-DATE
> Task :buildSrc:spotlessCheck UP-TO-DATE
> Task :buildSrc:pluginUnderTestMetadata
> Task :buildSrc:compileTestJava NO-SOURCE
> Task :buildSrc:compileTestGroovy NO-SOURCE
> Task :buildSrc:processTestResources NO-SOURCE
> Task :buildSrc:testClasses UP-TO-DATE
> Task :buildSrc:test NO-SOURCE
> Task :buildSrc:validatePlugins FROM-CACHE
> Task :buildSrc:check UP-TO-DATE
> Task :buildSrc:build

> Configure project :sdks:go:test
System Go installation: /snap/bin/go is go version go1.16.13 linux/amd64; Preparing to use /home/jenkins/go/bin/go1.16.12
go1.16.12: already downloaded in /home/jenkins/sdk/go1.16.12
GOCMD=/home/jenkins/go/bin/go1.16.12

FAILURE: Build failed with an exception.

* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src/sdks/java/testing/load-tests/build.gradle'> line: 113

* What went wrong:
A problem occurred evaluating project ':sdks:java:testing:load-tests'.
> Could not get unknown property 'dockerImageName' for project ':runners:google-cloud-dataflow-java' of type org.gradle.api.Project.

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 15s
10 actionable tasks: 4 executed, 4 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/vb2eyhfand7pm

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #223

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/223/display/redirect?page=changes>

Changes:

[artur.khanin] Privacy policy update regarding Apache Beam Playground

[Daniel Oliveira] [BEAM-13321] Fix exception with BigQuery StreamWriter TraceID.

[mmack] [BEAM-8807] Add integration test for SnsIO.write (Sdk v1 & v2)

[noreply] [BEAM-13736] Make lifting cache exact. (#16603)

[noreply] Merge pull request #16565 from [BEAM-13692][Playground]  Implement

[noreply] Merge pull request #16502 from [BEAM-13650][Playground] Add link for

[noreply] [BEAM-13310] remove call to get offset consumer config, which was rep…


------------------------------------------
[...truncated 51.41 KB...]
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Jan 26, 2022 12:46:10 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 204 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Jan 26, 2022 12:46:11 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 204 files cached, 0 files newly uploaded in 0 seconds
Jan 26, 2022 12:46:11 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Jan 26, 2022 12:46:11 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <114075 bytes, hash 2a52c358ab6afb015385470d26cfb618c296309cb5c727e96afee5f468d69acb> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-KlLDWKtq-wFThUcNJs-2GMKWMJy1xyfpav7l9GjWmss.pb
Jan 26, 2022 12:46:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Jan 26, 2022 12:46:13 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1947596f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3078cac, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@f6de586, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5f2bd6d9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7c847072, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43d9f1a2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@23f86d8a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2264ea32, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7d3c09ec, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@71e4b308, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@11900483, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@14a049f9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@94e51e8, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5de6cf3a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4cc36c19, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5a3a1bf9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2100d047, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4af45442, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1d805aa1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@176f7f3b]
Jan 26, 2022 12:46:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Jan 26, 2022 12:46:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Jan 26, 2022 12:46:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Jan 26, 2022 12:46:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Jan 26, 2022 12:46:13 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1df1ced0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5349b246, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@32b0876c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2aaf152b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@39bbd9e0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@27fe9713, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@11c3ff67, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4397a639, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@24841372, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77114efe, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@79a7bfbc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77f68df, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3e4e4c1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7e7f3cfd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ae126d1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@46a488c2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6242ae3b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@65ddee5a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@56399b9e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@34b9eb03]
Jan 26, 2022 12:46:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Jan 26, 2022 12:46:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Jan 26, 2022 12:46:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Jan 26, 2022 12:46:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Jan 26, 2022 12:46:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Jan 26, 2022 12:46:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Jan 26, 2022 12:46:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Jan 26, 2022 12:46:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Jan 26, 2022 12:46:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Jan 26, 2022 12:46:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Jan 26, 2022 12:46:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Jan 26, 2022 12:46:13 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.37.0-SNAPSHOT
Jan 26, 2022 12:46:14 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-01-26_04_46_13-13297979137814239911?project=apache-beam-testing
Jan 26, 2022 12:46:14 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2022-01-26_04_46_13-13297979137814239911
Jan 26, 2022 12:46:14 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2022-01-26_04_46_13-13297979137814239911
Jan 26, 2022 12:46:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2022-01-26T12:46:20.704Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-01-c0rs. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Jan 26, 2022 12:46:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-26T12:46:25.011Z: Worker configuration: e2-standard-2 in us-central1-b.
Jan 26, 2022 12:46:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-26T12:46:25.443Z: Expanding SplittableParDo operations into optimizable parts.
Jan 26, 2022 12:46:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-26T12:46:25.494Z: Expanding CollectionToSingleton operations into optimizable parts.
Jan 26, 2022 12:46:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-26T12:46:25.562Z: Expanding CoGroupByKey operations into optimizable parts.
Jan 26, 2022 12:46:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-26T12:46:25.647Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Jan 26, 2022 12:46:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-26T12:46:25.683Z: Expanding GroupByKey operations into streaming Read/Write steps
Jan 26, 2022 12:46:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-26T12:46:25.746Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jan 26, 2022 12:46:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-26T12:46:25.857Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jan 26, 2022 12:46:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-26T12:46:25.880Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Jan 26, 2022 12:46:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-26T12:46:25.911Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Jan 26, 2022 12:46:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-26T12:46:25.945Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Jan 26, 2022 12:46:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-26T12:46:25.977Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Jan 26, 2022 12:46:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-26T12:46:26.024Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jan 26, 2022 12:46:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-26T12:46:26.052Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jan 26, 2022 12:46:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-26T12:46:26.077Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jan 26, 2022 12:46:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-26T12:46:26.111Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Jan 26, 2022 12:46:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-26T12:46:26.143Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Jan 26, 2022 12:46:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-26T12:46:26.166Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Jan 26, 2022 12:46:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-26T12:46:26.198Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Jan 26, 2022 12:46:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-26T12:46:26.235Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jan 26, 2022 12:46:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-26T12:46:26.267Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jan 26, 2022 12:46:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-26T12:46:26.300Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jan 26, 2022 12:46:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-26T12:46:26.334Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Jan 26, 2022 12:46:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-26T12:46:26.368Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Jan 26, 2022 12:46:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-26T12:46:26.394Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Jan 26, 2022 12:46:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-26T12:46:26.419Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Jan 26, 2022 12:46:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-26T12:46:26.450Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Jan 26, 2022 12:46:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-26T12:46:26.497Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Jan 26, 2022 12:46:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-26T12:46:26.520Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Jan 26, 2022 12:46:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-26T12:46:26.553Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Jan 26, 2022 12:46:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-26T12:46:26.914Z: Starting 5 ****s in us-central1-b...
Jan 26, 2022 12:46:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-26T12:46:55.412Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jan 26, 2022 12:47:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-26T12:47:15.413Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Jan 26, 2022 12:48:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-26T12:48:13.892Z: Workers have started successfully.
Jan 26, 2022 12:48:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-26T12:48:13.920Z: Workers have started successfully.
Jan 26, 2022 4:00:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-26T16:00:34.523Z: Cancel request is committed for workflow job: 2022-01-26_04_46_13-13297979137814239911.
Jan 26, 2022 4:00:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-26T16:00:34.606Z: Cleaning up.
Jan 26, 2022 4:00:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-26T16:00:34.677Z: Stopping **** pool...
Jan 26, 2022 4:00:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-26T16:00:34.732Z: Stopping **** pool...
Jan 26, 2022 4:03:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-26T16:02:59.898Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Jan 26, 2022 4:03:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-26T16:02:59.936Z: Worker pool stopped.
Jan 26, 2022 4:03:05 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2022-01-26_04_46_13-13297979137814239911 finished with status CANCELLED.
Load test results for test (ID): 6278227b-1960-46af-85ae-97f03b8b4c29 and timestamp: 2022-01-26T12:46:07.536000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11427.155
dataflow_v2_java11_total_bytes_count             2.08754957E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220126124335
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:1eac405e6df8d224c616f83211c43eaeed5b86646f4dd0756ea0b82b8e196b3e
Deleted: sha256:9eb22c9566ce29161db940201866b381d4cfedcfadf02e7fb19608974f358eb7
Deleted: sha256:ab0e4583e6362d9fe7fb14f0df2436a45c703cbcda7231018cfe4e220ade0408
Deleted: sha256:12a9587fd3e513dc7e7a3fe943f72cd5df0dfc8ec8bdafcf6b1c199a6d54ac6a
Deleted: sha256:d6b51961f5cc54f863c8ec9f767081d334a09d4d61ab010f1c7816009e67f9e1
Deleted: sha256:700e4b274581b80b68722a5793a0179911697a48ddcee8099aa3a4f5bfd9b038
Deleted: sha256:1d9167783adb49dc0dcd57734f8fe0e08a324e7c7328c17e19a5ab3d93b7d77a
Deleted: sha256:2abd4cdffdfa74df65ea4a49f892326a65646b73484c5a39c48fa8b7911f16a1
Deleted: sha256:81140e570eb0faa7972b47c473cfcd46bccfb62c0669ebf1ec5e56e10c173768
Deleted: sha256:954f5e699208af0d61ef61f0ba5085ae1b7d6505b43d7d925dfadeb65d81e0b6
Deleted: sha256:fbd0feea80516223cfbff3e5e805f96236b1d9e14cc96ca7576d7ed388fa6ca2
Deleted: sha256:582e09d9f53dde9911134431821e898c60b972ec3c0b26663a3119885aed8489
Deleted: sha256:58084b5bb8e5382b7061da7146330e598313fb3131c9e87c20bc990e3c4f082e
Deleted: sha256:b989466baf5b1445f3d90b1f3b4d88cf1423860e106e5e85decec9428ced1b87
Deleted: sha256:203e4d209b4ec63c167cb8c5749d977f3a6c71dfb84b21fd6f2593d788ca3816
Deleted: sha256:f6f10a620a64d30963cbaf4888574ff79cce7e69d75f1a71bf9f00730acddd47
Deleted: sha256:ffd55aa6f7e45b8f943095b3e745e03cd4b6777f65f7ed49c9bf28eaa870af2f
Deleted: sha256:8af93658b6de747b2fdf2e3b956d3da5a00a833aefcfee84fce816aa8ba5d302
Deleted: sha256:0dade9d6e684b8b3191fd8209e6061b0f9a62800934311e3692d018e594a2883
Deleted: sha256:76f9f9d0271b9514526a70aaa3025c10ee9c5b6a5d856019b670cc9b587fab3f
Deleted: sha256:eb9b1d2640653af3426feae0bda64acb05e15ea318bc52c5ef724c3821908591
Deleted: sha256:e1bfbf9615f64429e0a507424cce9d14e45de72faa231a0b4917fd8f89bb2a1a
Deleted: sha256:5881013990107ab03b4b0203e40f0263d1ae639a1d5f68dd611cffc3a0a7e48d
Deleted: sha256:9bd84a28ab6f85af099c5781b18ead896dd53cb11e2c16bdc7aa95b427f2c0fe
Deleted: sha256:70c0ce5a7d406fda4fe5829e6429f9b57722fbe0afa0a5bf6eda90db1fd76f2b
Deleted: sha256:818617bb409153ff930968176f51ad6091a33fa52070657a1feb976662ae54f0
Deleted: sha256:d79a9d294b47df08825061ad0c89c65543f7f9f97bd4465f3aed01e96aededea
Deleted: sha256:df2fd65bfcb4307f90bb9fdf9613c3f24f5ea480c9b96558e46d7398c1997066
Deleted: sha256:2bf13fa4ccd639fa8d5e4ef69084a865d8431729baa6deceaddfbbf93be05a88
Deleted: sha256:2cbf787ec6a5c43d9fc3339986daf9f0aa8194268c5a4f3fbfee71f6fc82abba
Deleted: sha256:5c2fae66fbfca2ede4897e8ba8ca5fa96137e0e187d56a7937a04229d3d09582
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220126124335]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:1eac405e6df8d224c616f83211c43eaeed5b86646f4dd0756ea0b82b8e196b3e]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220126124335] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:1eac405e6df8d224c616f83211c43eaeed5b86646f4dd0756ea0b82b8e196b3e])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:1eac405e6df8d224c616f83211c43eaeed5b86646f4dd0756ea0b82b8e196b3e
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:1eac405e6df8d224c616f83211c43eaeed5b86646f4dd0756ea0b82b8e196b3e
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:1eac405e6df8d224c616f83211c43eaeed5b86646f4dd0756ea0b82b8e196b3e].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:c07e8f47c4537b9f91d3814dfe4b4fbed465cd40fe06f32257aed502f7fb4bb6
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:c07e8f47c4537b9f91d3814dfe4b4fbed465cd40fe06f32257aed502f7fb4bb6
ERROR: (gcloud.container.images.delete) Not found: response: {'status': '404', 'content-length': '168', 'x-xss-protection': '0', 'transfer-encoding': 'chunked', 'server': 'Docker Registry', '-content-encoding': 'gzip', 'docker-distribution-api-version': 'registry/2.0', 'cache-control': 'private', 'date': 'Wed, 26 Jan 2022 16:03:10 GMT', 'x-frame-options': 'SAMEORIGIN', 'content-type': 'application/json'}
Failed to compute blob liveness for manifest: 'sha256:c07e8f47c4537b9f91d3814dfe4b4fbed465cd40fe06f32257aed502f7fb4bb6': None

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages FAILED

FAILURE: Build completed with 2 failures.

1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
==============================================================================

2: Task failed with an exception.
-----------
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src/runners/google-cloud-dataflow-java/build.gradle'> line: 290

* What went wrong:
Execution failed for task ':runners:google-cloud-dataflow-java:cleanUpDockerImages'.
> Process 'command './scripts/cleanup_untagged_gcr_images.sh'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
==============================================================================

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings

Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.

BUILD FAILED in 3h 19m 49s
109 actionable tasks: 72 executed, 33 from cache, 4 up-to-date

Publishing build scan...
https://gradle.com/s/5umerf77ijq56

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #222

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/222/display/redirect?page=changes>

Changes:

[Heejong Lee] [BEAM-13716] Clear before creating a new virtual environment in

[mmack] [BEAM-13653] Make SnsIO.write topicArn optional. If provided, validate

[noreply] [BEAM-10897] Update the fastavro lower bound due to an issue on Windows

[noreply] [BEAM-13605] Update pandas_doctests_test denylists in preparation for

[noreply] Merge pull request #16538 from [BEAM-13676][Playground][Bugfix]Build Of

[noreply] Merge pull request #16582 from [BEAM-13711] [Playground] [Bugfix] Add

[noreply] Merge pull request #16515 from [BEAM-13636] [Playground] Checking the

[ningkang0957] [BEAM-13275] Removed the explicit selenium dependency from setup

[noreply] [BEAM-10206] Deprecate unused shallow cloning functions (#16600)

[noreply] Bump Dataflow container versions (#16602)

[noreply] Improved multi-language pipelines section of the programming guide

[mmack] [BEAM-13510] Don't retry on invalid SQS receipt handles.


------------------------------------------
[...truncated 53.48 KB...]
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@f6de586, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5f2bd6d9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7c847072, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43d9f1a2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@23f86d8a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2264ea32, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7d3c09ec, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@71e4b308, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@11900483, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@14a049f9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@94e51e8, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5de6cf3a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4cc36c19, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5a3a1bf9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2100d047, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4af45442, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1d805aa1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@176f7f3b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@30ca0779, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58740366]
Jan 25, 2022 12:46:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Jan 25, 2022 12:46:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Jan 25, 2022 12:46:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Jan 25, 2022 12:46:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Jan 25, 2022 12:46:13 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@32b0876c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2aaf152b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@39bbd9e0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@27fe9713, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@11c3ff67, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4397a639, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@24841372, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77114efe, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@79a7bfbc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77f68df, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3e4e4c1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7e7f3cfd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ae126d1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@46a488c2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6242ae3b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@65ddee5a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@56399b9e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@34b9eb03, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43fda8d9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@49d831c2]
Jan 25, 2022 12:46:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Jan 25, 2022 12:46:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Jan 25, 2022 12:46:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Jan 25, 2022 12:46:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Jan 25, 2022 12:46:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Jan 25, 2022 12:46:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Jan 25, 2022 12:46:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Jan 25, 2022 12:46:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Jan 25, 2022 12:46:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Jan 25, 2022 12:46:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Jan 25, 2022 12:46:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Jan 25, 2022 12:46:13 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.37.0-SNAPSHOT
Jan 25, 2022 12:46:15 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-01-25_04_46_13-6663904842635572357?project=apache-beam-testing
Jan 25, 2022 12:46:15 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2022-01-25_04_46_13-6663904842635572357
Jan 25, 2022 12:46:15 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2022-01-25_04_46_13-6663904842635572357
Jan 25, 2022 12:46:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2022-01-25T12:46:21.757Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-01-qyct. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Jan 25, 2022 12:46:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-25T12:46:25.608Z: Worker configuration: e2-standard-2 in us-central1-b.
Jan 25, 2022 12:46:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-25T12:46:26.253Z: Expanding SplittableParDo operations into optimizable parts.
Jan 25, 2022 12:46:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-25T12:46:26.288Z: Expanding CollectionToSingleton operations into optimizable parts.
Jan 25, 2022 12:46:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-25T12:46:26.350Z: Expanding CoGroupByKey operations into optimizable parts.
Jan 25, 2022 12:46:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-25T12:46:26.406Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Jan 25, 2022 12:46:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-25T12:46:26.434Z: Expanding GroupByKey operations into streaming Read/Write steps
Jan 25, 2022 12:46:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-25T12:46:26.513Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jan 25, 2022 12:46:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-25T12:46:26.636Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jan 25, 2022 12:46:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-25T12:46:26.673Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Jan 25, 2022 12:46:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-25T12:46:26.704Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Jan 25, 2022 12:46:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-25T12:46:26.738Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Jan 25, 2022 12:46:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-25T12:46:26.777Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Jan 25, 2022 12:46:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-25T12:46:26.810Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jan 25, 2022 12:46:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-25T12:46:26.856Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jan 25, 2022 12:46:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-25T12:46:26.881Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jan 25, 2022 12:46:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-25T12:46:26.914Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Jan 25, 2022 12:46:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-25T12:46:26.953Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Jan 25, 2022 12:46:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-25T12:46:26.991Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Jan 25, 2022 12:46:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-25T12:46:27.043Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Jan 25, 2022 12:46:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-25T12:46:27.079Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jan 25, 2022 12:46:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-25T12:46:27.113Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jan 25, 2022 12:46:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-25T12:46:27.148Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jan 25, 2022 12:46:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-25T12:46:27.184Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Jan 25, 2022 12:46:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-25T12:46:27.226Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Jan 25, 2022 12:46:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-25T12:46:27.255Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Jan 25, 2022 12:46:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-25T12:46:27.292Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Jan 25, 2022 12:46:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-25T12:46:27.318Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Jan 25, 2022 12:46:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-25T12:46:27.348Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Jan 25, 2022 12:46:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-25T12:46:27.382Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Jan 25, 2022 12:46:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-25T12:46:27.465Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Jan 25, 2022 12:46:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-25T12:46:27.811Z: Starting 5 ****s in us-central1-b...
Jan 25, 2022 12:46:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-25T12:46:36.867Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jan 25, 2022 12:47:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-25T12:47:12.308Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Jan 25, 2022 12:48:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-25T12:48:13.834Z: Workers have started successfully.
Jan 25, 2022 12:48:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-25T12:48:13.864Z: Workers have started successfully.
Jan 25, 2022 12:57:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-25T12:57:12.740Z: generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
Jan 25, 2022 4:00:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-25T16:00:26.302Z: Cancel request is committed for workflow job: 2022-01-25_04_46_13-6663904842635572357.
Jan 25, 2022 4:00:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-25T16:00:26.511Z: Cleaning up.
Jan 25, 2022 4:00:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-25T16:00:26.581Z: Stopping **** pool...
Jan 25, 2022 4:00:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-25T16:00:26.633Z: Stopping **** pool...
Jan 25, 2022 4:02:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-25T16:02:41.654Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Jan 25, 2022 4:02:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-25T16:02:41.686Z: Worker pool stopped.
Jan 25, 2022 4:02:47 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2022-01-25_04_46_13-6663904842635572357 finished with status CANCELLED.
Load test results for test (ID): 92761e44-82f7-4e56-8992-aea01f9be228 and timestamp: 2022-01-25T12:46:07.711000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11446.062
dataflow_v2_java11_total_bytes_count             1.68872296E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220125124338
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:a4047dda4444d67d6a79d9e4bba2dc5e29b61bfef6cfed4ca686031033c6c190
Deleted: sha256:9babd310ad459a08379a87217c495ab375b22d0707ff1ebe868036b9c3292b05
Deleted: sha256:fdcdbd43456f38101571bb73e7bfcaee4b829b9e2a68f2532393f595b6fe6b79
Deleted: sha256:4424635fdc69d5ae2fb391b25a1c1e423160dd42a25cfb1ba187825910bff23b
Deleted: sha256:3d4686a86ced7cfaaad797fcaba0f3ad6a8901c167e7ca4a48377910957c133d
Deleted: sha256:3d21f2ee0e4aa3d5f0b74bf962f4e858d405837fab8e1b6d1b26715b468930c3
Deleted: sha256:525e159289e730cd9adbd2f2157de76a010ef0c43df60ba954ffd55ae3437bc0
Deleted: sha256:69d76b712a50feb163b9d8df9aeff5ab8103821ae7f7810a47305b606d13eb60
Deleted: sha256:ac033b1867d44807d556004484da2037edebcfade0bbb88c820211f8039d5019
Deleted: sha256:6508accac77e7a1c89ec7bd6bf55fb27a6cea46802da8617cacbb9766118c1da
Deleted: sha256:c735ff24a9b7c07eb90da151024c50349187dd5d2612ccf568f8957b30c8e613
Deleted: sha256:5fdf57413c5acf1b9d9040629c40f8a6b642929762d46015edfc6809fc907875
Deleted: sha256:7ec8d1d04af15dd280804054233d9f7259975a79610c9b8eb443c65f91a1c6ba
Deleted: sha256:66ab16f9abd991c18e7a0fa1cddde8e7dbe22cfb12abd7c79cf747895d62216a
Deleted: sha256:acff4ebd6afa616703d982051100bb81097e3d3cb0e1161232c90e3797132d00
Deleted: sha256:82312c4331d2f775f9cee595ec24855ba7d4c03b1e495cd4db950146e9255cb3
Deleted: sha256:f0635a4b22b34abcc55ccb1a0d36a90bc09bb1ac4fe4eb794786dc364a5b1a8e
Deleted: sha256:517d460cfd6f455ef29e5aa1275861751ebf43eaf3f20de9600310f959a2b1b1
Deleted: sha256:cde7f1dc4c3d74430a6722aaf1b809fc9982fc950097e88fc0f2fc6342c98c49
Deleted: sha256:560cf8e48e29871552f76cc0186865d0e572b0a66260d1894dab6fcfeeab8347
Deleted: sha256:6c299191677a6afff0f99936e7c00d29680f0d277b3aa9eb925b9ed013df1de4
Deleted: sha256:b850cae7484e4c0526121c5b249c193472178d2c3c9c114eb260e9f948f4efc3
Deleted: sha256:522c983e836c163aa2fb8248b14702e35030ef3da0088fc41104440ef5f6f8ac
Deleted: sha256:d121b241bd4c7470989736c007f1318847c02f920835aa78f5fdef93a177ebc3
Deleted: sha256:076a1057f8ceb9a28bcd361fc0c15b19b322b8c973ac9798896105cb3356755f
Deleted: sha256:18d2f345ead2da5d9ecc0effd0c73ad5b3009f3d2fd853989f4223cc9f4c1845
Deleted: sha256:f4949d04a5d638756cc60880f8a2b9f0a993138bc536d5b74fb1d992cc80569c
Deleted: sha256:2210480836fbe9670830e279fe73cd4387e21ce3862d7a17469d81729bacb8ca
Deleted: sha256:5a03267d766826dbb5a4ac2898563a949356b4ba6e421967ad8f2c92dd4b833a
Deleted: sha256:058367e555d4e5c16f4d3ddb7a5fd117744ada6d5f937d18b07ec4a661590351
Deleted: sha256:96828d0f54da6dbacde273408bede56721aee594b5234a4d659b1bf3f094e257
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220125124338]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:a4047dda4444d67d6a79d9e4bba2dc5e29b61bfef6cfed4ca686031033c6c190]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220125124338] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:a4047dda4444d67d6a79d9e4bba2dc5e29b61bfef6cfed4ca686031033c6c190])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:a4047dda4444d67d6a79d9e4bba2dc5e29b61bfef6cfed4ca686031033c6c190
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:a4047dda4444d67d6a79d9e4bba2dc5e29b61bfef6cfed4ca686031033c6c190
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:a4047dda4444d67d6a79d9e4bba2dc5e29b61bfef6cfed4ca686031033c6c190].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings

Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.

BUILD FAILED in 3h 19m 31s
109 actionable tasks: 74 executed, 31 from cache, 4 up-to-date

Publishing build scan...
https://gradle.com/s/qkqi7i2oavhqs

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #221

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/221/display/redirect>

Changes:


------------------------------------------
[...truncated 46.88 KB...]
golang.org/x/text,https://go.googlesource.com/text/+/refs/heads/master/LICENSE,BSD-3-Clause
github.com/apache/beam/sdks/java/container,https://github.com/apache/beam/blob/master/LICENSE,Apache-2.0
github.com/apache/beam/sdks/go/pkg/beam,https://github.com/apache/beam/blob/master/sdks/go/README.md,Apache-2.0
golang.org/x/net,https://go.googlesource.com/net/+/refs/heads/master/LICENSE,BSD-3-Clause
+ chmod -R a+w /output/licenses

> Task :release:go-licenses:java:createLicenses
> Task :sdks:java:container:java11:copyGolangLicenses
> Task :sdks:java:container:java11:dockerPrepare
> Task :sdks:java:container:java11:docker

> Task :runners:google-cloud-dataflow-java:buildAndPushDockerContainer
WARNING: `gcloud docker` will not be supported for Docker client versions above 18.03.

As an alternative, use `gcloud auth configure-docker` to configure `docker` to
use `gcloud` as a credential helper, then use `docker` as you would for non-GCR
registries, e.g. `docker pull gcr.io/project-id/my-image`. Add
`--verbosity=error` to silence this warning: `gcloud docker
--verbosity=error -- pull gcr.io/project-id/my-image`.

See: https://cloud.google.com/container-registry/docs/support/deprecation-notices#gcloud-docker

The push refers to repository [us.gcr.io/apache-beam-testing/java-postcommit-it/java]
08881f8e5634: Preparing
3295642dd35d: Preparing
7f300e65609e: Preparing
33a2eb14bd4a: Preparing
3afd30fba654: Preparing
d2f1cdf0d861: Preparing
cd71ad19f1c7: Preparing
d5fb0eb74494: Preparing
4e28ff0d6619: Preparing
488f8b6417f7: Preparing
10732ed4a521: Preparing
8d4146ffe202: Preparing
17482a157a9b: Preparing
3bb5258f46d2: Preparing
832e177bb500: Preparing
f9e18e59a565: Preparing
d5fb0eb74494: Waiting
cd71ad19f1c7: Waiting
26a504e63be4: Preparing
8bf42db0de72: Preparing
4e28ff0d6619: Waiting
488f8b6417f7: Waiting
10732ed4a521: Waiting
31892cc314cb: Preparing
11936051f93b: Preparing
17482a157a9b: Waiting
3bb5258f46d2: Waiting
26a504e63be4: Waiting
832e177bb500: Waiting
f9e18e59a565: Waiting
8d4146ffe202: Waiting
11936051f93b: Waiting
8bf42db0de72: Waiting
3afd30fba654: Pushed
3295642dd35d: Pushed
7f300e65609e: Pushed
33a2eb14bd4a: Pushed
d2f1cdf0d861: Pushed
08881f8e5634: Pushed
d5fb0eb74494: Pushed
488f8b6417f7: Pushed
cd71ad19f1c7: Pushed
3bb5258f46d2: Layer already exists
8d4146ffe202: Pushed
f9e18e59a565: Layer already exists
832e177bb500: Layer already exists
26a504e63be4: Layer already exists
4e28ff0d6619: Pushed
8bf42db0de72: Layer already exists
31892cc314cb: Layer already exists
11936051f93b: Layer already exists
17482a157a9b: Pushed
10732ed4a521: Pushed
20220124124331: digest: sha256:389ec7cdac55c8dcc0448eea26d529f42361cd80a48e2ecfd4a8ad52b23834df size: 4520

> Task :sdks:java:testing:load-tests:run
Jan 24, 2022 12:45:29 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Jan 24, 2022 12:45:29 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 204 files. Enable logging at DEBUG level to see which files will be staged.
Jan 24, 2022 12:45:30 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Jan 24, 2022 12:45:30 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Jan 24, 2022 12:45:32 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 204 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Jan 24, 2022 12:45:33 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 204 files cached, 0 files newly uploaded in 0 seconds
Jan 24, 2022 12:45:33 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Jan 24, 2022 12:45:33 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <114075 bytes, hash de8261c71ef455048cd0e681b1f83d1ee90c895c5bafd4e9ec89f89a0714352b> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-3oJhxx70VQSM0OaBsfg9HukMiVxbr9Tp7In4mgcUNSs.pb
Jan 24, 2022 12:45:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Jan 24, 2022 12:45:35 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@44b29496, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1947596f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3078cac, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@f6de586, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5f2bd6d9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7c847072, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43d9f1a2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@23f86d8a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2264ea32, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7d3c09ec, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@71e4b308, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@11900483, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@14a049f9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@94e51e8, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5de6cf3a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4cc36c19, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5a3a1bf9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2100d047, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4af45442, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1d805aa1]
Jan 24, 2022 12:45:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Jan 24, 2022 12:45:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Jan 24, 2022 12:45:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Jan 24, 2022 12:45:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Jan 24, 2022 12:45:35 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@441b8382, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1df1ced0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5349b246, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@32b0876c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2aaf152b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@39bbd9e0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@27fe9713, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@11c3ff67, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4397a639, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@24841372, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77114efe, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@79a7bfbc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77f68df, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3e4e4c1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7e7f3cfd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ae126d1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@46a488c2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6242ae3b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@65ddee5a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@56399b9e]
Jan 24, 2022 12:45:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Jan 24, 2022 12:45:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Jan 24, 2022 12:45:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Jan 24, 2022 12:45:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Jan 24, 2022 12:45:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Jan 24, 2022 12:45:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Jan 24, 2022 12:45:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Jan 24, 2022 12:45:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Jan 24, 2022 12:45:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Jan 24, 2022 12:45:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Jan 24, 2022 12:45:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Jan 24, 2022 12:45:35 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.37.0-SNAPSHOT
Jan 24, 2022 12:45:36 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-01-24_04_45_35-445355469944687180?project=apache-beam-testing
Jan 24, 2022 12:45:36 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2022-01-24_04_45_35-445355469944687180
Jan 24, 2022 12:45:36 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2022-01-24_04_45_35-445355469944687180
Jan 24, 2022 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2022-01-24T12:45:43.051Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-01-mm39. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Jan 24, 2022 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-24T12:45:48.178Z: Worker configuration: e2-standard-2 in us-central1-b.
Jan 24, 2022 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-24T12:45:49.048Z: Expanding SplittableParDo operations into optimizable parts.
Jan 24, 2022 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-24T12:45:49.110Z: Expanding CollectionToSingleton operations into optimizable parts.
Jan 24, 2022 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-24T12:45:49.175Z: Expanding CoGroupByKey operations into optimizable parts.
Jan 24, 2022 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-24T12:45:49.246Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Jan 24, 2022 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-24T12:45:49.289Z: Expanding GroupByKey operations into streaming Read/Write steps
Jan 24, 2022 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-24T12:45:49.345Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jan 24, 2022 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-24T12:45:49.452Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jan 24, 2022 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-24T12:45:49.471Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Jan 24, 2022 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-24T12:45:49.498Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Jan 24, 2022 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-24T12:45:49.522Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Jan 24, 2022 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-24T12:45:49.550Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Jan 24, 2022 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-24T12:45:49.588Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jan 24, 2022 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-24T12:45:49.610Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jan 24, 2022 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-24T12:45:49.637Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jan 24, 2022 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-24T12:45:49.671Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Jan 24, 2022 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-24T12:45:49.694Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Jan 24, 2022 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-24T12:45:49.717Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Jan 24, 2022 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-24T12:45:49.748Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Jan 24, 2022 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-24T12:45:49.781Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jan 24, 2022 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-24T12:45:49.814Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jan 24, 2022 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-24T12:45:49.843Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jan 24, 2022 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-24T12:45:49.873Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Jan 24, 2022 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-24T12:45:49.907Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Jan 24, 2022 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-24T12:45:49.928Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Jan 24, 2022 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-24T12:45:49.961Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Jan 24, 2022 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-24T12:45:49.992Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Jan 24, 2022 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-24T12:45:50.026Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Jan 24, 2022 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-24T12:45:50.050Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Jan 24, 2022 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-24T12:45:50.071Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Jan 24, 2022 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-24T12:45:50.417Z: Starting 5 ****s in us-central1-b...
Jan 24, 2022 12:46:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-24T12:46:10.082Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jan 24, 2022 12:46:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-24T12:46:35.661Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Jan 24, 2022 12:47:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-24T12:47:34.434Z: Workers have started successfully.
Jan 24, 2022 12:47:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-24T12:47:34.468Z: Workers have started successfully.
Jan 24, 2022 2:56:19 PM org.apache.beam.sdk.metrics.MetricsEnvironment getCurrentContainer
WARNING: Reporting metrics are not supported in the current execution environment.
Build timed out (after 240 minutes). Marking the build as aborted.
FATAL: command execution failed
hudson.remoting.ChannelClosedException: Channel "hudson.remoting.Channel@3caf42b8:apache-beam-jenkins-7": Remote call on apache-beam-jenkins-7 failed. The channel is closing down or has closed down
	at hudson.remoting.Channel.call(Channel.java:994)
	at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:286)
	at com.sun.proxy.$Proxy123.isAlive(Unknown Source)
	at hudson.Launcher$RemoteLauncher$ProcImpl.isAlive(Launcher.java:1211)
	at hudson.Launcher$RemoteLauncher$ProcImpl.join(Launcher.java:1203)
	at hudson.Launcher$ProcStarter.join(Launcher.java:523)
	at hudson.plugins.gradle.Gradle.perform(Gradle.java:317)
	at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
	at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:803)
	at hudson.model.Build$BuildExecution.build(Build.java:197)
	at hudson.model.Build$BuildExecution.doRun(Build.java:163)
	at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:513)
	at hudson.model.Run.execute(Run.java:1906)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
	at hudson.model.ResourceController.execute(ResourceController.java:97)
	at hudson.model.Executor.run(Executor.java:429)
Caused by: java.io.IOException
	at hudson.remoting.Channel.close(Channel.java:1499)
	at hudson.remoting.Channel.close(Channel.java:1455)
	at hudson.slaves.SlaveComputer.closeChannel(SlaveComputer.java:884)
	at hudson.slaves.SlaveComputer.access$100(SlaveComputer.java:110)
	at hudson.slaves.SlaveComputer$2.run(SlaveComputer.java:765)
	at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
	at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:68)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
ERROR: apache-beam-jenkins-7 is offline; cannot locate jdk_1.8_latest

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #220

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/220/display/redirect>

Changes:


------------------------------------------
[...truncated 49.71 KB...]
92bdfc681768: Preparing
31db193b5341: Preparing
2b516344936b: Preparing
a382a235a51a: Preparing
3bb5258f46d2: Preparing
832e177bb500: Preparing
f9e18e59a565: Preparing
26a504e63be4: Preparing
8bf42db0de72: Preparing
31892cc314cb: Preparing
11936051f93b: Preparing
3bb5258f46d2: Waiting
a382a235a51a: Waiting
832e177bb500: Waiting
92bdfc681768: Waiting
f9e18e59a565: Waiting
31db193b5341: Waiting
8b832f1350cf: Waiting
26a504e63be4: Waiting
cc7c978d3013: Waiting
2b516344936b: Waiting
cf73a7ff016b: Waiting
8bf42db0de72: Waiting
f474f14b1fc4: Waiting
31892cc314cb: Waiting
dd53e43265f7: Pushed
22c5323e4404: Pushed
d0b4d25fddcf: Pushed
d7c9d110ad4a: Pushed
cc7c978d3013: Pushed
8b832f1350cf: Pushed
df9fa634f6bf: Pushed
f474f14b1fc4: Pushed
92bdfc681768: Pushed
cf73a7ff016b: Pushed
3bb5258f46d2: Layer already exists
832e177bb500: Layer already exists
f9e18e59a565: Layer already exists
26a504e63be4: Layer already exists
8bf42db0de72: Layer already exists
11936051f93b: Layer already exists
31892cc314cb: Layer already exists
2b516344936b: Pushed
a382a235a51a: Pushed
31db193b5341: Pushed
20220123124334: digest: sha256:327a5207d0536275f7c50fef32e64cdb7f68d18058aeaaa2fd8607d13c7e2f90 size: 4520

> Task :sdks:java:testing:load-tests:run
Jan 23, 2022 12:45:31 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Jan 23, 2022 12:45:32 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 204 files. Enable logging at DEBUG level to see which files will be staged.
Jan 23, 2022 12:45:32 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Jan 23, 2022 12:45:32 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Jan 23, 2022 12:45:35 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 204 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Jan 23, 2022 12:45:36 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 204 files cached, 0 files newly uploaded in 0 seconds
Jan 23, 2022 12:45:36 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Jan 23, 2022 12:45:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <114075 bytes, hash 2bcdc864d0594827cda6dea9091a81e96e9e861225765bb2408e31a0528aa236> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-K83IZNBZSCfNpt6pCRqB6W6ehhIldluyQI4xoFKKojY.pb
Jan 23, 2022 12:45:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Jan 23, 2022 12:45:37 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@44b29496, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1947596f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3078cac, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@f6de586, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5f2bd6d9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7c847072, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43d9f1a2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@23f86d8a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2264ea32, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7d3c09ec, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@71e4b308, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@11900483, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@14a049f9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@94e51e8, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5de6cf3a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4cc36c19, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5a3a1bf9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2100d047, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4af45442, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1d805aa1]
Jan 23, 2022 12:45:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Jan 23, 2022 12:45:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Jan 23, 2022 12:45:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Jan 23, 2022 12:45:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Jan 23, 2022 12:45:37 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@441b8382, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1df1ced0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5349b246, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@32b0876c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2aaf152b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@39bbd9e0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@27fe9713, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@11c3ff67, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4397a639, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@24841372, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77114efe, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@79a7bfbc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77f68df, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3e4e4c1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7e7f3cfd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ae126d1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@46a488c2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6242ae3b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@65ddee5a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@56399b9e]
Jan 23, 2022 12:45:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Jan 23, 2022 12:45:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Jan 23, 2022 12:45:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Jan 23, 2022 12:45:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Jan 23, 2022 12:45:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Jan 23, 2022 12:45:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Jan 23, 2022 12:45:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Jan 23, 2022 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Jan 23, 2022 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Jan 23, 2022 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Jan 23, 2022 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Jan 23, 2022 12:45:38 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.37.0-SNAPSHOT
Jan 23, 2022 12:45:39 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-01-23_04_45_38-9689623812230593792?project=apache-beam-testing
Jan 23, 2022 12:45:39 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2022-01-23_04_45_38-9689623812230593792
Jan 23, 2022 12:45:39 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2022-01-23_04_45_38-9689623812230593792
Jan 23, 2022 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2022-01-23T12:45:47.726Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-01-idef. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Jan 23, 2022 12:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-23T12:45:51.411Z: Worker configuration: e2-standard-2 in us-central1-b.
Jan 23, 2022 12:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-23T12:45:52.008Z: Expanding SplittableParDo operations into optimizable parts.
Jan 23, 2022 12:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-23T12:45:52.034Z: Expanding CollectionToSingleton operations into optimizable parts.
Jan 23, 2022 12:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-23T12:45:52.106Z: Expanding CoGroupByKey operations into optimizable parts.
Jan 23, 2022 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-23T12:45:52.153Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Jan 23, 2022 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-23T12:45:52.173Z: Expanding GroupByKey operations into streaming Read/Write steps
Jan 23, 2022 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-23T12:45:52.229Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jan 23, 2022 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-23T12:45:52.317Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jan 23, 2022 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-23T12:45:52.340Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Jan 23, 2022 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-23T12:45:52.361Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Jan 23, 2022 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-23T12:45:52.386Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Jan 23, 2022 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-23T12:45:52.414Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Jan 23, 2022 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-23T12:45:52.475Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jan 23, 2022 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-23T12:45:52.508Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jan 23, 2022 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-23T12:45:52.542Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jan 23, 2022 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-23T12:45:52.571Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Jan 23, 2022 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-23T12:45:52.591Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Jan 23, 2022 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-23T12:45:52.624Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Jan 23, 2022 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-23T12:45:52.657Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Jan 23, 2022 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-23T12:45:52.690Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jan 23, 2022 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-23T12:45:52.728Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jan 23, 2022 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-23T12:45:52.758Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jan 23, 2022 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-23T12:45:52.786Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Jan 23, 2022 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-23T12:45:52.816Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Jan 23, 2022 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-23T12:45:52.842Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Jan 23, 2022 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-23T12:45:52.899Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Jan 23, 2022 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-23T12:45:52.923Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Jan 23, 2022 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-23T12:45:52.949Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Jan 23, 2022 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-23T12:45:52.970Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Jan 23, 2022 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-23T12:45:53.002Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Jan 23, 2022 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-23T12:45:53.345Z: Starting 5 ****s in us-central1-b...
Jan 23, 2022 12:46:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-23T12:46:18.525Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jan 23, 2022 12:46:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-23T12:46:38.474Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Jan 23, 2022 12:47:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-23T12:47:41.435Z: Workers have started successfully.
Jan 23, 2022 12:47:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-23T12:47:41.452Z: Workers have started successfully.
Jan 23, 2022 4:00:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-23T16:00:34.846Z: Cancel request is committed for workflow job: 2022-01-23_04_45_38-9689623812230593792.
Jan 23, 2022 4:00:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-23T16:00:34.903Z: Cleaning up.
Jan 23, 2022 4:00:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-23T16:00:34.964Z: Stopping **** pool...
Jan 23, 2022 4:00:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-23T16:00:35.010Z: Stopping **** pool...
Jan 23, 2022 4:02:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-23T16:02:54.678Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Jan 23, 2022 4:02:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-23T16:02:54.726Z: Worker pool stopped.
Jan 23, 2022 4:03:01 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2022-01-23_04_45_38-9689623812230593792 finished with status CANCELLED.
Load test results for test (ID): b1a6fda8-8a23-47d9-8fd0-6353ad4ad2f7 and timestamp: 2022-01-23T12:45:32.517000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11550.603
dataflow_v2_java11_total_bytes_count             2.14111241E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220123124334
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:327a5207d0536275f7c50fef32e64cdb7f68d18058aeaaa2fd8607d13c7e2f90
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220123124334]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:327a5207d0536275f7c50fef32e64cdb7f68d18058aeaaa2fd8607d13c7e2f90]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220123124334] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:327a5207d0536275f7c50fef32e64cdb7f68d18058aeaaa2fd8607d13c7e2f90])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:327a5207d0536275f7c50fef32e64cdb7f68d18058aeaaa2fd8607d13c7e2f90
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:327a5207d0536275f7c50fef32e64cdb7f68d18058aeaaa2fd8607d13c7e2f90
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:327a5207d0536275f7c50fef32e64cdb7f68d18058aeaaa2fd8607d13c7e2f90].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings

Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.

BUILD FAILED in 3h 19m 45s
109 actionable tasks: 73 executed, 32 from cache, 4 up-to-date

Publishing build scan...
https://gradle.com/s/iuwmrkvptyk4s

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #219

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/219/display/redirect?page=changes>

Changes:

[ningkang0957] [BEAM-13687] Improved Spanner IO request count metrics

[noreply] [BEAM-10206] Add key for fields in wrapper (#16583)

[noreply] Merge pull request #16530 from Adding JSON support in SpannerIO and

[noreply] [BEAM-13685] Enable users to specify cache directory under Interactive


------------------------------------------
[...truncated 50.04 KB...]
34f0f234ac31: Pushed
a0fb00b6ea3c: Pushed
84ca33d4319b: Pushed
fac0861a38ed: Pushed
4188e5f0ea30: Pushed
af2fd23ee6b3: Pushed
3bb5258f46d2: Layer already exists
832e177bb500: Layer already exists
26a504e63be4: Layer already exists
8bf42db0de72: Layer already exists
31892cc314cb: Layer already exists
11936051f93b: Layer already exists
f9e18e59a565: Layer already exists
21938ef367d1: Pushed
900600f37aef: Pushed
20220122124331: digest: sha256:dcf593d4d33dfea412d9e05606500dbea8e7b0ac43ff3325081ceb1b25b66bf1 size: 4520

> Task :sdks:java:testing:load-tests:run
Jan 22, 2022 12:45:13 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Jan 22, 2022 12:45:14 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 204 files. Enable logging at DEBUG level to see which files will be staged.
Jan 22, 2022 12:45:14 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Jan 22, 2022 12:45:14 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Jan 22, 2022 12:45:16 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 204 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Jan 22, 2022 12:45:18 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 204 files cached, 0 files newly uploaded in 1 seconds
Jan 22, 2022 12:45:18 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Jan 22, 2022 12:45:18 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <114075 bytes, hash c56b06723a4b3327eb2ab11df8b2860b8f0ccbb752800051a618d40ddeb9ec2a> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-xWsGcjpLMyfrKrEd-LKGC48My7dSgABRphjUDd657Co.pb
Jan 22, 2022 12:45:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Jan 22, 2022 12:45:20 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1947596f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3078cac, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@f6de586, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5f2bd6d9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7c847072, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43d9f1a2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@23f86d8a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2264ea32, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7d3c09ec, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@71e4b308, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@11900483, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@14a049f9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@94e51e8, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5de6cf3a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4cc36c19, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5a3a1bf9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2100d047, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4af45442, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1d805aa1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@176f7f3b]
Jan 22, 2022 12:45:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Jan 22, 2022 12:45:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Jan 22, 2022 12:45:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Jan 22, 2022 12:45:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Jan 22, 2022 12:45:20 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1df1ced0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5349b246, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@32b0876c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2aaf152b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@39bbd9e0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@27fe9713, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@11c3ff67, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4397a639, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@24841372, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77114efe, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@79a7bfbc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77f68df, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3e4e4c1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7e7f3cfd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ae126d1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@46a488c2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6242ae3b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@65ddee5a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@56399b9e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@34b9eb03]
Jan 22, 2022 12:45:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Jan 22, 2022 12:45:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Jan 22, 2022 12:45:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Jan 22, 2022 12:45:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Jan 22, 2022 12:45:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Jan 22, 2022 12:45:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Jan 22, 2022 12:45:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Jan 22, 2022 12:45:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Jan 22, 2022 12:45:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Jan 22, 2022 12:45:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Jan 22, 2022 12:45:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Jan 22, 2022 12:45:20 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.37.0-SNAPSHOT
Jan 22, 2022 12:45:21 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-01-22_04_45_20-518053840289819975?project=apache-beam-testing
Jan 22, 2022 12:45:21 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2022-01-22_04_45_20-518053840289819975
Jan 22, 2022 12:45:21 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2022-01-22_04_45_20-518053840289819975
Jan 22, 2022 12:45:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2022-01-22T12:45:27.627Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-01-jp1v. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Jan 22, 2022 12:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-22T12:45:32.405Z: Worker configuration: e2-standard-2 in us-central1-b.
Jan 22, 2022 12:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-22T12:45:32.996Z: Expanding SplittableParDo operations into optimizable parts.
Jan 22, 2022 12:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-22T12:45:33.052Z: Expanding CollectionToSingleton operations into optimizable parts.
Jan 22, 2022 12:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-22T12:45:33.104Z: Expanding CoGroupByKey operations into optimizable parts.
Jan 22, 2022 12:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-22T12:45:33.170Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Jan 22, 2022 12:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-22T12:45:33.205Z: Expanding GroupByKey operations into streaming Read/Write steps
Jan 22, 2022 12:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-22T12:45:33.275Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jan 22, 2022 12:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-22T12:45:33.550Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jan 22, 2022 12:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-22T12:45:33.631Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Jan 22, 2022 12:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-22T12:45:33.667Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Jan 22, 2022 12:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-22T12:45:33.769Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Jan 22, 2022 12:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-22T12:45:33.884Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Jan 22, 2022 12:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-22T12:45:33.951Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jan 22, 2022 12:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-22T12:45:33.995Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jan 22, 2022 12:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-22T12:45:34.052Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jan 22, 2022 12:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-22T12:45:34.103Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Jan 22, 2022 12:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-22T12:45:34.138Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Jan 22, 2022 12:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-22T12:45:34.184Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Jan 22, 2022 12:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-22T12:45:34.232Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Jan 22, 2022 12:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-22T12:45:34.265Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jan 22, 2022 12:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-22T12:45:34.300Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jan 22, 2022 12:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-22T12:45:34.323Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jan 22, 2022 12:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-22T12:45:34.367Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Jan 22, 2022 12:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-22T12:45:34.403Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Jan 22, 2022 12:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-22T12:45:34.429Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Jan 22, 2022 12:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-22T12:45:34.461Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Jan 22, 2022 12:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-22T12:45:34.509Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Jan 22, 2022 12:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-22T12:45:34.543Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Jan 22, 2022 12:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-22T12:45:34.582Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Jan 22, 2022 12:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-22T12:45:34.626Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Jan 22, 2022 12:45:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-22T12:45:35.127Z: Starting 5 ****s in us-central1-b...
Jan 22, 2022 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-22T12:45:45.489Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jan 22, 2022 12:46:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-22T12:46:25.811Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Jan 22, 2022 12:47:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-22T12:47:24.768Z: Workers have started successfully.
Jan 22, 2022 12:47:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-22T12:47:24.806Z: Workers have started successfully.
Jan 22, 2022 4:00:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-22T16:00:35.369Z: Cancel request is committed for workflow job: 2022-01-22_04_45_20-518053840289819975.
Jan 22, 2022 4:00:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-22T16:00:39.938Z: Cleaning up.
Jan 22, 2022 4:00:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-22T16:00:40.015Z: Stopping **** pool...
Jan 22, 2022 4:00:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-22T16:00:40.080Z: Stopping **** pool...
Jan 22, 2022 4:03:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-22T16:03:04.360Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Jan 22, 2022 4:03:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-22T16:03:04.407Z: Worker pool stopped.
Jan 22, 2022 4:03:10 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2022-01-22_04_45_20-518053840289819975 finished with status CANCELLED.
Load test results for test (ID): a903fa10-e52f-4b2a-87a5-e7b8c96c6fdc and timestamp: 2022-01-22T12:45:14.320000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11575.749
dataflow_v2_java11_total_bytes_count             2.51320278E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220122124331
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:dcf593d4d33dfea412d9e05606500dbea8e7b0ac43ff3325081ceb1b25b66bf1
Deleted: sha256:4e1c80fd78d8b09138a20abf9c5cfdf0d8e49a8eddd852b6e3d4947fd743ec27
Deleted: sha256:6d31a5e58e2cd8f7e01d667ada326056e28da3f291acd7bc09f801cbda8f6dba
Deleted: sha256:5d5030023eca247ffe6553c913858f93c9491f382be7b692813d69a271be049c
Deleted: sha256:c4f1e14f9ff3e942b92e666cb0af292de03f0f1c1bc335acd709dbad468b9782
Deleted: sha256:cae6137207efc69dacf310fb3c2a5b21ede970e90dfbd4920e20b319cf38f68c
Deleted: sha256:9efe7b45d93b361fd6b12b480cfa06ab7f0ad1057d745b5dec1ab7271950ee2d
Deleted: sha256:1cb7a1d13cc5a1c1899d0751b6c160b37b396318a5a035278ef182a4d27192f0
Deleted: sha256:33ee8d2b1e4800d6511dc4a4dceec9a026f5ca0469408ef4ae88d902575eea15
Deleted: sha256:95e61cf0da401382128ef3197cc1f3576bf984e76dcaefb705f6e336e05867e7
Deleted: sha256:81dfd5fe0dc8e23c97868816ed042feb755a3758240724c87ddf44d5b3ff7170
Deleted: sha256:ffebe5549c493a49fcc433a153e727651726a1ad86122dca8b0fe0dc52db4f7a
Deleted: sha256:3caf58e139b1f9da0fc2de3450b5900a29f147b8341bb64f5a9d6f65914ff2cf
Deleted: sha256:652e05429ef1004a8aa0a77e30dc186b42b6226085f96eea30bb33606a4b290f
Deleted: sha256:fc22a56d9cde433936324c8f0b37b01f719b2ecfc410b37635c8a499095ed782
Deleted: sha256:0b4085c97763b09a9dc1380f061f5639a4a64847721d11a61449d410c610b754
Deleted: sha256:d8fef36923094678c7d9f183b3675bdefecf793041940ea80d8f355667ec8216
Deleted: sha256:1de5033d99b370203ccf83433f88fd90e804cfc5561f7a6b1711e9f1fd7d20df
Deleted: sha256:5fa2a78b46def8e9e155a73e8b3704408919bf8ad48027830b3501a5b4692c88
Deleted: sha256:beb2526e33def90ce80897eb255ec38c9321696dc564be1042de8913da9c9950
Deleted: sha256:e236b93c2acda09795b4f7520fa8e4fdca3d1543e149883a9ac37dada4ff1815
Deleted: sha256:695665f1f7700ec705ca7c73a2205b69de6f875c6b1afc355a8529114ce12510
Deleted: sha256:ddbf565541344ac88d437c5ed292b231063c2e65774f580f8fc02dd7852435a8
Deleted: sha256:25362bc602b9b8ecc6c654d0092f431b666cc797b12cff134cd34056a24b8db5
Deleted: sha256:856958f37eff4b4e417638e0e257781a9a9fb4c17b8eec9a6cf31bf01876670b
Deleted: sha256:b0f4d9170e0d9ee2bbbcc214ececc3ecb0a1df999b9abc90551b95f7caac1647
Deleted: sha256:491a6ce71c358747341e3b5176bde3d55b535a0e99e62e3c3a27170e247fa038
Deleted: sha256:dfd1b7d2feeb2eba5147e8651d39bad8333a62120016a9e0aaba255f4c613b5e
Deleted: sha256:681a40589d7c810e61acaec475faf9f4cbed23ca85a11a088a1abba98b3a3a73
Deleted: sha256:952ed8615615addfbc11a18fefff6be16fca4b585e08cf8bce86d42b02238792
Deleted: sha256:a4fc634f88c4e90d0a4e62f22017f71103002977c59d99bfd9308b53c39f34b6
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220122124331]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:dcf593d4d33dfea412d9e05606500dbea8e7b0ac43ff3325081ceb1b25b66bf1]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220122124331] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:dcf593d4d33dfea412d9e05606500dbea8e7b0ac43ff3325081ceb1b25b66bf1])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:dcf593d4d33dfea412d9e05606500dbea8e7b0ac43ff3325081ceb1b25b66bf1
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:dcf593d4d33dfea412d9e05606500dbea8e7b0ac43ff3325081ceb1b25b66bf1
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:dcf593d4d33dfea412d9e05606500dbea8e7b0ac43ff3325081ceb1b25b66bf1].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings

Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.

BUILD FAILED in 3h 19m 55s
109 actionable tasks: 72 executed, 33 from cache, 4 up-to-date

Publishing build scan...
https://gradle.com/s/ufviu2mjcksc4

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #218

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/218/display/redirect?page=changes>

Changes:

[thiagotnunes] [BEAM-12164]: Add SDF for reading change stream records

[tuyarer] [BEAM-13577] Beam Select's uniquifyNames function loses nullability of

[sergey.kalinin] Update GH Actions to use proper variables names and proper triggers

[dhuntsperger] edited README and comments in Python multi-lang pipes examples

[Pablo Estrada] Revert "Merge pull request #15863 from [BEAM-13184] Autosharding for

[Pablo Estrada] BEAM-13611 reactivating jdbcio xlang test

[Steve Niemitz] [BEAM-13689] Output token elements when BQ batch writes complete.

[noreply] Merge pull request #16371 from [BEAM-13518][Playground] Beam Playground

[noreply] Update Java FnAPI beam master (#16572)

[noreply] [BEAM-13699] Replace fnv with maphash. (#16573)

[noreply] [BEAM-13693] Bump

[noreply] [BEAM-10206] Remove Fatalf calls in non-test goroutines for

[noreply] [BEAM-13430] Re-add provided configuration (#16552)

[noreply] Merge pull request #16540 from [BEAM-13678][Playground]Update Github

[noreply] Merge pull request #16546 from [BEAM-13661] [BEAM-13704] [Playground]

[noreply] Merge pull request #16369 from [BEAM-13558] [Playground] Hide the Graph


------------------------------------------
[...truncated 49.43 KB...]
1b4f9dff9530: Preparing
dcf309103c81: Preparing
5278327e7316: Preparing
3bb5258f46d2: Preparing
832e177bb500: Preparing
f9e18e59a565: Preparing
26a504e63be4: Preparing
8bf42db0de72: Preparing
31892cc314cb: Preparing
11936051f93b: Preparing
8676a0c81ea4: Waiting
1b4f9dff9530: Waiting
dcf309103c81: Waiting
64ce84ead65a: Waiting
1c8cb9413709: Waiting
c0d2ced337ac: Waiting
5278327e7316: Waiting
26a504e63be4: Waiting
f9e18e59a565: Waiting
832e177bb500: Waiting
3bb5258f46d2: Waiting
31892cc314cb: Waiting
8bf42db0de72: Waiting
11936051f93b: Waiting
b7c0ee70947a: Waiting
e53be5bc9b84: Pushed
69d5eb2f8c50: Pushed
b432987d309f: Pushed
f20f31ec6a4c: Pushed
b7c0ee70947a: Pushed
cef3c17d4e4e: Pushed
c0d2ced337ac: Pushed
64ce84ead65a: Pushed
1c8cb9413709: Pushed
8676a0c81ea4: Pushed
dcf309103c81: Pushed
3bb5258f46d2: Layer already exists
832e177bb500: Layer already exists
f9e18e59a565: Layer already exists
26a504e63be4: Layer already exists
31892cc314cb: Layer already exists
11936051f93b: Layer already exists
8bf42db0de72: Layer already exists
5278327e7316: Pushed
1b4f9dff9530: Pushed
20220121124336: digest: sha256:3ec3973dc0b0b6d83932bd6a73291cb4b0c401b28b0e248a92abc8aac20a04d4 size: 4520

> Task :sdks:java:testing:load-tests:run
Jan 21, 2022 12:45:20 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Jan 21, 2022 12:45:21 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 204 files. Enable logging at DEBUG level to see which files will be staged.
Jan 21, 2022 12:45:21 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Jan 21, 2022 12:45:21 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Jan 21, 2022 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 204 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Jan 21, 2022 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 204 files cached, 0 files newly uploaded in 0 seconds
Jan 21, 2022 12:45:25 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Jan 21, 2022 12:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <114075 bytes, hash 4ab60571159b1b7812df15e1bad742c905ef00d0f9251b06e0a5d7047aa8bb0c> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-SrYFcRWbG3gS3xXhutdCyQXvAND5JRsG4KXXBHqouww.pb
Jan 21, 2022 12:45:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Jan 21, 2022 12:45:26 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1947596f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3078cac, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@f6de586, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5f2bd6d9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7c847072, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43d9f1a2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@23f86d8a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2264ea32, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7d3c09ec, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@71e4b308, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@11900483, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@14a049f9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@94e51e8, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5de6cf3a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4cc36c19, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5a3a1bf9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2100d047, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4af45442, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1d805aa1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@176f7f3b]
Jan 21, 2022 12:45:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Jan 21, 2022 12:45:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Jan 21, 2022 12:45:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Jan 21, 2022 12:45:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Jan 21, 2022 12:45:26 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1df1ced0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5349b246, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@32b0876c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2aaf152b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@39bbd9e0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@27fe9713, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@11c3ff67, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4397a639, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@24841372, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77114efe, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@79a7bfbc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77f68df, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3e4e4c1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7e7f3cfd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ae126d1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@46a488c2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6242ae3b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@65ddee5a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@56399b9e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@34b9eb03]
Jan 21, 2022 12:45:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Jan 21, 2022 12:45:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Jan 21, 2022 12:45:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Jan 21, 2022 12:45:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Jan 21, 2022 12:45:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Jan 21, 2022 12:45:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Jan 21, 2022 12:45:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Jan 21, 2022 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Jan 21, 2022 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Jan 21, 2022 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Jan 21, 2022 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Jan 21, 2022 12:45:27 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.37.0-SNAPSHOT
Jan 21, 2022 12:45:27 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-01-21_04_45_27-1238241219485704708?project=apache-beam-testing
Jan 21, 2022 12:45:27 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2022-01-21_04_45_27-1238241219485704708
Jan 21, 2022 12:45:27 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2022-01-21_04_45_27-1238241219485704708
Jan 21, 2022 12:45:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2022-01-21T12:45:34.267Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-01-jjl8. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Jan 21, 2022 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-21T12:45:39.624Z: Worker configuration: e2-standard-2 in us-central1-b.
Jan 21, 2022 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-21T12:45:40.805Z: Expanding SplittableParDo operations into optimizable parts.
Jan 21, 2022 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-21T12:45:40.831Z: Expanding CollectionToSingleton operations into optimizable parts.
Jan 21, 2022 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-21T12:45:40.901Z: Expanding CoGroupByKey operations into optimizable parts.
Jan 21, 2022 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-21T12:45:40.957Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Jan 21, 2022 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-21T12:45:40.987Z: Expanding GroupByKey operations into streaming Read/Write steps
Jan 21, 2022 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-21T12:45:41.054Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jan 21, 2022 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-21T12:45:41.154Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jan 21, 2022 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-21T12:45:41.183Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Jan 21, 2022 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-21T12:45:41.211Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Jan 21, 2022 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-21T12:45:41.234Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Jan 21, 2022 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-21T12:45:41.260Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Jan 21, 2022 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-21T12:45:41.292Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jan 21, 2022 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-21T12:45:41.313Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jan 21, 2022 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-21T12:45:41.369Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jan 21, 2022 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-21T12:45:41.398Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Jan 21, 2022 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-21T12:45:41.459Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Jan 21, 2022 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-21T12:45:41.489Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Jan 21, 2022 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-21T12:45:41.516Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Jan 21, 2022 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-21T12:45:41.551Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jan 21, 2022 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-21T12:45:41.589Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jan 21, 2022 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-21T12:45:41.612Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jan 21, 2022 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-21T12:45:41.637Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Jan 21, 2022 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-21T12:45:41.693Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Jan 21, 2022 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-21T12:45:41.727Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Jan 21, 2022 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-21T12:45:41.763Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Jan 21, 2022 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-21T12:45:41.798Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Jan 21, 2022 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-21T12:45:41.832Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Jan 21, 2022 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-21T12:45:41.868Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Jan 21, 2022 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-21T12:45:41.902Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Jan 21, 2022 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-21T12:45:42.266Z: Starting 5 ****s in us-central1-b...
Jan 21, 2022 12:46:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-21T12:46:11.251Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jan 21, 2022 12:46:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-21T12:46:33.098Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Jan 21, 2022 12:47:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-21T12:47:35.483Z: Workers have started successfully.
Jan 21, 2022 12:47:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-21T12:47:35.516Z: Workers have started successfully.
Jan 21, 2022 4:00:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-21T16:00:39.016Z: Cancel request is committed for workflow job: 2022-01-21_04_45_27-1238241219485704708.
Jan 21, 2022 4:00:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-21T16:00:39.121Z: Cleaning up.
Jan 21, 2022 4:00:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-21T16:00:39.207Z: Stopping **** pool...
Jan 21, 2022 4:00:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-21T16:00:39.251Z: Stopping **** pool...
Jan 21, 2022 4:03:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-21T16:03:01.320Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Jan 21, 2022 4:03:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-21T16:03:01.365Z: Worker pool stopped.
Jan 21, 2022 4:03:08 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2022-01-21_04_45_27-1238241219485704708 finished with status CANCELLED.
Load test results for test (ID): 031bac3d-6dad-4290-a04c-94f6c1f809bc and timestamp: 2022-01-21T12:45:21.410000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11550.924
dataflow_v2_java11_total_bytes_count             1.92177733E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220121124336
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:3ec3973dc0b0b6d83932bd6a73291cb4b0c401b28b0e248a92abc8aac20a04d4
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220121124336]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:3ec3973dc0b0b6d83932bd6a73291cb4b0c401b28b0e248a92abc8aac20a04d4]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220121124336] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:3ec3973dc0b0b6d83932bd6a73291cb4b0c401b28b0e248a92abc8aac20a04d4])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:3ec3973dc0b0b6d83932bd6a73291cb4b0c401b28b0e248a92abc8aac20a04d4
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:3ec3973dc0b0b6d83932bd6a73291cb4b0c401b28b0e248a92abc8aac20a04d4
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:3ec3973dc0b0b6d83932bd6a73291cb4b0c401b28b0e248a92abc8aac20a04d4].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings

Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.

BUILD FAILED in 3h 19m 49s
109 actionable tasks: 72 executed, 33 from cache, 4 up-to-date

Publishing build scan...
https://gradle.com/s/ttzkd24leiouc

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #217

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/217/display/redirect?page=changes>

Changes:

[noreply] [BEAM-13611] Skip test_xlang_jdbc_write (#16554)

[noreply] Merge pull request #16370 from [BEAM-13556] playground - color and

[noreply] Merge pull request #16531 from [BEAM-13567] [playground] Handle run code

[noreply] Merge pull request #16533 from [BEAM-13548] [Playground] Add example

[noreply] Merge pull request #16519 from [BEAM-13639] [Playground] Add

[noreply] Merge pull request #16518 from [BEAM-13619] [Playground] Add loading

[noreply] Merge pull request #16243 from

[noreply] [BEAM-13683] Make cross-language SQL example pipeline (#16567)

[noreply] [BEAM-13688] fixed type in BPG 4.5.3 window section (#16560)

[noreply] Remove obsolete commands from Inventory job. (#16564)

[noreply] Disable logging for memoization test. (#16556)

[noreply] Merge pull request #16472: [BEAM-13697] Add SchemaFieldNumber annotation

[noreply] Merge pull request #16373 from [BEAM-13515] [Playground] Hiding lines in


------------------------------------------
[...truncated 49.46 KB...]
b028f05ac4fb: Preparing
82e75fe2ed36: Preparing
c6a65b8e1d39: Preparing
3bb5258f46d2: Preparing
832e177bb500: Preparing
f9e18e59a565: Preparing
26a504e63be4: Preparing
8bf42db0de72: Preparing
31892cc314cb: Preparing
11936051f93b: Preparing
4c0cfed6299f: Waiting
fd8e4191e68e: Waiting
b028f05ac4fb: Waiting
6e1547a026db: Waiting
82e75fe2ed36: Waiting
c6a65b8e1d39: Waiting
c4a08df5a277: Waiting
3bb5258f46d2: Waiting
31892cc314cb: Waiting
26a504e63be4: Waiting
832e177bb500: Waiting
11936051f93b: Waiting
8bf42db0de72: Waiting
f9e18e59a565: Waiting
972fb55ee942: Waiting
dbd38091623a: Pushed
5bfcc17525be: Pushed
dfb08282e4da: Pushed
2e609110a9ca: Pushed
c97dc4b0f9a1: Pushed
972fb55ee942: Pushed
6e1547a026db: Pushed
c4a08df5a277: Pushed
fd8e4191e68e: Pushed
4c0cfed6299f: Pushed
3bb5258f46d2: Layer already exists
832e177bb500: Layer already exists
f9e18e59a565: Layer already exists
26a504e63be4: Layer already exists
8bf42db0de72: Layer already exists
31892cc314cb: Layer already exists
11936051f93b: Layer already exists
82e75fe2ed36: Pushed
c6a65b8e1d39: Pushed
b028f05ac4fb: Pushed
20220120124334: digest: sha256:1abde018df018c1b8a516c7fe21f822dba94e6c9ccd33a94e150c59c5f986ead size: 4520

> Task :sdks:java:testing:load-tests:run
Jan 20, 2022 12:45:19 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Jan 20, 2022 12:45:20 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 203 files. Enable logging at DEBUG level to see which files will be staged.
Jan 20, 2022 12:45:20 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Jan 20, 2022 12:45:20 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Jan 20, 2022 12:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 203 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Jan 20, 2022 12:45:23 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 203 files cached, 0 files newly uploaded in 0 seconds
Jan 20, 2022 12:45:23 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Jan 20, 2022 12:45:23 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <113730 bytes, hash 68fd4bf719ae42a1cc00045dca80329da6f5e765d6a791444c2bb3f08f3e9bd9> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-aP1L9xmuQqHMAARdyoAynab152XWp5FETCuz8I8-m9k.pb
Jan 20, 2022 12:45:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Jan 20, 2022 12:45:25 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7e446d92, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@57f9b467, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6d5c2745, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@44b29496, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1947596f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3078cac, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@f6de586, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5f2bd6d9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7c847072, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43d9f1a2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@23f86d8a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2264ea32, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7d3c09ec, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@71e4b308, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@11900483, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@14a049f9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@94e51e8, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5de6cf3a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4cc36c19, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5a3a1bf9]
Jan 20, 2022 12:45:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Jan 20, 2022 12:45:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Jan 20, 2022 12:45:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Jan 20, 2022 12:45:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Jan 20, 2022 12:45:25 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6fc6deb7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@367f0121, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7da39774, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@441b8382, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1df1ced0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5349b246, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@32b0876c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2aaf152b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@39bbd9e0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@27fe9713, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@11c3ff67, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4397a639, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@24841372, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77114efe, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@79a7bfbc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77f68df, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3e4e4c1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7e7f3cfd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ae126d1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@46a488c2]
Jan 20, 2022 12:45:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Jan 20, 2022 12:45:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Jan 20, 2022 12:45:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Jan 20, 2022 12:45:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Jan 20, 2022 12:45:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Jan 20, 2022 12:45:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Jan 20, 2022 12:45:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Jan 20, 2022 12:45:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Jan 20, 2022 12:45:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Jan 20, 2022 12:45:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Jan 20, 2022 12:45:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Jan 20, 2022 12:45:25 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.37.0-SNAPSHOT
Jan 20, 2022 12:45:26 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-01-20_04_45_25-3139908799002107795?project=apache-beam-testing
Jan 20, 2022 12:45:26 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2022-01-20_04_45_25-3139908799002107795
Jan 20, 2022 12:45:26 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2022-01-20_04_45_25-3139908799002107795
Jan 20, 2022 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2022-01-20T12:45:37.783Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-01-39nt. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Jan 20, 2022 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-20T12:45:42.615Z: Worker configuration: e2-standard-2 in us-central1-b.
Jan 20, 2022 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-20T12:45:43.301Z: Expanding SplittableParDo operations into optimizable parts.
Jan 20, 2022 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-20T12:45:43.349Z: Expanding CollectionToSingleton operations into optimizable parts.
Jan 20, 2022 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-20T12:45:43.420Z: Expanding CoGroupByKey operations into optimizable parts.
Jan 20, 2022 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-20T12:45:43.562Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Jan 20, 2022 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-20T12:45:43.658Z: Expanding GroupByKey operations into streaming Read/Write steps
Jan 20, 2022 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-20T12:45:43.740Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jan 20, 2022 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-20T12:45:43.876Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jan 20, 2022 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-20T12:45:43.908Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Jan 20, 2022 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-20T12:45:43.937Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Jan 20, 2022 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-20T12:45:43.981Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Jan 20, 2022 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-20T12:45:44.011Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Jan 20, 2022 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-20T12:45:44.037Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jan 20, 2022 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-20T12:45:44.074Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jan 20, 2022 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-20T12:45:44.107Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jan 20, 2022 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-20T12:45:44.135Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Jan 20, 2022 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-20T12:45:44.169Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Jan 20, 2022 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-20T12:45:44.227Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Jan 20, 2022 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-20T12:45:44.268Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Jan 20, 2022 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-20T12:45:44.301Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jan 20, 2022 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-20T12:45:44.350Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jan 20, 2022 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-20T12:45:44.386Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jan 20, 2022 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-20T12:45:44.411Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Jan 20, 2022 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-20T12:45:44.437Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Jan 20, 2022 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-20T12:45:44.463Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Jan 20, 2022 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-20T12:45:44.505Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Jan 20, 2022 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-20T12:45:44.542Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Jan 20, 2022 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-20T12:45:44.572Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Jan 20, 2022 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-20T12:45:44.620Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Jan 20, 2022 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-20T12:45:44.648Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Jan 20, 2022 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-20T12:45:45.044Z: Starting 5 ****s in us-central1-b...
Jan 20, 2022 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-20T12:45:56.135Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jan 20, 2022 12:46:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-20T12:46:37.056Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Jan 20, 2022 12:47:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-20T12:47:39.549Z: Workers have started successfully.
Jan 20, 2022 12:47:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-20T12:47:39.584Z: Workers have started successfully.
Jan 20, 2022 4:00:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-20T16:00:27.344Z: Cancel request is committed for workflow job: 2022-01-20_04_45_25-3139908799002107795.
Jan 20, 2022 4:00:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-20T16:00:27.583Z: Cleaning up.
Jan 20, 2022 4:00:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-20T16:00:27.688Z: Stopping **** pool...
Jan 20, 2022 4:00:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-20T16:00:27.781Z: Stopping **** pool...
Jan 20, 2022 4:02:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-20T16:02:49.441Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Jan 20, 2022 4:02:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-20T16:02:49.487Z: Worker pool stopped.
Jan 20, 2022 4:02:56 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2022-01-20_04_45_25-3139908799002107795 finished with status CANCELLED.
Load test results for test (ID): fee768d0-065a-4997-8dc2-d65c601a0597 and timestamp: 2022-01-20T12:45:20.377000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11416.143
dataflow_v2_java11_total_bytes_count              2.0160757E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220120124334
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:1abde018df018c1b8a516c7fe21f822dba94e6c9ccd33a94e150c59c5f986ead
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220120124334]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:1abde018df018c1b8a516c7fe21f822dba94e6c9ccd33a94e150c59c5f986ead]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220120124334] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:1abde018df018c1b8a516c7fe21f822dba94e6c9ccd33a94e150c59c5f986ead])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:1abde018df018c1b8a516c7fe21f822dba94e6c9ccd33a94e150c59c5f986ead
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:1abde018df018c1b8a516c7fe21f822dba94e6c9ccd33a94e150c59c5f986ead
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:1abde018df018c1b8a516c7fe21f822dba94e6c9ccd33a94e150c59c5f986ead].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings

Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.

BUILD FAILED in 3h 19m 38s
109 actionable tasks: 72 executed, 33 from cache, 4 up-to-date

Publishing build scan...
https://gradle.com/s/4nok7kysabrsa

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #216

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/216/display/redirect?page=changes>

Changes:

[Robert Bradshaw] Bump beam container version.

[alexander.zhuravlev] [BEAM-13680] Fixed code_repository (added pipelineUuid to RunCodeResult

[Robert Bradshaw] Also bump FnAPI container.

[noreply] [BEAM-13616][BEAM-13645] Switch to vendored grpc 1.43.2 (#16543)

[noreply] [BEAM-13616][BEAM-13646] Upgrade vendored calcite to 1.28.0:0.2 (#16544)

[noreply] Merge pull request #16486 from [BEAM-13544][Playground] Add logs to

[noreply] [BEAM-13683] Correct SQL transform schema, fix expansion address

[noreply] Update walkthrough.md (#16512)

[noreply] [BEAM-11808][BEAM-9879] Support aggregate functions with two arguments

[noreply] Merge pull request #16506 from [BEAM-13652][Playground] Send examples'

[noreply] Merge pull request #16322 from [BEAM-13407] [Playground] Preload fonts

[noreply] [BEAM-13665] Make SpannerIO projectID optional again (#16547)

[noreply] [BEAM-13015] Add state caching capability to be used as hint for runners

[noreply] Merge pull request #16309: [BEAM-13503] Set a default value to

[noreply] [BEAM-13015] Provide caching statistics in the status client. (#16495)


------------------------------------------
[...truncated 46.42 KB...]
Note: Recompile with -Xlint:unchecked for details.

> Task :sdks:java:expansion-service:classes
> Task :sdks:java:expansion-service:jar

> Task :sdks:java:io:kafka:compileJava
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.

> Task :sdks:java:io:kafka:classes
> Task :sdks:java:io:kafka:jar
> Task :sdks:java:container:java11:copyDockerfileDependencies
> Task :sdks:java:container:java11:dockerPrepare
> Task :sdks:java:io:google-cloud-platform:compileJava
> Task :sdks:java:container:java11:docker

> Task :runners:google-cloud-dataflow-java:buildAndPushDockerContainer
WARNING: `gcloud docker` will not be supported for Docker client versions above 18.03.

As an alternative, use `gcloud auth configure-docker` to configure `docker` to
use `gcloud` as a credential helper, then use `docker` as you would for non-GCR
registries, e.g. `docker pull gcr.io/project-id/my-image`. Add
`--verbosity=error` to silence this warning: `gcloud docker
--verbosity=error -- pull gcr.io/project-id/my-image`.

See: https://cloud.google.com/container-registry/docs/support/deprecation-notices#gcloud-docker


> Task :sdks:java:io:google-cloud-platform:compileJava
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.

> Task :sdks:java:io:google-cloud-platform:classes
> Task :sdks:java:io:google-cloud-platform:jar

> Task :sdks:java:testing:load-tests:compileJava
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.

> Task :sdks:java:testing:load-tests:classes
> Task :sdks:java:testing:load-tests:jar
> Task :runners:google-cloud-dataflow-java:compileJava
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.

> Task :runners:google-cloud-dataflow-java:classes
> Task :runners:google-cloud-dataflow-java:jar

> Task :sdks:java:testing:load-tests:run
Jan 19, 2022 12:53:35 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Jan 19, 2022 12:53:35 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 203 files. Enable logging at DEBUG level to see which files will be staged.
Jan 19, 2022 12:53:36 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Jan 19, 2022 12:53:36 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Jan 19, 2022 12:53:38 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 203 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Jan 19, 2022 12:53:39 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 203 files cached, 0 files newly uploaded in 0 seconds
Jan 19, 2022 12:53:39 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Jan 19, 2022 12:53:39 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <113730 bytes, hash b13ce9a435c8fb499b4e71174a40c9be6948c42fd7194752387cc8b3b0f09c85> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-sTzppDXI-0mbTnEXSkDJvmlIxC_XGUdSOHzIs7DwnIU.pb
Jan 19, 2022 12:53:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Jan 19, 2022 12:53:41 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6d5c2745, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@44b29496, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1947596f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3078cac, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@f6de586, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5f2bd6d9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7c847072, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43d9f1a2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@23f86d8a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2264ea32, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7d3c09ec, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@71e4b308, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@11900483, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@14a049f9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@94e51e8, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5de6cf3a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4cc36c19, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5a3a1bf9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2100d047, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4af45442]
Jan 19, 2022 12:53:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Jan 19, 2022 12:53:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Jan 19, 2022 12:53:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Jan 19, 2022 12:53:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Jan 19, 2022 12:53:41 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7da39774, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@441b8382, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1df1ced0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5349b246, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@32b0876c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2aaf152b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@39bbd9e0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@27fe9713, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@11c3ff67, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4397a639, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@24841372, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77114efe, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@79a7bfbc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77f68df, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3e4e4c1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7e7f3cfd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ae126d1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@46a488c2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6242ae3b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@65ddee5a]
Jan 19, 2022 12:53:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Jan 19, 2022 12:53:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Jan 19, 2022 12:53:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Jan 19, 2022 12:53:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Jan 19, 2022 12:53:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Jan 19, 2022 12:53:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Jan 19, 2022 12:53:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Jan 19, 2022 12:53:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Jan 19, 2022 12:53:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Jan 19, 2022 12:53:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Jan 19, 2022 12:53:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Jan 19, 2022 12:53:41 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.37.0-SNAPSHOT
Jan 19, 2022 12:53:42 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-01-19_04_53_41-169840831742093810?project=apache-beam-testing
Jan 19, 2022 12:53:42 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2022-01-19_04_53_41-169840831742093810
Jan 19, 2022 12:53:42 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2022-01-19_04_53_41-169840831742093810
Jan 19, 2022 12:53:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2022-01-19T12:53:48.696Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-01-rpwo. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Jan 19, 2022 12:53:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-19T12:53:53.161Z: Worker configuration: e2-standard-2 in us-central1-b.
Jan 19, 2022 12:53:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-19T12:53:53.913Z: Expanding SplittableParDo operations into optimizable parts.
Jan 19, 2022 12:53:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-19T12:53:53.945Z: Expanding CollectionToSingleton operations into optimizable parts.
Jan 19, 2022 12:53:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-19T12:53:54.032Z: Expanding CoGroupByKey operations into optimizable parts.
Jan 19, 2022 12:53:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-19T12:53:54.119Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Jan 19, 2022 12:53:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-19T12:53:54.152Z: Expanding GroupByKey operations into streaming Read/Write steps
Jan 19, 2022 12:53:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-19T12:53:54.223Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jan 19, 2022 12:53:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-19T12:53:54.335Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jan 19, 2022 12:53:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-19T12:53:54.361Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Jan 19, 2022 12:53:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-19T12:53:54.409Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Jan 19, 2022 12:53:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-19T12:53:54.448Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Jan 19, 2022 12:53:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-19T12:53:54.470Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Jan 19, 2022 12:53:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-19T12:53:54.495Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jan 19, 2022 12:53:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-19T12:53:54.528Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jan 19, 2022 12:53:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-19T12:53:54.563Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jan 19, 2022 12:53:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-19T12:53:54.609Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Jan 19, 2022 12:53:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-19T12:53:54.651Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Jan 19, 2022 12:53:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-19T12:53:54.685Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Jan 19, 2022 12:53:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-19T12:53:54.710Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Jan 19, 2022 12:53:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-19T12:53:54.735Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jan 19, 2022 12:53:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-19T12:53:54.782Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jan 19, 2022 12:53:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-19T12:53:54.812Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jan 19, 2022 12:53:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-19T12:53:54.867Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Jan 19, 2022 12:53:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-19T12:53:54.904Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Jan 19, 2022 12:53:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-19T12:53:54.930Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Jan 19, 2022 12:53:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-19T12:53:54.965Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Jan 19, 2022 12:53:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-19T12:53:55.000Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Jan 19, 2022 12:53:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-19T12:53:55.034Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Jan 19, 2022 12:53:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-19T12:53:55.068Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Jan 19, 2022 12:53:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-19T12:53:55.102Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Jan 19, 2022 12:53:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-19T12:53:55.504Z: Starting 5 ****s in us-central1-b...
Jan 19, 2022 12:54:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-19T12:54:12.846Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jan 19, 2022 12:54:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-19T12:54:44.235Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Jan 19, 2022 12:55:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-19T12:55:44.242Z: Workers have started successfully.
Jan 19, 2022 12:55:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-19T12:55:44.264Z: Workers have started successfully.
Jan 19, 2022 4:00:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-19T16:00:30.187Z: Cancel request is committed for workflow job: 2022-01-19_04_53_41-169840831742093810.
Jan 19, 2022 4:00:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-19T16:00:30.523Z: Cleaning up.
Jan 19, 2022 4:00:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-19T16:00:30.613Z: Stopping **** pool...
Jan 19, 2022 4:00:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-19T16:00:30.671Z: Stopping **** pool...
Jan 19, 2022 4:02:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-19T16:02:48.828Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Jan 19, 2022 4:02:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-19T16:02:48.868Z: Worker pool stopped.
Jan 19, 2022 4:02:54 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2022-01-19_04_53_41-169840831742093810 finished with status CANCELLED.
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220119124332]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:c81798c428844021c982c267f58fd91bbb039db9e0a4a46441ff3542772c72aa]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220119124332] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:c81798c428844021c982c267f58fd91bbb039db9e0a4a46441ff3542772c72aa])].
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:c81798c428844021c982c267f58fd91bbb039db9e0a4a46441ff3542772c72aa
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:c81798c428844021c982c267f58fd91bbb039db9e0a4a46441ff3542772c72aa].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings

Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.

BUILD FAILED in 3h 19m 41s
109 actionable tasks: 91 executed, 14 from cache, 4 up-to-date

Publishing build scan...
https://gradle.com/s/3oxkindqranj6

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #215

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/215/display/redirect?page=changes>

Changes:

[mmack] [BEAM-8806] Integration test for SqsIO

[mmack] [BEAM-13631] Add deterministic SQS message coder to fix reading from SQS

[aydar.zaynutdinov] [BEAM-13641][Playground]

[noreply] Remove jcenter repositories from gradle configuration. (#16532)

[noreply] [BEAM-13430] Remove jcenter which will no longer contain any updates.

[noreply] [BEAM-13616] Update com.google.cloud:libraries-bom to 24.2.0 (#16509)


------------------------------------------
[...truncated 49.52 KB...]
832e177bb500: Preparing
f9e18e59a565: Preparing
26a504e63be4: Preparing
8bf42db0de72: Preparing
31892cc314cb: Preparing
11936051f93b: Preparing
722271f3a152: Waiting
f9e18e59a565: Waiting
832e177bb500: Waiting
26a504e63be4: Waiting
50c07935745a: Waiting
8bf42db0de72: Waiting
9212210b6007: Waiting
31892cc314cb: Waiting
11936051f93b: Waiting
23e63f1124c0: Waiting
3bb5258f46d2: Waiting
0e7875135127: Waiting
9c32afa495b6: Waiting
58641dfa8ea3: Waiting
677631deac9b: Waiting
b399fdbb2aa4: Pushed
e88494a0693b: Pushed
ad9410a238b7: Pushed
58641dfa8ea3: Pushed
b5a971611ba8: Pushed
722271f3a152: Pushed
310f66ea4350: Pushed
9212210b6007: Pushed
677631deac9b: Pushed
3bb5258f46d2: Layer already exists
832e177bb500: Layer already exists
50c07935745a: Pushed
23e63f1124c0: Pushed
f9e18e59a565: Layer already exists
26a504e63be4: Layer already exists
8bf42db0de72: Layer already exists
31892cc314cb: Layer already exists
11936051f93b: Layer already exists
9c32afa495b6: Pushed
0e7875135127: Pushed
20220118124848: digest: sha256:dac836ca72005995cd72a445950041cc9c35ee05d8e3de1c3a86c4c78874b9ba size: 4520

> Task :sdks:java:testing:load-tests:run
Jan 18, 2022 12:50:44 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Jan 18, 2022 12:50:44 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 203 files. Enable logging at DEBUG level to see which files will be staged.
Jan 18, 2022 12:50:45 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Jan 18, 2022 12:50:45 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Jan 18, 2022 12:50:47 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 203 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Jan 18, 2022 12:50:48 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 203 files cached, 0 files newly uploaded in 1 seconds
Jan 18, 2022 12:50:48 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Jan 18, 2022 12:50:49 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <113698 bytes, hash 0cb206bb522ecd7715256b5a9ca6c678b9b783a15d28417f502064636f4f73a2> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-DLIGu1IuzXcVJWtanKbGeLm3g6FdKEF_UCBkY29Pc6I.pb
Jan 18, 2022 12:50:51 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Jan 18, 2022 12:50:51 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@71c17a57, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@640ab13c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1e0a864d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@440e3ce6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3e67f5f2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@fd53053, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4527f70a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@707b1a44, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7132a9dc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@57435801, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2da66a44, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@527fc8e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@61bfc9bf, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2c7106d9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@329bad59, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@862f408, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@178f268a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@33ecbd6c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5c723f2d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@432f521f]
Jan 18, 2022 12:50:51 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Jan 18, 2022 12:50:51 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Jan 18, 2022 12:50:51 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Jan 18, 2022 12:50:51 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Jan 18, 2022 12:50:51 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ac3f6f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1abebef3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@18f55704, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@67cefd84, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5fbe155, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6add8e3f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58a2b917, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@48904d5a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12bbfc54, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1491344a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@59b65dce, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1386313f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5e922647, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@433c6abb, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@288f173f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@a22c4d8, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@45cd7bc5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6504a875, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@35e26d05, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@29fa6b65]
Jan 18, 2022 12:50:51 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Jan 18, 2022 12:50:51 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Jan 18, 2022 12:50:51 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Jan 18, 2022 12:50:51 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Jan 18, 2022 12:50:51 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Jan 18, 2022 12:50:51 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Jan 18, 2022 12:50:51 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Jan 18, 2022 12:50:51 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Jan 18, 2022 12:50:51 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Jan 18, 2022 12:50:51 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Jan 18, 2022 12:50:51 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Jan 18, 2022 12:50:51 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.37.0-SNAPSHOT
Jan 18, 2022 12:50:52 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-01-18_04_50_51-14740612117850174335?project=apache-beam-testing
Jan 18, 2022 12:50:52 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2022-01-18_04_50_51-14740612117850174335
Jan 18, 2022 12:50:52 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2022-01-18_04_50_51-14740612117850174335
Jan 18, 2022 12:50:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2022-01-18T12:50:59.151Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-01-8hbk. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Jan 18, 2022 12:51:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-18T12:51:04.294Z: Worker configuration: e2-standard-2 in us-central1-b.
Jan 18, 2022 12:51:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-18T12:51:05.015Z: Expanding SplittableParDo operations into optimizable parts.
Jan 18, 2022 12:51:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-18T12:51:05.052Z: Expanding CollectionToSingleton operations into optimizable parts.
Jan 18, 2022 12:51:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-18T12:51:05.165Z: Expanding CoGroupByKey operations into optimizable parts.
Jan 18, 2022 12:51:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-18T12:51:05.237Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Jan 18, 2022 12:51:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-18T12:51:05.271Z: Expanding GroupByKey operations into streaming Read/Write steps
Jan 18, 2022 12:51:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-18T12:51:05.319Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jan 18, 2022 12:51:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-18T12:51:05.412Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jan 18, 2022 12:51:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-18T12:51:05.439Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Jan 18, 2022 12:51:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-18T12:51:05.461Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Jan 18, 2022 12:51:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-18T12:51:05.493Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Jan 18, 2022 12:51:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-18T12:51:05.517Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Jan 18, 2022 12:51:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-18T12:51:05.549Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jan 18, 2022 12:51:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-18T12:51:05.580Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jan 18, 2022 12:51:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-18T12:51:05.605Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jan 18, 2022 12:51:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-18T12:51:05.640Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Jan 18, 2022 12:51:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-18T12:51:05.675Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Jan 18, 2022 12:51:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-18T12:51:05.709Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Jan 18, 2022 12:51:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-18T12:51:05.733Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Jan 18, 2022 12:51:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-18T12:51:05.764Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jan 18, 2022 12:51:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-18T12:51:05.799Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jan 18, 2022 12:51:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-18T12:51:05.832Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jan 18, 2022 12:51:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-18T12:51:05.865Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Jan 18, 2022 12:51:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-18T12:51:05.913Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Jan 18, 2022 12:51:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-18T12:51:05.947Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Jan 18, 2022 12:51:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-18T12:51:05.982Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Jan 18, 2022 12:51:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-18T12:51:06.016Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Jan 18, 2022 12:51:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-18T12:51:06.048Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Jan 18, 2022 12:51:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-18T12:51:06.083Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Jan 18, 2022 12:51:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-18T12:51:06.116Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Jan 18, 2022 12:51:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-18T12:51:07.054Z: Starting 5 ****s in us-central1-b...
Jan 18, 2022 12:51:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-18T12:51:10.965Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jan 18, 2022 12:51:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-18T12:51:58.721Z: Autoscaling: Raised the number of ****s to 4 so that the pipeline can catch up with its backlog and keep up with its input rate.
Jan 18, 2022 12:51:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-18T12:51:58.755Z: Resized **** pool to 4, though goal was 5.  This could be a quota issue.
Jan 18, 2022 12:52:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-18T12:52:09.147Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Jan 18, 2022 12:52:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-18T12:52:55.717Z: Workers have started successfully.
Jan 18, 2022 12:52:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-18T12:52:55.747Z: Workers have started successfully.
Jan 18, 2022 4:00:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-18T16:00:41.201Z: Cancel request is committed for workflow job: 2022-01-18_04_50_51-14740612117850174335.
Jan 18, 2022 4:00:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-18T16:00:41.269Z: Cleaning up.
Jan 18, 2022 4:00:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-18T16:00:41.347Z: Stopping **** pool...
Jan 18, 2022 4:00:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-18T16:00:41.401Z: Stopping **** pool...
Jan 18, 2022 4:03:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-18T16:03:09.667Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Jan 18, 2022 4:03:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-18T16:03:09.739Z: Worker pool stopped.
Jan 18, 2022 4:03:17 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2022-01-18_04_50_51-14740612117850174335 finished with status CANCELLED.
Load test results for test (ID): b632a23d-4ce0-4257-b4f0-aac41b00737d and timestamp: 2022-01-18T12:50:45.103000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11157.359
dataflow_v2_java11_total_bytes_count             2.09129968E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220118124848
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:dac836ca72005995cd72a445950041cc9c35ee05d8e3de1c3a86c4c78874b9ba
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220118124848]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:dac836ca72005995cd72a445950041cc9c35ee05d8e3de1c3a86c4c78874b9ba]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220118124848] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:dac836ca72005995cd72a445950041cc9c35ee05d8e3de1c3a86c4c78874b9ba])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:dac836ca72005995cd72a445950041cc9c35ee05d8e3de1c3a86c4c78874b9ba
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:dac836ca72005995cd72a445950041cc9c35ee05d8e3de1c3a86c4c78874b9ba
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:dac836ca72005995cd72a445950041cc9c35ee05d8e3de1c3a86c4c78874b9ba].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings

Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.

BUILD FAILED in 3h 14m 46s
109 actionable tasks: 72 executed, 33 from cache, 4 up-to-date

Publishing build scan...
https://gradle.com/s/tfdgdoymiafw4

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #214

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/214/display/redirect?page=changes>

Changes:

[mmack] [BEAM-8806] Integration test for SqsIO using Localstack

[noreply] Merge pull request #16507: [BEAM-13137] Fixes ES utest size flakiness


------------------------------------------
[...truncated 49.41 KB...]
f1ef6929757d: Preparing
b77b9f639dab: Preparing
86e18190cb28: Preparing
957d46b6abc4: Preparing
3bb5258f46d2: Preparing
832e177bb500: Preparing
f9e18e59a565: Preparing
26a504e63be4: Preparing
8bf42db0de72: Preparing
31892cc314cb: Preparing
11936051f93b: Preparing
86e18190cb28: Waiting
b7487470add1: Waiting
f1ef6929757d: Waiting
957d46b6abc4: Waiting
7fbc39dcc9b0: Waiting
c6a90acafd41: Waiting
b77b9f639dab: Waiting
3bb5258f46d2: Waiting
832e177bb500: Waiting
f9e18e59a565: Waiting
8bf42db0de72: Waiting
f8843777e89a: Waiting
31892cc314cb: Waiting
11936051f93b: Waiting
eab39fe6ed46: Pushed
1f3138295f14: Pushed
1786af9fad54: Pushed
f8843777e89a: Pushed
4d370c837981: Pushed
522378014087: Pushed
7fbc39dcc9b0: Pushed
f1ef6929757d: Pushed
c6a90acafd41: Pushed
86e18190cb28: Pushed
3bb5258f46d2: Layer already exists
b7487470add1: Pushed
832e177bb500: Layer already exists
f9e18e59a565: Layer already exists
26a504e63be4: Layer already exists
8bf42db0de72: Layer already exists
31892cc314cb: Layer already exists
11936051f93b: Layer already exists
957d46b6abc4: Pushed
b77b9f639dab: Pushed
20220117124334: digest: sha256:5f6a819271593d7720664ac95381482ee91945556134d4ca521f74c6a85b7afe size: 4520

> Task :sdks:java:testing:load-tests:run
Jan 17, 2022 12:45:17 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Jan 17, 2022 12:45:18 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 203 files. Enable logging at DEBUG level to see which files will be staged.
Jan 17, 2022 12:45:18 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Jan 17, 2022 12:45:18 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Jan 17, 2022 12:45:21 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 203 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Jan 17, 2022 12:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 203 files cached, 0 files newly uploaded in 1 seconds
Jan 17, 2022 12:45:22 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Jan 17, 2022 12:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <113691 bytes, hash f157ebddf4a9bcb2f28a37f5ea034d72e600d06ada3d99785eb6d8dab444fc4d> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-8Vfr3fSpvLLyijf16gNNcuYA0GraPZl4XrbY2rRE_E0.pb
Jan 17, 2022 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Jan 17, 2022 12:45:24 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@71c17a57, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@640ab13c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1e0a864d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@440e3ce6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3e67f5f2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@fd53053, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4527f70a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@707b1a44, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7132a9dc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@57435801, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2da66a44, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@527fc8e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@61bfc9bf, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2c7106d9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@329bad59, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@862f408, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@178f268a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@33ecbd6c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5c723f2d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@432f521f]
Jan 17, 2022 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Jan 17, 2022 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Jan 17, 2022 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Jan 17, 2022 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Jan 17, 2022 12:45:24 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ac3f6f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1abebef3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@18f55704, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@67cefd84, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5fbe155, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6add8e3f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58a2b917, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@48904d5a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12bbfc54, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1491344a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@59b65dce, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1386313f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5e922647, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@433c6abb, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@288f173f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@a22c4d8, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@45cd7bc5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6504a875, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@35e26d05, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@29fa6b65]
Jan 17, 2022 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Jan 17, 2022 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Jan 17, 2022 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Jan 17, 2022 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Jan 17, 2022 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Jan 17, 2022 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Jan 17, 2022 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Jan 17, 2022 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Jan 17, 2022 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Jan 17, 2022 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Jan 17, 2022 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Jan 17, 2022 12:45:24 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.37.0-SNAPSHOT
Jan 17, 2022 12:45:25 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-01-17_04_45_24-468121990315778216?project=apache-beam-testing
Jan 17, 2022 12:45:25 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2022-01-17_04_45_24-468121990315778216
Jan 17, 2022 12:45:25 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2022-01-17_04_45_24-468121990315778216
Jan 17, 2022 12:45:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2022-01-17T12:45:31.917Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-01-rn5p. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Jan 17, 2022 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-17T12:45:36.069Z: Worker configuration: e2-standard-2 in us-central1-b.
Jan 17, 2022 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-17T12:45:36.894Z: Expanding SplittableParDo operations into optimizable parts.
Jan 17, 2022 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-17T12:45:36.930Z: Expanding CollectionToSingleton operations into optimizable parts.
Jan 17, 2022 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-17T12:45:37.006Z: Expanding CoGroupByKey operations into optimizable parts.
Jan 17, 2022 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-17T12:45:37.064Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Jan 17, 2022 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-17T12:45:37.090Z: Expanding GroupByKey operations into streaming Read/Write steps
Jan 17, 2022 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-17T12:45:37.155Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jan 17, 2022 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-17T12:45:37.268Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jan 17, 2022 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-17T12:45:37.300Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Jan 17, 2022 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-17T12:45:37.335Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Jan 17, 2022 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-17T12:45:37.371Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Jan 17, 2022 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-17T12:45:37.409Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Jan 17, 2022 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-17T12:45:37.441Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jan 17, 2022 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-17T12:45:37.475Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jan 17, 2022 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-17T12:45:37.508Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jan 17, 2022 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-17T12:45:37.529Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Jan 17, 2022 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-17T12:45:37.589Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Jan 17, 2022 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-17T12:45:37.627Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Jan 17, 2022 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-17T12:45:37.661Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Jan 17, 2022 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-17T12:45:37.694Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jan 17, 2022 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-17T12:45:37.716Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jan 17, 2022 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-17T12:45:37.749Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jan 17, 2022 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-17T12:45:37.785Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Jan 17, 2022 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-17T12:45:37.818Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Jan 17, 2022 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-17T12:45:37.843Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Jan 17, 2022 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-17T12:45:37.882Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Jan 17, 2022 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-17T12:45:37.906Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Jan 17, 2022 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-17T12:45:37.936Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Jan 17, 2022 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-17T12:45:37.960Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Jan 17, 2022 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-17T12:45:37.985Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Jan 17, 2022 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-17T12:45:38.345Z: Starting 5 ****s in us-central1-b...
Jan 17, 2022 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-17T12:45:51.065Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jan 17, 2022 12:46:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-17T12:46:23.338Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Jan 17, 2022 12:47:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-17T12:47:24.814Z: Workers have started successfully.
Jan 17, 2022 12:47:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-17T12:47:24.844Z: Workers have started successfully.
Jan 17, 2022 4:00:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-17T16:00:33.108Z: Cancel request is committed for workflow job: 2022-01-17_04_45_24-468121990315778216.
Jan 17, 2022 4:00:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-17T16:00:33.197Z: Cleaning up.
Jan 17, 2022 4:00:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-17T16:00:33.275Z: Stopping **** pool...
Jan 17, 2022 4:00:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-17T16:00:33.327Z: Stopping **** pool...
Jan 17, 2022 4:02:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-17T16:02:55.002Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Jan 17, 2022 4:02:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-17T16:02:55.038Z: Worker pool stopped.
Jan 17, 2022 4:03:02 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2022-01-17_04_45_24-468121990315778216 finished with status CANCELLED.
Load test results for test (ID): 50b2011e-619f-4c3f-95aa-82d9b4353f54 and timestamp: 2022-01-17T12:45:18.494000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11547.851
dataflow_v2_java11_total_bytes_count             1.39522555E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220117124334
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:5f6a819271593d7720664ac95381482ee91945556134d4ca521f74c6a85b7afe
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220117124334]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:5f6a819271593d7720664ac95381482ee91945556134d4ca521f74c6a85b7afe]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220117124334] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:5f6a819271593d7720664ac95381482ee91945556134d4ca521f74c6a85b7afe])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:5f6a819271593d7720664ac95381482ee91945556134d4ca521f74c6a85b7afe
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:5f6a819271593d7720664ac95381482ee91945556134d4ca521f74c6a85b7afe
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:5f6a819271593d7720664ac95381482ee91945556134d4ca521f74c6a85b7afe].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings

Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.

BUILD FAILED in 3h 19m 46s
109 actionable tasks: 72 executed, 33 from cache, 4 up-to-date

Publishing build scan...
https://scans.gradle.com/s/hcdqv6n4fypmi

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #213

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/213/display/redirect>

Changes:


------------------------------------------
[...truncated 48.70 KB...]
888359ca9712: Preparing
09abd1700af8: Preparing
a06a45d6c989: Preparing
9b4a3df3f9a5: Preparing
e251f6e49413: Preparing
3bb5258f46d2: Preparing
832e177bb500: Preparing
f9e18e59a565: Preparing
26a504e63be4: Preparing
8bf42db0de72: Preparing
31892cc314cb: Preparing
11936051f93b: Preparing
b71c3ea6d4ad: Waiting
5679b7a8343c: Waiting
832e177bb500: Waiting
625d95bb813d: Waiting
f9e18e59a565: Waiting
e251f6e49413: Waiting
3bb5258f46d2: Waiting
26a504e63be4: Waiting
9b4a3df3f9a5: Waiting
a06a45d6c989: Waiting
09abd1700af8: Waiting
8bf42db0de72: Waiting
11936051f93b: Waiting
f0012716c6c4: Pushed
347f860594a1: Pushed
026b708a42c6: Pushed
26d23edbd68b: Pushed
0587ec3095e2: Pushed
b71c3ea6d4ad: Pushed
625d95bb813d: Pushed
5679b7a8343c: Pushed
888359ca9712: Pushed
09abd1700af8: Pushed
9b4a3df3f9a5: Pushed
3bb5258f46d2: Layer already exists
832e177bb500: Layer already exists
f9e18e59a565: Layer already exists
26a504e63be4: Layer already exists
8bf42db0de72: Layer already exists
31892cc314cb: Layer already exists
11936051f93b: Layer already exists
e251f6e49413: Pushed
a06a45d6c989: Pushed
20220116124336: digest: sha256:3ee0eba6aa49a6a90fcee7d8d8be447d4fdbb22aefddc371ed450cfba84d3394 size: 4520

> Task :sdks:java:testing:load-tests:run
Jan 16, 2022 12:45:20 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Jan 16, 2022 12:45:20 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 203 files. Enable logging at DEBUG level to see which files will be staged.
Jan 16, 2022 12:45:21 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Jan 16, 2022 12:45:21 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Jan 16, 2022 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 203 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Jan 16, 2022 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 203 files cached, 0 files newly uploaded in 0 seconds
Jan 16, 2022 12:45:24 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Jan 16, 2022 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <113691 bytes, hash a8f5a88cc7b46c9247a1c1824df9bbe430eb2f10449bf63e5af36dcb2888903e> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-qPWojMe0bJJHocGCTfm75DDrLxBEm_Y-WvNtyyiIkD4.pb
Jan 16, 2022 12:45:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Jan 16, 2022 12:45:26 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4aa21f9d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@71c17a57, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@640ab13c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1e0a864d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@440e3ce6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3e67f5f2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@fd53053, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4527f70a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@707b1a44, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7132a9dc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@57435801, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2da66a44, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@527fc8e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@61bfc9bf, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2c7106d9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@329bad59, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@862f408, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@178f268a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@33ecbd6c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5c723f2d]
Jan 16, 2022 12:45:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Jan 16, 2022 12:45:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Jan 16, 2022 12:45:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Jan 16, 2022 12:45:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Jan 16, 2022 12:45:26 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@24eb65e3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ac3f6f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1abebef3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@18f55704, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@67cefd84, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5fbe155, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6add8e3f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58a2b917, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@48904d5a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12bbfc54, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1491344a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@59b65dce, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1386313f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5e922647, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@433c6abb, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@288f173f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@a22c4d8, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@45cd7bc5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6504a875, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@35e26d05]
Jan 16, 2022 12:45:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Jan 16, 2022 12:45:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Jan 16, 2022 12:45:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Jan 16, 2022 12:45:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Jan 16, 2022 12:45:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Jan 16, 2022 12:45:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Jan 16, 2022 12:45:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Jan 16, 2022 12:45:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Jan 16, 2022 12:45:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Jan 16, 2022 12:45:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Jan 16, 2022 12:45:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Jan 16, 2022 12:45:26 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.37.0-SNAPSHOT
Jan 16, 2022 12:45:27 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-01-16_04_45_26-1402912682750613138?project=apache-beam-testing
Jan 16, 2022 12:45:27 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2022-01-16_04_45_26-1402912682750613138
Jan 16, 2022 12:45:27 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2022-01-16_04_45_26-1402912682750613138
Jan 16, 2022 12:45:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2022-01-16T12:45:35.106Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-01-25i5. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Jan 16, 2022 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-16T12:45:38.714Z: Worker configuration: e2-standard-2 in us-central1-b.
Jan 16, 2022 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-16T12:45:39.342Z: Expanding SplittableParDo operations into optimizable parts.
Jan 16, 2022 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-16T12:45:39.365Z: Expanding CollectionToSingleton operations into optimizable parts.
Jan 16, 2022 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-16T12:45:39.421Z: Expanding CoGroupByKey operations into optimizable parts.
Jan 16, 2022 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-16T12:45:39.483Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Jan 16, 2022 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-16T12:45:39.520Z: Expanding GroupByKey operations into streaming Read/Write steps
Jan 16, 2022 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-16T12:45:39.573Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jan 16, 2022 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-16T12:45:39.675Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jan 16, 2022 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-16T12:45:39.703Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Jan 16, 2022 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-16T12:45:39.739Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Jan 16, 2022 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-16T12:45:39.761Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Jan 16, 2022 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-16T12:45:39.791Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Jan 16, 2022 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-16T12:45:39.828Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jan 16, 2022 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-16T12:45:39.853Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jan 16, 2022 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-16T12:45:39.880Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jan 16, 2022 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-16T12:45:39.917Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Jan 16, 2022 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-16T12:45:39.950Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Jan 16, 2022 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-16T12:45:39.977Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Jan 16, 2022 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-16T12:45:40.004Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Jan 16, 2022 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-16T12:45:40.039Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jan 16, 2022 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-16T12:45:40.068Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jan 16, 2022 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-16T12:45:40.125Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jan 16, 2022 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-16T12:45:40.147Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Jan 16, 2022 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-16T12:45:40.176Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Jan 16, 2022 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-16T12:45:40.212Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Jan 16, 2022 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-16T12:45:40.249Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Jan 16, 2022 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-16T12:45:40.279Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Jan 16, 2022 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-16T12:45:40.309Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Jan 16, 2022 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-16T12:45:40.343Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Jan 16, 2022 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-16T12:45:40.370Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Jan 16, 2022 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-16T12:45:40.715Z: Starting 5 ****s in us-central1-b...
Jan 16, 2022 12:46:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-16T12:46:06.849Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jan 16, 2022 12:46:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-16T12:46:26.711Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Jan 16, 2022 12:47:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-16T12:47:28.280Z: Workers have started successfully.
Jan 16, 2022 12:47:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-16T12:47:28.313Z: Workers have started successfully.
Jan 16, 2022 4:00:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-16T16:00:36.094Z: Cancel request is committed for workflow job: 2022-01-16_04_45_26-1402912682750613138.
Jan 16, 2022 4:00:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-16T16:00:36.158Z: Cleaning up.
Jan 16, 2022 4:00:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-16T16:00:36.236Z: Stopping **** pool...
Jan 16, 2022 4:00:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-16T16:00:36.332Z: Stopping **** pool...
Jan 16, 2022 4:02:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-16T16:02:56.841Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Jan 16, 2022 4:02:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-16T16:02:56.878Z: Worker pool stopped.
Jan 16, 2022 4:03:02 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2022-01-16_04_45_26-1402912682750613138 finished with status CANCELLED.
Load test results for test (ID): 2bad41c9-b3e9-4dc3-b4e5-e4d914e9b16b and timestamp: 2022-01-16T12:45:21.253000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11561.821
dataflow_v2_java11_total_bytes_count              1.7153606E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220116124336
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:3ee0eba6aa49a6a90fcee7d8d8be447d4fdbb22aefddc371ed450cfba84d3394
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220116124336]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:3ee0eba6aa49a6a90fcee7d8d8be447d4fdbb22aefddc371ed450cfba84d3394]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220116124336] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:3ee0eba6aa49a6a90fcee7d8d8be447d4fdbb22aefddc371ed450cfba84d3394])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:3ee0eba6aa49a6a90fcee7d8d8be447d4fdbb22aefddc371ed450cfba84d3394
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:3ee0eba6aa49a6a90fcee7d8d8be447d4fdbb22aefddc371ed450cfba84d3394
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:3ee0eba6aa49a6a90fcee7d8d8be447d4fdbb22aefddc371ed450cfba84d3394].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings

Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.

BUILD FAILED in 3h 19m 45s
109 actionable tasks: 72 executed, 33 from cache, 4 up-to-date

Publishing build scan...
https://scans.gradle.com/s/ixnm2xcvfr2hy

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #212

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/212/display/redirect?page=changes>

Changes:

[daria.malkova] Split builder into several builder for each step of pipeline execution

[Valentyn Tymofieiev] Provide API to check whether a hint is known.

[Valentyn Tymofieiev] [BEAM-12558] Fix doc typo.

[arietis27] [BEAM-13400] JDBC IO does not support UUID and JSONB PostgreSQL types

[jrmccluskey] [BEAM-10206] Resolve go vet errors in protox package

[noreply] Merge pull request #16482 from [BEAM-13429][Playground] Add builder for

[noreply] [BEAM-13590] Fix  abc imports from collections (#15850)

[jrmccluskey] Fix staticcheck errors in transforms directory

[jrmccluskey] Remove unnecessary fmt.Sprintf() in partition.go

[jrmccluskey] Replace bytes.Compare() with bytes.Equal() in test cases

[jrmccluskey] Replace string(buf.Bytes()) with buf.String() in coder_test.go

[jrmccluskey] Remove unnecessary blank identifier assignment in harness.go

[jrmccluskey] fix capitalized error strings in expansionx

[jrmccluskey] Clean up string cast of bytes in vet.go and corresponding tests

[jrmccluskey] Remove unnecessary fmt call in universal.go

[Robert Bradshaw] Remove tab from source.

[noreply] Redirecting cross-language transforms content (#16504)

[noreply] doc tweaks (#16498)

[noreply] [BEAM-12621] - Update Jenkins VMs to modern Ubuntu version (#16457)

[noreply] [BEAM-13664] Fix Primitives hashing benchmark (#16523)


------------------------------------------
[...truncated 50.03 KB...]
a38927abada2: Pushed
9d52d5eebd34: Pushed
a10ad78746e7: Pushed
2e4cadf9d5c9: Pushed
3bb5258f46d2: Layer already exists
832e177bb500: Layer already exists
5a5577dc5815: Pushed
26a504e63be4: Layer already exists
f9e18e59a565: Layer already exists
8bf42db0de72: Layer already exists
31892cc314cb: Layer already exists
11936051f93b: Layer already exists
1f004923d3a8: Pushed
9d2773eff5f0: Pushed
30c60089bba5: Pushed
20220115124341: digest: sha256:b4aab8fa68841baf5d9743fe7f91af1d690f6220e07724d9c38ac2c9914c8987 size: 4520

> Task :sdks:java:testing:load-tests:run
Jan 15, 2022 12:45:36 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Jan 15, 2022 12:45:37 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 203 files. Enable logging at DEBUG level to see which files will be staged.
Jan 15, 2022 12:45:37 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Jan 15, 2022 12:45:37 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Jan 15, 2022 12:45:39 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 203 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Jan 15, 2022 12:45:40 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 203 files cached, 0 files newly uploaded in 0 seconds
Jan 15, 2022 12:45:40 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Jan 15, 2022 12:45:40 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <113691 bytes, hash 45a7534e666ed3951601614982644eb7c6862ce5a11db442404000696bcaba8b> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-RadTTmZu05UWAWFJgmROt8aGLOWhHbRCQEAAaWvKuos.pb
Jan 15, 2022 12:45:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Jan 15, 2022 12:45:42 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@71c17a57, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@640ab13c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1e0a864d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@440e3ce6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3e67f5f2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@fd53053, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4527f70a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@707b1a44, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7132a9dc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@57435801, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2da66a44, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@527fc8e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@61bfc9bf, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2c7106d9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@329bad59, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@862f408, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@178f268a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@33ecbd6c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5c723f2d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@432f521f]
Jan 15, 2022 12:45:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Jan 15, 2022 12:45:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Jan 15, 2022 12:45:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Jan 15, 2022 12:45:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Jan 15, 2022 12:45:42 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ac3f6f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1abebef3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@18f55704, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@67cefd84, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5fbe155, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6add8e3f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58a2b917, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@48904d5a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12bbfc54, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1491344a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@59b65dce, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1386313f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5e922647, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@433c6abb, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@288f173f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@a22c4d8, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@45cd7bc5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6504a875, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@35e26d05, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@29fa6b65]
Jan 15, 2022 12:45:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Jan 15, 2022 12:45:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Jan 15, 2022 12:45:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Jan 15, 2022 12:45:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Jan 15, 2022 12:45:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Jan 15, 2022 12:45:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Jan 15, 2022 12:45:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Jan 15, 2022 12:45:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Jan 15, 2022 12:45:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Jan 15, 2022 12:45:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Jan 15, 2022 12:45:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Jan 15, 2022 12:45:42 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.37.0-SNAPSHOT
Jan 15, 2022 12:45:43 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-01-15_04_45_42-5972495277282877666?project=apache-beam-testing
Jan 15, 2022 12:45:43 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2022-01-15_04_45_42-5972495277282877666
Jan 15, 2022 12:45:43 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2022-01-15_04_45_42-5972495277282877666
Jan 15, 2022 12:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2022-01-15T12:45:50.079Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-01-rqwp. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Jan 15, 2022 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-15T12:45:54.494Z: Worker configuration: e2-standard-2 in us-central1-b.
Jan 15, 2022 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-15T12:45:55.278Z: Expanding SplittableParDo operations into optimizable parts.
Jan 15, 2022 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-15T12:45:55.304Z: Expanding CollectionToSingleton operations into optimizable parts.
Jan 15, 2022 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-15T12:45:55.374Z: Expanding CoGroupByKey operations into optimizable parts.
Jan 15, 2022 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-15T12:45:55.438Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Jan 15, 2022 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-15T12:45:55.463Z: Expanding GroupByKey operations into streaming Read/Write steps
Jan 15, 2022 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-15T12:45:55.518Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jan 15, 2022 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-15T12:45:55.609Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jan 15, 2022 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-15T12:45:55.636Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Jan 15, 2022 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-15T12:45:55.671Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Jan 15, 2022 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-15T12:45:55.695Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Jan 15, 2022 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-15T12:45:55.721Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Jan 15, 2022 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-15T12:45:55.757Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jan 15, 2022 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-15T12:45:55.791Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jan 15, 2022 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-15T12:45:55.823Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jan 15, 2022 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-15T12:45:55.859Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Jan 15, 2022 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-15T12:45:55.891Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Jan 15, 2022 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-15T12:45:55.923Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Jan 15, 2022 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-15T12:45:55.967Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Jan 15, 2022 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-15T12:45:55.998Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jan 15, 2022 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-15T12:45:56.024Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jan 15, 2022 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-15T12:45:56.055Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jan 15, 2022 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-15T12:45:56.091Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Jan 15, 2022 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-15T12:45:56.122Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Jan 15, 2022 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-15T12:45:56.157Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Jan 15, 2022 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-15T12:45:56.191Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Jan 15, 2022 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-15T12:45:56.217Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Jan 15, 2022 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-15T12:45:56.254Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Jan 15, 2022 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-15T12:45:56.287Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Jan 15, 2022 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-15T12:45:56.310Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Jan 15, 2022 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-15T12:45:56.692Z: Starting 5 ****s in us-central1-b...
Jan 15, 2022 12:46:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-15T12:46:02.931Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jan 15, 2022 12:46:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-15T12:46:47.362Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Jan 15, 2022 12:47:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-15T12:47:52.664Z: Workers have started successfully.
Jan 15, 2022 12:47:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-15T12:47:52.696Z: Workers have started successfully.
Jan 15, 2022 4:00:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-15T16:00:38.442Z: Cancel request is committed for workflow job: 2022-01-15_04_45_42-5972495277282877666.
Jan 15, 2022 4:00:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-15T16:00:38.513Z: Cleaning up.
Jan 15, 2022 4:00:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-15T16:00:38.586Z: Stopping **** pool...
Jan 15, 2022 4:00:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-15T16:00:38.627Z: Stopping **** pool...
Jan 15, 2022 4:02:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-15T16:02:56.710Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Jan 15, 2022 4:02:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-15T16:02:56.745Z: Worker pool stopped.
Jan 15, 2022 4:03:03 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2022-01-15_04_45_42-5972495277282877666 finished with status CANCELLED.
Load test results for test (ID): ac5c9b78-e09b-414f-aa13-8d8f6f1b5186 and timestamp: 2022-01-15T12:45:37.363000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11543.296
dataflow_v2_java11_total_bytes_count              2.3769515E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220115124341
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:b4aab8fa68841baf5d9743fe7f91af1d690f6220e07724d9c38ac2c9914c8987
Deleted: sha256:6ec89fcd6dead460de8cd87816975a20e293d629f57e0f8aa1981b39370e2040
Deleted: sha256:9406e0918aa6a65159c6371950d2273b035ad79951eda982ed379436b1689e57
Deleted: sha256:dd9514ceb6ee45a4457d87ec0bf0b5bb189d4492cdbf449cd84cfacc21cda27d
Deleted: sha256:6a99389e62d556f6ac71f56691be344caad6548ffc6cf41ea014b8c91d64d7c8
Deleted: sha256:721d0e0120ae6e4047050032cc5bb22752d74800c579d3e8a673eba9c447805e
Deleted: sha256:802a3df89ae80be2b998d12ff94c22e3a0022411bb44726458fc3318d3120831
Deleted: sha256:b8af2d9d05f5c118aefffa98f3bd5b64a84412973cb055e000a6db5e2109145c
Deleted: sha256:e383524933f3f8e12c64759addeadc875d29760e7cc56aad30b7bc1de72f9e4c
Deleted: sha256:93426dade9c37475acad3f81a1cc8599441f2a1266b62507cfb1922279aae667
Deleted: sha256:b3f4bc6205bebc85299e08d46f5210d164c31b6ff94b7e143e7b018f6b706b90
Deleted: sha256:369815a2cc65062115ceead7dfdb1e190ced068001d990b992763b9c46d8ed07
Deleted: sha256:99185a549500d62eb458a53da1aef8973d06ba789664978d0a84a859d284d4ea
Deleted: sha256:887d72ed2a58e05999be76b910e954b14b1a5bccc7d452052e3ca1fc7073599f
Deleted: sha256:9803b77d27647efb5784ea0f56de4cd501e286014fdb4337d577628cc50ffe4e
Deleted: sha256:9a469452d2c5fa5b0d577dfba41a05e711ed2813da0a806919bb2859f8cab6e5
Deleted: sha256:0030d1934c18ad3a16d46ad62a96d56e62b295048f09c1e724c19acebc0b3e84
Deleted: sha256:d4e4e4ee1c4ae5b73007c248c9008245084d02d12b3f853f0789d45564531349
Deleted: sha256:08a1cf90ff937882dbd5453bd9b143c3065f46e4d5c074a651567af46ddfac85
Deleted: sha256:dfba063e8571cbb3571c0f15b8a07c8a3a0ea8f427b2b5a1452f8fc74533588a
Deleted: sha256:125b72147cf2bc55607b07c7a33484111fd818f340e46f877b9450534816fd93
Deleted: sha256:f8b4ce50e8e4a24af0ea5a1c4c920eb51ed299041d3df04fe0bde194a3681710
Deleted: sha256:0bdf15d948a75b72ad25465d206dab850503dc0e29d38251aafcfcc0417959f2
Deleted: sha256:d0f9b4d91ffbc5840208672b93ed50a68685c760f5641af4c280ca04311ed71e
Deleted: sha256:732ade92fd45a248c02906f2d610281e57b8a11a5995b8d4208a23ba4cd7180e
Deleted: sha256:35176e0f2523094dcbf74f6f254c9886ba62ded7033895e321303a2661073ef8
Deleted: sha256:79ab0703975668a2cb13182563c6c38f6282bd6ab481f9cbae4cc81e26999d58
Deleted: sha256:60d13ff9ddaf0288f8a0863f7221202d9be383cb19bcaf551358eef08626acdf
Deleted: sha256:c771463041a724f302dc5b9d507d2e374a132a1357a08403fda243f49cce7a16
Deleted: sha256:641d763d8fe2505905e663e0800378f074dd57e37ff4516054ec7291e5723199
Deleted: sha256:bdf0afb294ff605ab8e448018187cbe17186a6bcf40e19d23da6c5bbc7b6d063
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220115124341]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:b4aab8fa68841baf5d9743fe7f91af1d690f6220e07724d9c38ac2c9914c8987]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220115124341] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:b4aab8fa68841baf5d9743fe7f91af1d690f6220e07724d9c38ac2c9914c8987])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:b4aab8fa68841baf5d9743fe7f91af1d690f6220e07724d9c38ac2c9914c8987
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:b4aab8fa68841baf5d9743fe7f91af1d690f6220e07724d9c38ac2c9914c8987
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:b4aab8fa68841baf5d9743fe7f91af1d690f6220e07724d9c38ac2c9914c8987].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings

Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.

BUILD FAILED in 3h 19m 43s
109 actionable tasks: 72 executed, 33 from cache, 4 up-to-date

Publishing build scan...
https://scans.gradle.com/s/n2hfr3kab5jhy

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #211

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/211/display/redirect?page=changes>

Changes:

[blais] python sdk examples: Fixed typo in wordcount example.

[noreply] [BEAM-13480] Increase pipeline timeout for

[noreply] Stronger typing inference for CoGBK. (#16465)

[noreply] [BEAM-12464] Change ProtoSchemaTranslator beam schema creation to match

[noreply] Introduce the notion of a JoinIndex for fewer shuffles. (#16101)

[noreply] Merge pull request #16467 from [BEAM-12164]: SpannerIO

[noreply] Merge pull request #16385 from [BEAM-13535] [Playground] add cancel

[noreply] Merge pull request #16485 from [BEAM-13486] [Playground] For unit tests

[heejong] [BEAM-13455] Remove duplicated artifacts when using multiple

[noreply] [BEAM-12572] Run java examples on multiple runners (#16450)


------------------------------------------
[...truncated 49.39 KB...]
cb43c1fe0996: Preparing
f0a1111d4491: Preparing
d7c4e2d10daf: Preparing
3bb5258f46d2: Preparing
832e177bb500: Preparing
f9e18e59a565: Preparing
26a504e63be4: Preparing
8bf42db0de72: Preparing
31892cc314cb: Preparing
11936051f93b: Preparing
5b11b246262a: Waiting
832e177bb500: Waiting
f9e18e59a565: Waiting
f0a1111d4491: Waiting
397b05db04c8: Waiting
26a504e63be4: Waiting
1244ddaeae0c: Waiting
d7c4e2d10daf: Waiting
9e77d401e742: Waiting
8bf42db0de72: Waiting
cb43c1fe0996: Waiting
3bb5258f46d2: Waiting
3249656ac991: Waiting
31892cc314cb: Waiting
11936051f93b: Waiting
38ab8d346dc1: Pushed
132ed9479fa7: Pushed
463ecb5f1c66: Pushed
7cdd743ca3f8: Pushed
3249656ac991: Pushed
5b11b246262a: Pushed
6562c9c587c2: Pushed
1244ddaeae0c: Pushed
9e77d401e742: Pushed
397b05db04c8: Pushed
f0a1111d4491: Pushed
3bb5258f46d2: Layer already exists
832e177bb500: Layer already exists
f9e18e59a565: Layer already exists
26a504e63be4: Layer already exists
8bf42db0de72: Layer already exists
31892cc314cb: Layer already exists
11936051f93b: Layer already exists
d7c4e2d10daf: Pushed
cb43c1fe0996: Pushed
20220114124337: digest: sha256:3311899c6758cae2f19eef6b7f8558b3bd5d4408f1c381879372a16dee6ea8d5 size: 4520

> Task :sdks:java:testing:load-tests:run
Jan 14, 2022 12:45:19 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Jan 14, 2022 12:45:20 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 203 files. Enable logging at DEBUG level to see which files will be staged.
Jan 14, 2022 12:45:20 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Jan 14, 2022 12:45:20 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Jan 14, 2022 12:45:23 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 203 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Jan 14, 2022 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 203 files cached, 0 files newly uploaded in 0 seconds
Jan 14, 2022 12:45:24 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Jan 14, 2022 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <113691 bytes, hash 9e97fc0410de10f7ade4bbba422a763d2d1d1ac1941ae34f9a9938dc9cb88544> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-npf8BBDeEPet5Lu6Qip2PS0dGsGUGuNPmpk43Jy4hUQ.pb
Jan 14, 2022 12:45:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Jan 14, 2022 12:45:26 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@71c17a57, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@640ab13c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1e0a864d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@440e3ce6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3e67f5f2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@fd53053, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4527f70a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@707b1a44, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7132a9dc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@57435801, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2da66a44, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@527fc8e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@61bfc9bf, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2c7106d9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@329bad59, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@862f408, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@178f268a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@33ecbd6c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5c723f2d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@432f521f]
Jan 14, 2022 12:45:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Jan 14, 2022 12:45:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Jan 14, 2022 12:45:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Jan 14, 2022 12:45:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Jan 14, 2022 12:45:26 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ac3f6f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1abebef3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@18f55704, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@67cefd84, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5fbe155, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6add8e3f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58a2b917, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@48904d5a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12bbfc54, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1491344a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@59b65dce, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1386313f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5e922647, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@433c6abb, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@288f173f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@a22c4d8, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@45cd7bc5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6504a875, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@35e26d05, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@29fa6b65]
Jan 14, 2022 12:45:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Jan 14, 2022 12:45:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Jan 14, 2022 12:45:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Jan 14, 2022 12:45:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Jan 14, 2022 12:45:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Jan 14, 2022 12:45:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Jan 14, 2022 12:45:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Jan 14, 2022 12:45:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Jan 14, 2022 12:45:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Jan 14, 2022 12:45:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Jan 14, 2022 12:45:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Jan 14, 2022 12:45:26 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.37.0-SNAPSHOT
Jan 14, 2022 12:45:27 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-01-14_04_45_26-9432682779076989600?project=apache-beam-testing
Jan 14, 2022 12:45:27 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2022-01-14_04_45_26-9432682779076989600
Jan 14, 2022 12:45:27 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2022-01-14_04_45_26-9432682779076989600
Jan 14, 2022 12:45:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2022-01-14T12:45:33.703Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-01-5xzc. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Jan 14, 2022 12:45:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-14T12:45:37.074Z: Worker configuration: e2-standard-2 in us-central1-b.
Jan 14, 2022 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-14T12:45:37.888Z: Expanding SplittableParDo operations into optimizable parts.
Jan 14, 2022 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-14T12:45:37.918Z: Expanding CollectionToSingleton operations into optimizable parts.
Jan 14, 2022 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-14T12:45:37.973Z: Expanding CoGroupByKey operations into optimizable parts.
Jan 14, 2022 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-14T12:45:38.023Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Jan 14, 2022 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-14T12:45:38.055Z: Expanding GroupByKey operations into streaming Read/Write steps
Jan 14, 2022 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-14T12:45:38.120Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jan 14, 2022 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-14T12:45:38.212Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jan 14, 2022 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-14T12:45:38.239Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Jan 14, 2022 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-14T12:45:38.259Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Jan 14, 2022 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-14T12:45:38.294Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Jan 14, 2022 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-14T12:45:38.324Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Jan 14, 2022 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-14T12:45:38.360Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jan 14, 2022 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-14T12:45:38.395Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jan 14, 2022 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-14T12:45:38.434Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jan 14, 2022 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-14T12:45:38.466Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Jan 14, 2022 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-14T12:45:38.499Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Jan 14, 2022 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-14T12:45:38.524Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Jan 14, 2022 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-14T12:45:38.556Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Jan 14, 2022 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-14T12:45:38.594Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jan 14, 2022 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-14T12:45:38.624Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jan 14, 2022 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-14T12:45:38.661Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jan 14, 2022 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-14T12:45:38.685Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Jan 14, 2022 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-14T12:45:38.721Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Jan 14, 2022 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-14T12:45:38.759Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Jan 14, 2022 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-14T12:45:38.786Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Jan 14, 2022 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-14T12:45:38.815Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Jan 14, 2022 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-14T12:45:38.833Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Jan 14, 2022 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-14T12:45:38.864Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Jan 14, 2022 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-14T12:45:38.895Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Jan 14, 2022 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-14T12:45:39.303Z: Starting 5 ****s in us-central1-b...
Jan 14, 2022 12:46:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-14T12:46:10.454Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jan 14, 2022 12:46:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-14T12:46:29.365Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Jan 14, 2022 12:47:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-14T12:47:29.629Z: Workers have started successfully.
Jan 14, 2022 12:47:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-14T12:47:29.660Z: Workers have started successfully.
Jan 14, 2022 4:01:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-14T16:01:31.599Z: Cancel request is committed for workflow job: 2022-01-14_04_45_26-9432682779076989600.
Jan 14, 2022 4:01:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-14T16:01:31.689Z: Cleaning up.
Jan 14, 2022 4:01:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-14T16:01:31.773Z: Stopping **** pool...
Jan 14, 2022 4:01:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-14T16:01:31.838Z: Stopping **** pool...
Jan 14, 2022 4:03:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-14T16:03:56.825Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Jan 14, 2022 4:03:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-14T16:03:56.862Z: Worker pool stopped.
Jan 14, 2022 4:04:03 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2022-01-14_04_45_26-9432682779076989600 finished with status CANCELLED.
Load test results for test (ID): 70334f63-b685-4db2-b8ee-d7c099a30f54 and timestamp: 2022-01-14T12:45:20.469000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11607.913
dataflow_v2_java11_total_bytes_count             2.42735823E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220114124337
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:3311899c6758cae2f19eef6b7f8558b3bd5d4408f1c381879372a16dee6ea8d5
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220114124337]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:3311899c6758cae2f19eef6b7f8558b3bd5d4408f1c381879372a16dee6ea8d5]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220114124337] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:3311899c6758cae2f19eef6b7f8558b3bd5d4408f1c381879372a16dee6ea8d5])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:3311899c6758cae2f19eef6b7f8558b3bd5d4408f1c381879372a16dee6ea8d5
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:3311899c6758cae2f19eef6b7f8558b3bd5d4408f1c381879372a16dee6ea8d5
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:3311899c6758cae2f19eef6b7f8558b3bd5d4408f1c381879372a16dee6ea8d5].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings

Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.

BUILD FAILED in 3h 20m 44s
109 actionable tasks: 72 executed, 33 from cache, 4 up-to-date

Publishing build scan...
https://scans.gradle.com/s/m7hbphjovuh3y

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #210

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/210/display/redirect?page=changes>

Changes:

[mmack] [BEAM-13638] Datatype of timestamp fields in SqsMessage for AWS IOs for

[noreply] Merge pull request #16469 from [BEAM-13623][Playground] [Bugfix] During

[noreply] Merge pull request #16149 from [BEAM-13113] [Playground] playground

[noreply] Merge pull request #16363 from [BEAM-13557] [Playground] show code

[noreply] Merge pull request #16374 from [BEAM-13398][Playground] Split LifeCycle

[noreply] [BEAM-13616][BEAM-13646] Update vendored calcite 1.28.0 with protobuf

[chamikaramj] Adds several example multi-language Python pipelines

[noreply] Merge pull request #16325 from [BEAM-13471] [Playground] Tag existing

[noreply] [BEAM-13399] Move service liveness polling to Runner type (#16487)


------------------------------------------
[...truncated 49.98 KB...]
92cadc1c340f: Pushed
015c4fab276a: Pushed
dd6d2620e6eb: Pushed
3df9b6f31d16: Pushed
5753c7b8c27f: Pushed
1fff288752e7: Pushed
3bb5258f46d2: Layer already exists
832e177bb500: Layer already exists
f9e18e59a565: Layer already exists
26a504e63be4: Layer already exists
8bf42db0de72: Layer already exists
31892cc314cb: Layer already exists
11936051f93b: Layer already exists
27d306cfcb2a: Pushed
afcfd170532b: Pushed
663df8e9d96f: Pushed
7875d03a61dd: Pushed
20220113124340: digest: sha256:b81bb13a3c6a55bdec04affd0d5c721e3fcf863ec4bad69f3c570661454df943 size: 4520

> Task :sdks:java:testing:load-tests:run
Jan 13, 2022 12:45:37 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Jan 13, 2022 12:45:37 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 203 files. Enable logging at DEBUG level to see which files will be staged.
Jan 13, 2022 12:45:38 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Jan 13, 2022 12:45:38 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Jan 13, 2022 12:45:41 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 203 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Jan 13, 2022 12:45:41 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 203 files cached, 0 files newly uploaded in 0 seconds
Jan 13, 2022 12:45:41 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Jan 13, 2022 12:45:41 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <113691 bytes, hash b1d0db656305729a8d6ed15cad89f5603f346a0ec7631f586db2029b214ca77c> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-sdDbZWMFcpqNbtFcrYn1YD80ag7HYx9YbbICmyFMp3w.pb
Jan 13, 2022 12:45:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Jan 13, 2022 12:45:43 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@71c17a57, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@640ab13c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1e0a864d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@440e3ce6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3e67f5f2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@fd53053, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4527f70a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@707b1a44, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7132a9dc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@57435801, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2da66a44, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@527fc8e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@61bfc9bf, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2c7106d9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@329bad59, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@862f408, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@178f268a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@33ecbd6c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5c723f2d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@432f521f]
Jan 13, 2022 12:45:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Jan 13, 2022 12:45:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Jan 13, 2022 12:45:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Jan 13, 2022 12:45:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Jan 13, 2022 12:45:43 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ac3f6f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1abebef3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@18f55704, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@67cefd84, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5fbe155, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6add8e3f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58a2b917, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@48904d5a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12bbfc54, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1491344a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@59b65dce, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1386313f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5e922647, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@433c6abb, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@288f173f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@a22c4d8, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@45cd7bc5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6504a875, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@35e26d05, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@29fa6b65]
Jan 13, 2022 12:45:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Jan 13, 2022 12:45:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Jan 13, 2022 12:45:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Jan 13, 2022 12:45:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Jan 13, 2022 12:45:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Jan 13, 2022 12:45:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Jan 13, 2022 12:45:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Jan 13, 2022 12:45:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Jan 13, 2022 12:45:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Jan 13, 2022 12:45:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Jan 13, 2022 12:45:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Jan 13, 2022 12:45:43 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.37.0-SNAPSHOT
Jan 13, 2022 12:45:44 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-01-13_04_45_43-18335074291361404234?project=apache-beam-testing
Jan 13, 2022 12:45:44 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2022-01-13_04_45_43-18335074291361404234
Jan 13, 2022 12:45:44 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2022-01-13_04_45_43-18335074291361404234
Jan 13, 2022 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2022-01-13T12:45:51.003Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-01-4j6r. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Jan 13, 2022 12:45:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-13T12:45:54.925Z: Worker configuration: e2-standard-2 in us-central1-b.
Jan 13, 2022 12:45:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-13T12:45:55.667Z: Expanding SplittableParDo operations into optimizable parts.
Jan 13, 2022 12:45:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-13T12:45:55.699Z: Expanding CollectionToSingleton operations into optimizable parts.
Jan 13, 2022 12:45:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-13T12:45:55.760Z: Expanding CoGroupByKey operations into optimizable parts.
Jan 13, 2022 12:45:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-13T12:45:55.833Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Jan 13, 2022 12:45:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-13T12:45:55.874Z: Expanding GroupByKey operations into streaming Read/Write steps
Jan 13, 2022 12:45:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-13T12:45:55.942Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jan 13, 2022 12:45:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-13T12:45:56.066Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jan 13, 2022 12:45:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-13T12:45:56.110Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Jan 13, 2022 12:45:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-13T12:45:56.145Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Jan 13, 2022 12:45:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-13T12:45:56.184Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Jan 13, 2022 12:45:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-13T12:45:56.219Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Jan 13, 2022 12:45:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-13T12:45:56.240Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jan 13, 2022 12:45:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-13T12:45:56.274Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jan 13, 2022 12:45:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-13T12:45:56.299Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jan 13, 2022 12:45:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-13T12:45:56.334Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Jan 13, 2022 12:45:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-13T12:45:56.363Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Jan 13, 2022 12:45:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-13T12:45:56.405Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Jan 13, 2022 12:45:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-13T12:45:56.447Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Jan 13, 2022 12:45:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-13T12:45:56.480Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jan 13, 2022 12:45:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-13T12:45:56.510Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jan 13, 2022 12:45:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-13T12:45:56.555Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jan 13, 2022 12:45:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-13T12:45:56.608Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Jan 13, 2022 12:45:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-13T12:45:56.656Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Jan 13, 2022 12:45:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-13T12:45:56.697Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Jan 13, 2022 12:45:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-13T12:45:56.726Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Jan 13, 2022 12:45:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-13T12:45:56.768Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Jan 13, 2022 12:45:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-13T12:45:56.808Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Jan 13, 2022 12:45:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-13T12:45:56.830Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Jan 13, 2022 12:45:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-13T12:45:56.860Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Jan 13, 2022 12:45:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-13T12:45:57.222Z: Starting 5 ****s in us-central1-b...
Jan 13, 2022 12:46:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-13T12:46:29.068Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jan 13, 2022 12:46:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-13T12:46:41.900Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Jan 13, 2022 12:47:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-13T12:47:43.323Z: Workers have started successfully.
Jan 13, 2022 12:47:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-13T12:47:43.343Z: Workers have started successfully.
Jan 13, 2022 3:12:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-13T15:12:57.420Z: Staged package animal-sniffer-annotations-1.20-vt1E38otwrj1wIzR1vDgznQJTsZ3gSYJaOA_wOd1Iqw.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/animal-sniffer-annotations-1.20-vt1E38otwrj1wIzR1vDgznQJTsZ3gSYJaOA_wOd1Iqw.jar' is inaccessible.
Jan 13, 2022 3:12:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-13T15:12:58.297Z: Staged package beam-model-job-management-2.37.0-SNAPSHOT-8QB2g9ssFGaKGT56IbOMrjcQcE9XP-LgiRhvff3RHeg.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.37.0-SNAPSHOT-8QB2g9ssFGaKGT56IbOMrjcQcE9XP-LgiRhvff3RHeg.jar' is inaccessible.
Jan 13, 2022 3:12:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-13T15:12:58.371Z: Staged package beam-runners-core-java-2.37.0-SNAPSHOT-sTk1eIXjfmaDIL0P3bF0KPVVvPGkThUdBb57OJlTv5U.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-java-2.37.0-SNAPSHOT-sTk1eIXjfmaDIL0P3bF0KPVVvPGkThUdBb57OJlTv5U.jar' is inaccessible.
Jan 13, 2022 3:12:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-13T15:12:58.495Z: Staged package beam-sdks-java-expansion-service-2.37.0-SNAPSHOT-CMK5mJnjvuuagVTF6zvpZgE5-7pMS8_SldvJq3otYp4.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.37.0-SNAPSHOT-CMK5mJnjvuuagVTF6zvpZgE5-7pMS8_SldvJq3otYp4.jar' is inaccessible.
Jan 13, 2022 3:12:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-13T15:12:58.542Z: Staged package beam-sdks-java-extensions-arrow-2.37.0-SNAPSHOT--6o0gUFJinkN4upVAkTzRdayVoYWlnjhjcsg2E3us1A.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-arrow-2.37.0-SNAPSHOT--6o0gUFJinkN4upVAkTzRdayVoYWlnjhjcsg2E3us1A.jar' is inaccessible.
Jan 13, 2022 3:12:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-13T15:12:58.613Z: Staged package beam-sdks-java-extensions-google-cloud-platform-core-2.37.0-SNAPSHOT-SrfdnSv8coZuWbihhjsTZYCINtfTYjEt4AvCESw7-dE.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.37.0-SNAPSHOT-SrfdnSv8coZuWbihhjsTZYCINtfTYjEt4AvCESw7-dE.jar' is inaccessible.
Jan 13, 2022 3:12:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-13T15:12:58.660Z: Staged package beam-sdks-java-extensions-protobuf-2.37.0-SNAPSHOT-hxuOLL6xOZT1Brk6taQ-FXgM8GLM-VmNPHe33xYH4j0.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.37.0-SNAPSHOT-hxuOLL6xOZT1Brk6taQ-FXgM8GLM-VmNPHe33xYH4j0.jar' is inaccessible.
Jan 13, 2022 3:12:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-13T15:12:58.753Z: Staged package beam-sdks-java-io-kafka-2.37.0-SNAPSHOT-BHRQGDx1kM7DSZHo50ZsxR2Y_2IANhpSpKUlptYGXGI.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-kafka-2.37.0-SNAPSHOT-BHRQGDx1kM7DSZHo50ZsxR2Y_2IANhpSpKUlptYGXGI.jar' is inaccessible.
Jan 13, 2022 3:12:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-13T15:12:58.795Z: Staged package beam-sdks-java-io-kinesis-2.37.0-SNAPSHOT-K5NpTSuLXSF3kKXHV9tZ2pX6bAANx1QDeOnHWoH7U1A.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-kinesis-2.37.0-SNAPSHOT-K5NpTSuLXSF3kKXHV9tZ2pX6bAANx1QDeOnHWoH7U1A.jar' is inaccessible.
Jan 13, 2022 3:12:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-13T15:12:58.842Z: Staged package beam-sdks-java-io-synthetic-2.37.0-SNAPSHOT-jarS5mJehhs9eb-NZbUKGKDXV420CBUVy0a4JE4KVs4.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-synthetic-2.37.0-SNAPSHOT-jarS5mJehhs9eb-NZbUKGKDXV420CBUVy0a4JE4KVs4.jar' is inaccessible.
Jan 13, 2022 3:12:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-13T15:12:58.893Z: Staged package beam-sdks-java-load-tests-2.37.0-SNAPSHOT-Mb3UZLeMQrMhn1ge-C2r1r257amJgt9KHHOfFBeIOhQ.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-load-tests-2.37.0-SNAPSHOT-Mb3UZLeMQrMhn1ge-C2r1r257amJgt9KHHOfFBeIOhQ.jar' is inaccessible.
Jan 13, 2022 3:12:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-13T15:12:58.938Z: Staged package beam-sdks-java-test-utils-2.37.0-SNAPSHOT-uDXAbi02WU3WDaNGqETzLc5tbQsW_968W1HOImTwz-s.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.37.0-SNAPSHOT-uDXAbi02WU3WDaNGqETzLc5tbQsW_968W1HOImTwz-s.jar' is inaccessible.
Jan 13, 2022 3:13:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-13T15:13:01.491Z: Staged package opencensus-contrib-http-util-0.28.0-ScPbKinx_bL3OSjL6pab0dQPq3zFu2JzAiur2W96eJs.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/opencensus-contrib-http-util-0.28.0-ScPbKinx_bL3OSjL6pab0dQPq3zFu2JzAiur2W96eJs.jar' is inaccessible.
Jan 13, 2022 3:13:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2022-01-13T15:13:02.061Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jan 13, 2022 4:00:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-13T16:00:33.694Z: Cancel request is committed for workflow job: 2022-01-13_04_45_43-18335074291361404234.
Jan 13, 2022 4:00:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-13T16:00:33.758Z: Cleaning up.
Jan 13, 2022 4:00:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-13T16:00:33.832Z: Stopping **** pool...
Jan 13, 2022 4:00:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-13T16:00:33.883Z: Stopping **** pool...
Jan 13, 2022 4:02:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-13T16:02:56.273Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Jan 13, 2022 4:02:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-13T16:02:56.308Z: Worker pool stopped.
Jan 13, 2022 4:03:02 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2022-01-13_04_45_43-18335074291361404234 finished with status CANCELLED.
Load test results for test (ID): aca1ea46-0b56-402b-8e02-839cf487ae34 and timestamp: 2022-01-13T12:45:37.952000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11530.456
dataflow_v2_java11_total_bytes_count             2.41150937E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220113124340
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:b81bb13a3c6a55bdec04affd0d5c721e3fcf863ec4bad69f3c570661454df943
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220113124340]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:b81bb13a3c6a55bdec04affd0d5c721e3fcf863ec4bad69f3c570661454df943]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220113124340] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:b81bb13a3c6a55bdec04affd0d5c721e3fcf863ec4bad69f3c570661454df943])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:b81bb13a3c6a55bdec04affd0d5c721e3fcf863ec4bad69f3c570661454df943
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:b81bb13a3c6a55bdec04affd0d5c721e3fcf863ec4bad69f3c570661454df943
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:b81bb13a3c6a55bdec04affd0d5c721e3fcf863ec4bad69f3c570661454df943].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings

Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.

BUILD FAILED in 3h 19m 42s
109 actionable tasks: 72 executed, 33 from cache, 4 up-to-date

Publishing build scan...
https://gradle.com/s/ypmx5uhgkzxjg

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #209

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/209/display/redirect?page=changes>

Changes:

[ningkang0957] [BEAM-13602] Prevented metrics gathering from failing bigtable io

[ningkang0957] make the code more pythonic

[mmack] [BEAM-13243][BEAM-8374] Add support for missing PublishResponse fields

[Robert Bradshaw] Optional args and kwargs for named external transforms.

[noreply] [BEAM-13628] Update SideInputCache to use full Transform and

[noreply] [BEAM-13432] Skip ExpansionService creation in Job Server (#16222)

[noreply] [BEAM-13616] Initial files for vendored gRPC 1.43.2 (#16460)


------------------------------------------
[...truncated 51.90 KB...]
Jan 12, 2022 12:45:20 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Jan 12, 2022 12:45:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <113691 bytes, hash 6522d52742e16e054120c6f4efe9c40732f4b02644e43f9cbc0517ce707fc1d5> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-ZSLVJ0LhbgVBIMb07-nEBzL0sCZE5D-cvAUXznB_wdU.pb
Jan 12, 2022 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Jan 12, 2022 12:45:22 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@71c17a57, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@640ab13c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1e0a864d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@440e3ce6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3e67f5f2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@fd53053, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4527f70a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@707b1a44, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7132a9dc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@57435801, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2da66a44, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@527fc8e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@61bfc9bf, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2c7106d9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@329bad59, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@862f408, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@178f268a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@33ecbd6c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5c723f2d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@432f521f]
Jan 12, 2022 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Jan 12, 2022 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Jan 12, 2022 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Jan 12, 2022 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Jan 12, 2022 12:45:22 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ac3f6f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1abebef3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@18f55704, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@67cefd84, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5fbe155, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6add8e3f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58a2b917, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@48904d5a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12bbfc54, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1491344a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@59b65dce, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1386313f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5e922647, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@433c6abb, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@288f173f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@a22c4d8, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@45cd7bc5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6504a875, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@35e26d05, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@29fa6b65]
Jan 12, 2022 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Jan 12, 2022 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Jan 12, 2022 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Jan 12, 2022 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Jan 12, 2022 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Jan 12, 2022 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Jan 12, 2022 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Jan 12, 2022 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Jan 12, 2022 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Jan 12, 2022 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Jan 12, 2022 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Jan 12, 2022 12:45:22 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.37.0-SNAPSHOT
Jan 12, 2022 12:45:26 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-01-12_04_45_22-2112328189684646921?project=apache-beam-testing
Jan 12, 2022 12:45:26 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2022-01-12_04_45_22-2112328189684646921
Jan 12, 2022 12:45:26 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2022-01-12_04_45_22-2112328189684646921
Jan 12, 2022 12:45:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2022-01-12T12:45:32.394Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-01-zia0. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Jan 12, 2022 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-12T12:45:37.324Z: Worker configuration: e2-standard-2 in us-central1-b.
Jan 12, 2022 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-12T12:45:37.975Z: Expanding SplittableParDo operations into optimizable parts.
Jan 12, 2022 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-12T12:45:38.013Z: Expanding CollectionToSingleton operations into optimizable parts.
Jan 12, 2022 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-12T12:45:38.080Z: Expanding CoGroupByKey operations into optimizable parts.
Jan 12, 2022 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-12T12:45:38.177Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Jan 12, 2022 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-12T12:45:38.205Z: Expanding GroupByKey operations into streaming Read/Write steps
Jan 12, 2022 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-12T12:45:38.270Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jan 12, 2022 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-12T12:45:38.362Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jan 12, 2022 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-12T12:45:38.397Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Jan 12, 2022 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-12T12:45:38.429Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Jan 12, 2022 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-12T12:45:38.463Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Jan 12, 2022 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-12T12:45:38.499Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Jan 12, 2022 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-12T12:45:38.533Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jan 12, 2022 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-12T12:45:38.575Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jan 12, 2022 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-12T12:45:38.621Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jan 12, 2022 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-12T12:45:38.651Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Jan 12, 2022 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-12T12:45:38.675Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Jan 12, 2022 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-12T12:45:38.699Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Jan 12, 2022 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-12T12:45:38.735Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Jan 12, 2022 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-12T12:45:38.761Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jan 12, 2022 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-12T12:45:38.785Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jan 12, 2022 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-12T12:45:38.823Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jan 12, 2022 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-12T12:45:38.846Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Jan 12, 2022 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-12T12:45:38.872Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Jan 12, 2022 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-12T12:45:38.909Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Jan 12, 2022 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-12T12:45:38.941Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Jan 12, 2022 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-12T12:45:38.979Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Jan 12, 2022 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-12T12:45:39.033Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Jan 12, 2022 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-12T12:45:39.057Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Jan 12, 2022 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-12T12:45:39.092Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Jan 12, 2022 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-12T12:45:39.442Z: Starting 5 ****s in us-central1-b...
Jan 12, 2022 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-12T12:46:02.553Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jan 12, 2022 12:46:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-12T12:46:22.971Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Jan 12, 2022 12:47:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-12T12:47:24.007Z: Workers have started successfully.
Jan 12, 2022 12:47:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-12T12:47:24.043Z: Workers have started successfully.
Jan 12, 2022 1:25:54 PM org.apache.beam.sdk.metrics.MetricsEnvironment getCurrentContainer
WARNING: Reporting metrics are not supported in the current execution environment.
Jan 12, 2022 1:51:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-12T13:51:40.929Z: Staged package gcsio-2.2.4-p9i4-xCmDBUow6YptIUTZYGtZZhNat4tocR78u1Q8kQ.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/gcsio-2.2.4-p9i4-xCmDBUow6YptIUTZYGtZZhNat4tocR78u1Q8kQ.jar' is inaccessible.
Jan 12, 2022 1:51:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-12T13:51:41.206Z: Staged package google-api-services-iamcredentials-v1-rev20210326-1.32.1-rzLlTs22sMZOFz3yEOfAB70ZZw2Gl99hr42n_k-f8-M.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-api-services-iamcredentials-v1-rev20210326-1.32.1-rzLlTs22sMZOFz3yEOfAB70ZZw2Gl99hr42n_k-f8-M.jar' is inaccessible.
Jan 12, 2022 1:51:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-12T13:51:43.788Z: Staged package util-2.2.4-r-7-okFosTGJSr5au-fNqmC_EgCN_DyG3X-dgI2RZFA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/util-2.2.4-r-7-okFosTGJSr5au-fNqmC_EgCN_DyG3X-dgI2RZFA.jar' is inaccessible.
Jan 12, 2022 1:51:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2022-01-12T13:51:43.863Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jan 12, 2022 1:54:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2022-01-12T13:54:44.130Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jan 12, 2022 1:57:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-12T13:57:41.082Z: Staged package gcsio-2.2.4-p9i4-xCmDBUow6YptIUTZYGtZZhNat4tocR78u1Q8kQ.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/gcsio-2.2.4-p9i4-xCmDBUow6YptIUTZYGtZZhNat4tocR78u1Q8kQ.jar' is inaccessible.
Jan 12, 2022 1:57:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-12T13:57:41.267Z: Staged package google-api-services-iamcredentials-v1-rev20210326-1.32.1-rzLlTs22sMZOFz3yEOfAB70ZZw2Gl99hr42n_k-f8-M.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-api-services-iamcredentials-v1-rev20210326-1.32.1-rzLlTs22sMZOFz3yEOfAB70ZZw2Gl99hr42n_k-f8-M.jar' is inaccessible.
Jan 12, 2022 1:57:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-12T13:57:43.441Z: Staged package util-2.2.4-r-7-okFosTGJSr5au-fNqmC_EgCN_DyG3X-dgI2RZFA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/util-2.2.4-r-7-okFosTGJSr5au-fNqmC_EgCN_DyG3X-dgI2RZFA.jar' is inaccessible.
Jan 12, 2022 1:57:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2022-01-12T13:57:43.533Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jan 12, 2022 2:00:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2022-01-12T14:00:43.788Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jan 12, 2022 2:03:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-12T14:03:41.562Z: Staged package gcsio-2.2.4-p9i4-xCmDBUow6YptIUTZYGtZZhNat4tocR78u1Q8kQ.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/gcsio-2.2.4-p9i4-xCmDBUow6YptIUTZYGtZZhNat4tocR78u1Q8kQ.jar' is inaccessible.
Jan 12, 2022 2:03:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-12T14:03:41.735Z: Staged package google-api-services-iamcredentials-v1-rev20210326-1.32.1-rzLlTs22sMZOFz3yEOfAB70ZZw2Gl99hr42n_k-f8-M.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-api-services-iamcredentials-v1-rev20210326-1.32.1-rzLlTs22sMZOFz3yEOfAB70ZZw2Gl99hr42n_k-f8-M.jar' is inaccessible.
Jan 12, 2022 2:03:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-12T14:03:44.117Z: Staged package util-2.2.4-r-7-okFosTGJSr5au-fNqmC_EgCN_DyG3X-dgI2RZFA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/util-2.2.4-r-7-okFosTGJSr5au-fNqmC_EgCN_DyG3X-dgI2RZFA.jar' is inaccessible.
Jan 12, 2022 2:03:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2022-01-12T14:03:44.199Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jan 12, 2022 4:00:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-12T16:00:37.254Z: Cancel request is committed for workflow job: 2022-01-12_04_45_22-2112328189684646921.
Jan 12, 2022 4:00:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-12T16:00:37.292Z: Cleaning up.
Jan 12, 2022 4:00:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-12T16:00:37.361Z: Stopping **** pool...
Jan 12, 2022 4:00:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-12T16:00:37.419Z: Stopping **** pool...
Jan 12, 2022 4:03:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-12T16:03:08.424Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Jan 12, 2022 4:03:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-12T16:03:08.462Z: Worker pool stopped.
Jan 12, 2022 4:03:14 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2022-01-12_04_45_22-2112328189684646921 finished with status CANCELLED.
Load test results for test (ID): d69e0a5f-b8fc-4db2-b8f3-b7f64275daa1 and timestamp: 2022-01-12T12:45:17.603000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11556.566
dataflow_v2_java11_total_bytes_count              2.5748142E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220112124333
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:644ea5771678e45504dcc8c323ea4f587c10cd548c21ff535811bd26e6e6f511
Deleted: sha256:0dfc9854b29ecdf055f1a42d1e398367977085dba482d0c8327d549aff737cf5
Deleted: sha256:a7ce9f3e98d3a30793328e3d86904ebe993029b65db25c864dbc67ab3376f2aa
Deleted: sha256:bb793563763b6e29ab48bb341b75591515dbbd697a5c0034333368da152e80d0
Deleted: sha256:53d40fcfaa8360999ac629d6ddc422ddad2fabced1b475953b75d819a6b458f2
Deleted: sha256:3bebf7a0f89af4357ae770d74649c3825cb3f35948685ad3eaa553e8d4f53ba1
Deleted: sha256:3ad6a3a5ded08454a5c93b3829329999ba1d6ed96e2ba1c4972adbb6f142fcae
Deleted: sha256:3364957c99fc3e105cd364238d8cf0de55e1310d933b4fb5a10f80396ad5d21b
Deleted: sha256:5b4fc224828c6ed98d577a185f18175c3fc506a7b82300e8e0fcd434e531ad63
Deleted: sha256:9b9cb659fcf54a259715307f353c0be84706c7789e34adf1b804d6fa5c097314
Deleted: sha256:b49bb393c62913aae01b4bb1ca8430c039bdb4848d175bf97cc73b638a804081
Deleted: sha256:d0c06bdcd092469d21c5586c75d8b31eb3f4246d3586c126d49e5a9c14a8537a
Deleted: sha256:15ff7d38261dbfffac9fcc75bd2754d5a90bf9a465e5756ec3087f24147a8a89
Deleted: sha256:7155b87e21468b2b3cf94b07e4b1786d75d50bfa33878c0edb5fe3cb9b40b16e
Deleted: sha256:29c6c85b48dbe99fcee69697fc91fcf888576fe8d9c445b120af1a3369231101
Deleted: sha256:e79aae3f0bfa8cd09685a25fde7fb97d57b38caf01b3546a9a3fc2a3d7e5747c
Deleted: sha256:3e8890237274d668fea6f34298ea6f4f31959102dfaa9f0cc237db9bfbab8cf8
Deleted: sha256:59fe8ac607ce86a2e423006de90e0a2de4a526e5d98ececadd19f9f42d7409ac
Deleted: sha256:b46fffbf14394c0a3eac04831d609a53ce93da7b0a4510d1a27e879ef4ba0fb6
Deleted: sha256:717cc93369dd7e8dc2e29fd3fc8d65e42b7764bf3cd0c793f3cf080734d24df8
Deleted: sha256:363270ea08d7b1d60c83d3587f8818a07ad30ef25fb4bad2029025e587bf14f0
Deleted: sha256:1a8c546957bd16daecd5308fa3d3433010ae9f208aa1cf606a56163608ec8f6f
Deleted: sha256:b81e48de15e461cde6c538a541bdd2cb9458411a81b0a199cab4fdba0b36272b
Deleted: sha256:3d8b798767443fcc97c23db81bd5244495b322e51cb9c2fcc94921c999e57689
Deleted: sha256:e1850bd3ab12ef03ed4a58607abf2861a8ee248a88f37747bcf1ae976d846bb7
Deleted: sha256:7791a3541d594230ac548fd9296107cbcc9a9d61f60c596358d3628df622e517
Deleted: sha256:eb685e3b53aa6bf21e9a15625559b33994f5079a423bebbf63c037b8a3ee4b13
Deleted: sha256:e7668c49143ff404a6e4a559adc0a61178ca2e044d809dc2e1bd2c5b08d8d076
Deleted: sha256:cda3b5a3ef13623d08b2489ffcff3bcdc090e0c931d8c190abd5aeb09e6cc8c7
Deleted: sha256:9bb76884f8cfe43892bf80649b821cbf4dc50ad6a3f3d9c880c753028b34fcf6
Deleted: sha256:7b86fc8d765768e6ddf9399618660d19a9ff1315be38d2db2a469e95b21f1c8b
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220112124333]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:644ea5771678e45504dcc8c323ea4f587c10cd548c21ff535811bd26e6e6f511]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220112124333] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:644ea5771678e45504dcc8c323ea4f587c10cd548c21ff535811bd26e6e6f511])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:644ea5771678e45504dcc8c323ea4f587c10cd548c21ff535811bd26e6e6f511
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:644ea5771678e45504dcc8c323ea4f587c10cd548c21ff535811bd26e6e6f511
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:644ea5771678e45504dcc8c323ea4f587c10cd548c21ff535811bd26e6e6f511].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings

Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.

BUILD FAILED in 3h 19m 58s
109 actionable tasks: 72 executed, 33 from cache, 4 up-to-date

Publishing build scan...
https://gradle.com/s/s27airpsigb3w

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #208

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/208/display/redirect?page=changes>

Changes:

[relax] don't close streams in finalize

[noreply] Loosen typing extensions bound

[Alexey Romanenko] [BEAM-4868] Bump com.amazonaws to 1.12.135

[noreply] [BEAM-12092] Bump jedis to version 4.0.1 (#16287)

[noreply] [BEAM-13534] Add automated port polling to expansion service runner if

[noreply] Merge pull request #16344 from [BEAM-13536][Playground][Bugfix] CI step

[noreply] Merge pull request #16359 from [BEAM-13545][Playground] Add

[noreply] Merge pull request #16384 from [BEAM-13308] [Playground] Getting

[noreply] Merge pull request #16306 from [BEAM-13447] [Playground] Add filling of

[noreply] Merge pull request #16361 from [BEAM-13543][Playground] Add logic of

[noreply] [BEAM-12562] Dataframe pipe implementation (#16256)

[noreply] Merge pull request #16338 from [BEAM-13528][Playground] Add liveness

[noreply] [BEAM-13626] Remap expanded outputs after merging. (#16471)

[noreply] Merge pull request #16147 from [BEAM-13359] [Playground] Tag existing

[noreply] [BEAM-3221] Improve documentation in model pipeline protos (#16474)

[noreply] [BEAM-13614] Add OnWindowExpiration support to the Java SDK harness and

[noreply] Merge pull request #16156 from [BEAM-13391] Fix temporary file format in

[mmack] [adhoc] Run spotlessApply on java examples to fix master


------------------------------------------
[...truncated 50.31 KB...]
79e74bb66b4c: Pushed
767109ab0085: Pushed
82f2be288639: Pushed
bc316e8f0ec8: Pushed
3bb5258f46d2: Layer already exists
832e177bb500: Layer already exists
f9e18e59a565: Layer already exists
26a504e63be4: Layer already exists
a3899668c851: Pushed
2441839b891a: Pushed
8bf42db0de72: Layer already exists
31892cc314cb: Layer already exists
11936051f93b: Layer already exists
0d6b484a5cd0: Pushed
92e46d8242ac: Pushed
20220111124551: digest: sha256:1ddc582ebc1c60bfe9a1b8605dbad5ff0f60246b05a617d1ff911f9c33443eee size: 4520

> Task :sdks:java:testing:load-tests:run
Jan 11, 2022 12:50:13 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Jan 11, 2022 12:50:14 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 203 files. Enable logging at DEBUG level to see which files will be staged.
Jan 11, 2022 12:50:17 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Jan 11, 2022 12:50:17 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Jan 11, 2022 12:50:25 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 203 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Jan 11, 2022 12:50:31 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 203 files cached, 0 files newly uploaded in 5 seconds
Jan 11, 2022 12:50:31 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Jan 11, 2022 12:50:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <113691 bytes, hash f70c1dafa2c18a8791e54430fcce566ee19b5d2602ac9f089c3f988c575f1e5f> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-9wwdr6LBioeR5UQw_M5WbuGbXSYCrJ8InD-YjFdfHl8.pb
Jan 11, 2022 12:50:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Jan 11, 2022 12:50:38 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1e0a864d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@440e3ce6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3e67f5f2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@fd53053, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4527f70a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@707b1a44, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7132a9dc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@57435801, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2da66a44, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@527fc8e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@61bfc9bf, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2c7106d9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@329bad59, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@862f408, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@178f268a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@33ecbd6c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5c723f2d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@432f521f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2d7a9786, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7bab5898]
Jan 11, 2022 12:50:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Jan 11, 2022 12:50:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Jan 11, 2022 12:50:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Jan 11, 2022 12:50:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Jan 11, 2022 12:50:38 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@18f55704, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@67cefd84, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5fbe155, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6add8e3f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58a2b917, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@48904d5a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12bbfc54, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1491344a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@59b65dce, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1386313f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5e922647, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@433c6abb, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@288f173f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@a22c4d8, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@45cd7bc5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6504a875, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@35e26d05, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@29fa6b65, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7c72ecc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@47406941]
Jan 11, 2022 12:50:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Jan 11, 2022 12:50:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Jan 11, 2022 12:50:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Jan 11, 2022 12:50:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Jan 11, 2022 12:50:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Jan 11, 2022 12:50:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Jan 11, 2022 12:50:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Jan 11, 2022 12:50:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Jan 11, 2022 12:50:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Jan 11, 2022 12:50:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Jan 11, 2022 12:50:39 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Jan 11, 2022 12:50:39 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.37.0-SNAPSHOT
Jan 11, 2022 12:50:40 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-01-11_04_50_39-13865195789965900416?project=apache-beam-testing
Jan 11, 2022 12:50:40 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2022-01-11_04_50_39-13865195789965900416
Jan 11, 2022 12:50:40 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2022-01-11_04_50_39-13865195789965900416
Jan 11, 2022 12:50:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2022-01-11T12:50:47.162Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-01-ssqc. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Jan 11, 2022 12:50:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-11T12:50:50.799Z: Worker configuration: e2-standard-2 in us-central1-f.
Jan 11, 2022 12:50:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-11T12:50:51.592Z: Expanding SplittableParDo operations into optimizable parts.
Jan 11, 2022 12:50:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-11T12:50:51.627Z: Expanding CollectionToSingleton operations into optimizable parts.
Jan 11, 2022 12:50:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-11T12:50:51.708Z: Expanding CoGroupByKey operations into optimizable parts.
Jan 11, 2022 12:50:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-11T12:50:51.784Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Jan 11, 2022 12:50:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-11T12:50:51.824Z: Expanding GroupByKey operations into streaming Read/Write steps
Jan 11, 2022 12:50:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-11T12:50:51.892Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jan 11, 2022 12:50:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-11T12:50:52.011Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jan 11, 2022 12:50:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-11T12:50:52.052Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Jan 11, 2022 12:50:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-11T12:50:52.090Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Jan 11, 2022 12:50:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-11T12:50:52.129Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Jan 11, 2022 12:50:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-11T12:50:52.158Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Jan 11, 2022 12:50:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-11T12:50:52.194Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jan 11, 2022 12:50:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-11T12:50:52.233Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jan 11, 2022 12:50:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-11T12:50:52.277Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jan 11, 2022 12:50:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-11T12:50:52.300Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Jan 11, 2022 12:50:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-11T12:50:52.348Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Jan 11, 2022 12:50:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-11T12:50:52.418Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Jan 11, 2022 12:50:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-11T12:50:52.468Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Jan 11, 2022 12:50:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-11T12:50:52.516Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jan 11, 2022 12:50:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-11T12:50:52.553Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jan 11, 2022 12:50:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-11T12:50:52.579Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jan 11, 2022 12:50:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-11T12:50:52.611Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Jan 11, 2022 12:50:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-11T12:50:52.653Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Jan 11, 2022 12:50:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-11T12:50:52.687Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Jan 11, 2022 12:50:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-11T12:50:52.716Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Jan 11, 2022 12:50:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-11T12:50:52.761Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Jan 11, 2022 12:50:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-11T12:50:52.793Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Jan 11, 2022 12:50:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-11T12:50:52.817Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Jan 11, 2022 12:50:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-11T12:50:52.859Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Jan 11, 2022 12:50:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-11T12:50:53.230Z: Starting 5 ****s in us-central1-f...
Jan 11, 2022 12:51:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-11T12:51:08.681Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jan 11, 2022 12:51:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-11T12:51:37.357Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Jan 11, 2022 12:52:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-11T12:52:40.678Z: Workers have started successfully.
Jan 11, 2022 12:52:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-11T12:52:40.723Z: Workers have started successfully.
Jan 11, 2022 4:00:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-11T16:00:37.859Z: Cancel request is committed for workflow job: 2022-01-11_04_50_39-13865195789965900416.
Jan 11, 2022 4:00:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-11T16:00:37.959Z: Cleaning up.
Jan 11, 2022 4:00:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-11T16:00:38.045Z: Stopping **** pool...
Jan 11, 2022 4:00:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-11T16:00:38.115Z: Stopping **** pool...
Jan 11, 2022 4:03:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-11T16:03:00.877Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Jan 11, 2022 4:03:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-11T16:03:00.938Z: Worker pool stopped.
Jan 11, 2022 4:03:09 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2022-01-11_04_50_39-13865195789965900416 finished with status CANCELLED.
Load test results for test (ID): 6c7e72b3-10d3-4bdf-b17c-c728559d1dc5 and timestamp: 2022-01-11T12:50:15.651000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11246.394
dataflow_v2_java11_total_bytes_count             1.67860974E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220111124551
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:1ddc582ebc1c60bfe9a1b8605dbad5ff0f60246b05a617d1ff911f9c33443eee
Deleted: sha256:fa8c0e241abe97db0594353ebe104dde22b84b5c1b1abb582ce661bcf7e820c0
Deleted: sha256:3d30f8eed14cdd09eb1a17d800a6b7adb5b95bdbbf538817a7cf9778c7e32e56
Deleted: sha256:d0ef289bdb7c4c3b91f06964ff3a922260dd21d5e134a1d0148489cf1090c39f
Deleted: sha256:01509efe3a40fa8506cde87ccbc6a428fb9036ca414bbbc0620f1b53e13dcf9b
Deleted: sha256:c51f7c819bf73e620e85c93a562eb3e8a4b466cb7a7a7423b9720eebe2259952
Deleted: sha256:bd0f302f96aecae043163f9dd2b75596f43b2153ba624055789b92e8f818948e
Deleted: sha256:aa827bc37ccfb1e87382b2fc6a44882edb662a8b2859634fb8a919dcaad9f5be
Deleted: sha256:fdf67147c0f4b3f49e3fd7fe205cc384733d7934e87e83148552aba67cd98abc
Deleted: sha256:6a37d7672c308246fafcaba4581e2fda5cf9439e161c3dce60c5a905e0fbef54
Deleted: sha256:3b1a05917abcd7bb5655b3ae611f16f718fb83a2091a90ee6db3e8157e2a4b13
Deleted: sha256:468a1a98f868e4c5b0a38cc87489d434b0ab8dd0622184ceaac641a119ce35f0
Deleted: sha256:b34d75f5fde59b04ef9a91354dffb789dbef1bff8411c3abe23bf81b303f1a2b
Deleted: sha256:d32fc5d59db8365ec7cba18fea4026399c6c8ebeb8be30c44102a94bb855cf27
Deleted: sha256:af93d20e19f0c83d3733beeb8697a9517e87a3ddc779e59865f396304b71b531
Deleted: sha256:419ded4f7a31274087ef908da2628bdd5f8a3bf062577f031384c02f458ff49c
Deleted: sha256:60654b57228d25313b5fabea64fe91733b102b90fa84ec4adc5a6f71d58f08e1
Deleted: sha256:512fa6cd261a4bcd296d6d160d636ceb65984e217f65f146b3a9c9d4b57fe0b7
Deleted: sha256:2e2505dcda6216420c6935ed34ba16f65f9137522bafa6bf9ac46ebbe4026a4d
Deleted: sha256:026364ec2081d163bf76d84e8b7a967fa6ea874c301153ad6081560cef3546dc
Deleted: sha256:b6a9710454434f0f9801d342a4e02557f3201158a7430aa3a78694b4e75605f3
Deleted: sha256:fd7200751f4c714cf14f4fe8d1d5e53d4654f0ae682eb8a8b17b351c0353969c
Deleted: sha256:f4814528739ca31004037e9810860294f9c0e26fefecddc1473f5418965d9897
Deleted: sha256:661ece5de9e7d4608364905efd781773f79f11640c99e7c5f3dff71c6ecce5c2
Deleted: sha256:c12cf600ef5d02af63441cdf565cac7684f4d2644c071b7530c27906110b6cdf
Deleted: sha256:22853f80256fbb2351ce641821b1d6cc8f5c8f9e42e0211cd675399cdf8972ef
Deleted: sha256:3e47d4a2c6d65227a761f6084dbd7c2df28b555780cb4c01129a00b121461653
Deleted: sha256:e73c6408e63e53722487e8292c4971c8aebd1ee5141f5f38211cd9b5246c4127
Deleted: sha256:2a42f8e9eb56c8a1568e0b4254c3bea31bb71fe3db14e424341db7cd434694bc
Deleted: sha256:ead634f1494fe7553a5c0196e52a80beedf1167daba8104cd2ab37de5a9df0cb
Deleted: sha256:68e886c63ba834f77a04b7b58c9c5773e5105accdf5c31d3405e7c1873351819
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220111124551]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:1ddc582ebc1c60bfe9a1b8605dbad5ff0f60246b05a617d1ff911f9c33443eee]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220111124551] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:1ddc582ebc1c60bfe9a1b8605dbad5ff0f60246b05a617d1ff911f9c33443eee])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:1ddc582ebc1c60bfe9a1b8605dbad5ff0f60246b05a617d1ff911f9c33443eee
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:1ddc582ebc1c60bfe9a1b8605dbad5ff0f60246b05a617d1ff911f9c33443eee
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:1ddc582ebc1c60bfe9a1b8605dbad5ff0f60246b05a617d1ff911f9c33443eee].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings

Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.

BUILD FAILED in 3h 18m 34s
109 actionable tasks: 72 executed, 33 from cache, 4 up-to-date

Publishing build scan...
https://gradle.com/s/njrz3wln5udns

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #207

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/207/display/redirect?page=changes>

Changes:

[Alexey Romanenko] [BEAM-8727] Bump software.amazon.awssdk to 2.17.106


------------------------------------------
[...truncated 50.03 KB...]
dc39cbdf68ec: Pushed
8dff6234b371: Pushed
aaa06dfc4b09: Pushed
781753bd57dc: Pushed
3bb5258f46d2: Layer already exists
97b5df71f94d: Pushed
4065f5dd1ce4: Pushed
832e177bb500: Layer already exists
f9e18e59a565: Layer already exists
26a504e63be4: Layer already exists
31892cc314cb: Layer already exists
11936051f93b: Layer already exists
8bf42db0de72: Layer already exists
2433e8a37a43: Pushed
2cfca14c1553: Pushed
20220110125822: digest: sha256:63aa69ba85114b9a7e55d1e8cf4c7c5ef2151b9ae2729504823103f9c2e7130c size: 4520

> Task :sdks:java:testing:load-tests:run
Jan 10, 2022 1:00:07 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Jan 10, 2022 1:00:07 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 203 files. Enable logging at DEBUG level to see which files will be staged.
Jan 10, 2022 1:00:08 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Jan 10, 2022 1:00:08 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Jan 10, 2022 1:00:10 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 203 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Jan 10, 2022 1:00:11 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 203 files cached, 0 files newly uploaded in 0 seconds
Jan 10, 2022 1:00:11 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Jan 10, 2022 1:00:11 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <113691 bytes, hash 388b486776f0e09d258c614d195174106d80d9980d6b4542c5a2f9b30810eef3> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-OItIZ3bw4J0ljGFNGVF0EG2A2ZgNa0VCxaL5swgQ7vM.pb
Jan 10, 2022 1:00:12 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Jan 10, 2022 1:00:12 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@37045b48, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@60b34931, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4aa21f9d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@71c17a57, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@640ab13c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1e0a864d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@440e3ce6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3e67f5f2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@fd53053, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4527f70a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@707b1a44, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7132a9dc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@57435801, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2da66a44, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@527fc8e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@61bfc9bf, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2c7106d9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@329bad59, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@862f408, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@178f268a]
Jan 10, 2022 1:00:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Jan 10, 2022 1:00:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Jan 10, 2022 1:00:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Jan 10, 2022 1:00:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Jan 10, 2022 1:00:13 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@791c12e3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@b112b13, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@24eb65e3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ac3f6f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1abebef3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@18f55704, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@67cefd84, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5fbe155, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6add8e3f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58a2b917, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@48904d5a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12bbfc54, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1491344a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@59b65dce, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1386313f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5e922647, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@433c6abb, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@288f173f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@a22c4d8, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@45cd7bc5]
Jan 10, 2022 1:00:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Jan 10, 2022 1:00:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Jan 10, 2022 1:00:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Jan 10, 2022 1:00:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Jan 10, 2022 1:00:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Jan 10, 2022 1:00:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Jan 10, 2022 1:00:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Jan 10, 2022 1:00:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Jan 10, 2022 1:00:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Jan 10, 2022 1:00:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Jan 10, 2022 1:00:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Jan 10, 2022 1:00:13 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.37.0-SNAPSHOT
Jan 10, 2022 1:00:14 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-01-10_05_00_13-957728361604083376?project=apache-beam-testing
Jan 10, 2022 1:00:14 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2022-01-10_05_00_13-957728361604083376
Jan 10, 2022 1:00:14 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2022-01-10_05_00_13-957728361604083376
Jan 10, 2022 1:00:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2022-01-10T13:00:22.542Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-01-pqck. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Jan 10, 2022 1:00:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-10T13:00:25.918Z: Worker configuration: e2-standard-2 in us-central1-f.
Jan 10, 2022 1:00:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-10T13:00:26.521Z: Expanding SplittableParDo operations into optimizable parts.
Jan 10, 2022 1:00:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-10T13:00:26.551Z: Expanding CollectionToSingleton operations into optimizable parts.
Jan 10, 2022 1:00:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-10T13:00:26.615Z: Expanding CoGroupByKey operations into optimizable parts.
Jan 10, 2022 1:00:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-10T13:00:26.686Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Jan 10, 2022 1:00:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-10T13:00:26.713Z: Expanding GroupByKey operations into streaming Read/Write steps
Jan 10, 2022 1:00:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-10T13:00:26.779Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jan 10, 2022 1:00:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-10T13:00:26.886Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jan 10, 2022 1:00:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-10T13:00:26.930Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Jan 10, 2022 1:00:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-10T13:00:26.957Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Jan 10, 2022 1:00:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-10T13:00:26.981Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Jan 10, 2022 1:00:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-10T13:00:27.015Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Jan 10, 2022 1:00:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-10T13:00:27.039Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jan 10, 2022 1:00:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-10T13:00:27.061Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jan 10, 2022 1:00:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-10T13:00:27.096Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jan 10, 2022 1:00:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-10T13:00:27.117Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Jan 10, 2022 1:00:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-10T13:00:27.141Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Jan 10, 2022 1:00:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-10T13:00:27.165Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Jan 10, 2022 1:00:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-10T13:00:27.200Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Jan 10, 2022 1:00:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-10T13:00:27.235Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jan 10, 2022 1:00:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-10T13:00:27.257Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jan 10, 2022 1:00:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-10T13:00:27.279Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jan 10, 2022 1:00:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-10T13:00:27.306Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Jan 10, 2022 1:00:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-10T13:00:27.341Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Jan 10, 2022 1:00:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-10T13:00:27.371Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Jan 10, 2022 1:00:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-10T13:00:27.405Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Jan 10, 2022 1:00:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-10T13:00:27.466Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Jan 10, 2022 1:00:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-10T13:00:27.502Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Jan 10, 2022 1:00:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-10T13:00:27.534Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Jan 10, 2022 1:00:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-10T13:00:27.558Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Jan 10, 2022 1:00:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-10T13:00:27.878Z: Starting 5 ****s in us-central1-f...
Jan 10, 2022 1:00:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-10T13:00:43.129Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jan 10, 2022 1:01:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-10T13:01:13.115Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Jan 10, 2022 1:02:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-10T13:02:12.988Z: Workers have started successfully.
Jan 10, 2022 1:02:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-10T13:02:13.052Z: Workers have started successfully.
Jan 10, 2022 4:00:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-10T16:00:41.704Z: Cancel request is committed for workflow job: 2022-01-10_05_00_13-957728361604083376.
Jan 10, 2022 4:00:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-10T16:00:41.762Z: Cleaning up.
Jan 10, 2022 4:00:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-10T16:00:41.833Z: Stopping **** pool...
Jan 10, 2022 4:00:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-10T16:00:41.898Z: Stopping **** pool...
Jan 10, 2022 4:03:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-10T16:03:09.599Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Jan 10, 2022 4:03:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-10T16:03:09.629Z: Worker pool stopped.
Jan 10, 2022 4:03:16 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2022-01-10_05_00_13-957728361604083376 finished with status CANCELLED.
Load test results for test (ID): e507aab1-8236-48ec-a3fe-a38e8f856745 and timestamp: 2022-01-10T13:00:07.869000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 10672.675
dataflow_v2_java11_total_bytes_count             1.42755092E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220110125822
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:63aa69ba85114b9a7e55d1e8cf4c7c5ef2151b9ae2729504823103f9c2e7130c
Deleted: sha256:fa0e1a4ce5b525b74b8cf0aafcdbdb045329d0d7370e6696ba4ad26cef3489ef
Deleted: sha256:85acfc7854af4e97f17d405b70ac5e7a7e17e7a09178e588316b1dedc5d1706b
Deleted: sha256:ace68236c694a0a5433bc06318009de33e862b7e03bf29532edcc1455bbde412
Deleted: sha256:01b34adcd0d0293859c5f33ec501837743a1ce87b42adf3caeea91d0bd3ec26c
Deleted: sha256:1139ea4cfbd758b08ce84d6a90e1bc54e6427d4df91098338ccfd5eb1676c367
Deleted: sha256:4d427d4c444fc118741eb98a1c81bb9e4de9147c3fd70486fa8982e7544eff16
Deleted: sha256:05d6c5c335f013051c496942b3f8572d24551b8623e1ee2c4b157d31b5242070
Deleted: sha256:87f143ec89ef0e6684282a8f9ab743241ec1c2701f1a5818ffc1779aa5ccb292
Deleted: sha256:556cee2737c37c3fb362525348494d84870cef7461ff38d976daf7f3d90df16b
Deleted: sha256:bccc893ab7dfd97aec6a49d1c4758a8997bcb4d7a596a8a7c685edb7cb5910b5
Deleted: sha256:a7ef68fa8820af19b21ba3c986f7ceb305393a408b758f3671b78ef72fb489a2
Deleted: sha256:957e6d572f767bf39858b557ac23276264cda27459d2366ff405e958e0e88284
Deleted: sha256:c9fdcfd0e1f3465fd0192b48f8c4a4ff539559892c039c389baa057b4d9114ac
Deleted: sha256:1c98ab856d749af49de5c8d8d518225dc250206ccde0f728535a51669d847bf2
Deleted: sha256:d9e66597176a1120109786215b146cd6c23a285b9246624ee9203892f969e2b0
Deleted: sha256:c20972ec3c14861f6769c488569233d6b66e48e96fb5434faf9e979f5d16018a
Deleted: sha256:d3dd2cda4fbba257a2d384735e2460a6396505e0c2f6329b5eb9232f1334d571
Deleted: sha256:9db193f2f1eac5bbefc67715dc10547b8dffe5221068d0ecb777586d0d1fc1e4
Deleted: sha256:751b35ff270e0728b29808c54150b830ee37b9404c3af033f8a96e276444c5d1
Deleted: sha256:4a316c6804be6bcf4a5bf1f46649a67090f421a7ffe6561059d7a04214de30f2
Deleted: sha256:99a6f2c086cc67f50b6a307b6c2781176404ecfb58b2d74bd18a4480adc5999f
Deleted: sha256:cc3203d7ab3cf78fc7efce3dfa05c125ea70c36282b04597e84db367df456db3
Deleted: sha256:b946168f2939b86c9a4c9fb5617ba79b95d9cb507560784c8703b49953a45dfb
Deleted: sha256:666dd19db6bbc81187c4ec4fd7f4bd13649f956b219dd9923603c3df94de6be1
Deleted: sha256:884d8f324381440e42556e0478a0c68ee7aee10e49a0f5d03d37d22c71cf8cef
Deleted: sha256:b7e9f8b8820a373a4a4f792b235f0af3b95588e85aab2913e311e9467a63a8a5
Deleted: sha256:7b3c51279f492fc862c1ae6e48abe722061d3c350c81e5919cbd227389bfb22a
Deleted: sha256:15cb9b0e04f34ad82d94a969f66dd3644b559b3c5c15d7d30cf35442c9d47b86
Deleted: sha256:c36325729922a2f2eb07663a16ffdd4cef1b753ace054985e421904986b8d802
Deleted: sha256:67e1725730cbb360174ffe8d552143a62cfec96cdd07dd27857cbf025fd58fcc
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220110125822]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:63aa69ba85114b9a7e55d1e8cf4c7c5ef2151b9ae2729504823103f9c2e7130c]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220110125822] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:63aa69ba85114b9a7e55d1e8cf4c7c5ef2151b9ae2729504823103f9c2e7130c])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:63aa69ba85114b9a7e55d1e8cf4c7c5ef2151b9ae2729504823103f9c2e7130c
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:63aa69ba85114b9a7e55d1e8cf4c7c5ef2151b9ae2729504823103f9c2e7130c
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:63aa69ba85114b9a7e55d1e8cf4c7c5ef2151b9ae2729504823103f9c2e7130c].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings

Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.

BUILD FAILED in 3h 5m 12s
109 actionable tasks: 72 executed, 33 from cache, 4 up-to-date

Publishing build scan...
https://gradle.com/s/r75wcrrleeucw

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #206

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/206/display/redirect>

Changes:


------------------------------------------
[...truncated 49.62 KB...]
59d5b5a212d5: Preparing
0923754a550c: Preparing
2722fe1fa18d: Preparing
36d4b61891b3: Preparing
3bb5258f46d2: Preparing
832e177bb500: Preparing
f9e18e59a565: Preparing
26a504e63be4: Preparing
8bf42db0de72: Preparing
31892cc314cb: Preparing
11936051f93b: Preparing
59d5b5a212d5: Waiting
26a504e63be4: Waiting
31892cc314cb: Waiting
f9e18e59a565: Waiting
3bb5258f46d2: Waiting
11936051f93b: Waiting
140b8d74f702: Waiting
832e177bb500: Waiting
0923754a550c: Waiting
7fd4d9613c12: Waiting
36d4b61891b3: Waiting
2722fe1fa18d: Waiting
e935230b7b37: Waiting
81ec05982540: Waiting
a63813f52acb: Pushed
bd8bb9523d67: Pushed
47e8e840661e: Pushed
140b8d74f702: Pushed
81ec05982540: Pushed
7b677137a2dd: Pushed
721d4902504f: Pushed
59d5b5a212d5: Pushed
2722fe1fa18d: Pushed
3bb5258f46d2: Layer already exists
7fd4d9613c12: Pushed
832e177bb500: Layer already exists
f9e18e59a565: Layer already exists
26a504e63be4: Layer already exists
31892cc314cb: Layer already exists
8bf42db0de72: Layer already exists
e935230b7b37: Pushed
11936051f93b: Layer already exists
36d4b61891b3: Pushed
0923754a550c: Pushed
20220109124334: digest: sha256:21470cf6ac2a11570c9e965019fca4a3334847485e2c986a2b77a13b8f1ec5ce size: 4520

> Task :sdks:java:testing:load-tests:run
Jan 09, 2022 12:45:28 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Jan 09, 2022 12:45:28 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 203 files. Enable logging at DEBUG level to see which files will be staged.
Jan 09, 2022 12:45:29 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Jan 09, 2022 12:45:29 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Jan 09, 2022 12:45:31 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 203 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Jan 09, 2022 12:45:32 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 203 files cached, 0 files newly uploaded in 0 seconds
Jan 09, 2022 12:45:32 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Jan 09, 2022 12:45:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <113691 bytes, hash 9fdaaec445b9463556b4ee46e1c99c3406a4c287e2f35a5cd83183e9e4c0f9b8> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-n9quxEW5RjVWtO5G4cmcNAakwofi81pc2DGD6eTA-bg.pb
Jan 09, 2022 12:45:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Jan 09, 2022 12:45:34 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4b039c6d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7f5b9db, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@507d64aa, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@37045b48, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@60b34931, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4aa21f9d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@71c17a57, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@640ab13c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1e0a864d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@440e3ce6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3e67f5f2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@fd53053, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4527f70a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@707b1a44, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7132a9dc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@57435801, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2da66a44, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@527fc8e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@61bfc9bf, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2c7106d9]
Jan 09, 2022 12:45:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Jan 09, 2022 12:45:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Jan 09, 2022 12:45:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Jan 09, 2022 12:45:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Jan 09, 2022 12:45:34 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4a37191a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5854a18, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@d5556bf, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@791c12e3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@b112b13, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@24eb65e3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ac3f6f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1abebef3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@18f55704, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@67cefd84, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5fbe155, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6add8e3f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58a2b917, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@48904d5a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12bbfc54, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1491344a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@59b65dce, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1386313f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5e922647, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@433c6abb]
Jan 09, 2022 12:45:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Jan 09, 2022 12:45:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Jan 09, 2022 12:45:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Jan 09, 2022 12:45:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Jan 09, 2022 12:45:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Jan 09, 2022 12:45:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Jan 09, 2022 12:45:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Jan 09, 2022 12:45:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Jan 09, 2022 12:45:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Jan 09, 2022 12:45:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Jan 09, 2022 12:45:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Jan 09, 2022 12:45:34 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.37.0-SNAPSHOT
Jan 09, 2022 12:45:35 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-01-09_04_45_34-13620770890621873228?project=apache-beam-testing
Jan 09, 2022 12:45:35 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2022-01-09_04_45_34-13620770890621873228
Jan 09, 2022 12:45:35 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2022-01-09_04_45_34-13620770890621873228
Jan 09, 2022 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2022-01-09T12:45:41.886Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-01-z7xc. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Jan 09, 2022 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-09T12:45:44.864Z: Worker configuration: e2-standard-2 in us-central1-f.
Jan 09, 2022 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-09T12:45:45.526Z: Expanding SplittableParDo operations into optimizable parts.
Jan 09, 2022 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-09T12:45:45.553Z: Expanding CollectionToSingleton operations into optimizable parts.
Jan 09, 2022 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-09T12:45:45.622Z: Expanding CoGroupByKey operations into optimizable parts.
Jan 09, 2022 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-09T12:45:45.698Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Jan 09, 2022 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-09T12:45:45.726Z: Expanding GroupByKey operations into streaming Read/Write steps
Jan 09, 2022 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-09T12:45:45.792Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jan 09, 2022 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-09T12:45:45.901Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jan 09, 2022 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-09T12:45:45.925Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Jan 09, 2022 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-09T12:45:45.959Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Jan 09, 2022 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-09T12:45:46.004Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Jan 09, 2022 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-09T12:45:46.036Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Jan 09, 2022 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-09T12:45:46.068Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jan 09, 2022 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-09T12:45:46.103Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jan 09, 2022 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-09T12:45:46.136Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jan 09, 2022 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-09T12:45:46.169Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Jan 09, 2022 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-09T12:45:46.203Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Jan 09, 2022 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-09T12:45:46.236Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Jan 09, 2022 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-09T12:45:46.272Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Jan 09, 2022 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-09T12:45:46.304Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jan 09, 2022 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-09T12:45:46.336Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jan 09, 2022 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-09T12:45:46.367Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jan 09, 2022 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-09T12:45:46.401Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Jan 09, 2022 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-09T12:45:46.428Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Jan 09, 2022 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-09T12:45:46.454Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Jan 09, 2022 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-09T12:45:46.501Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Jan 09, 2022 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-09T12:45:46.532Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Jan 09, 2022 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-09T12:45:46.557Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Jan 09, 2022 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-09T12:45:46.590Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Jan 09, 2022 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-09T12:45:46.627Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Jan 09, 2022 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-09T12:45:46.988Z: Starting 5 ****s in us-central1-f...
Jan 09, 2022 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-09T12:46:18.284Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jan 09, 2022 12:46:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-09T12:46:36.994Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Jan 09, 2022 12:47:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-09T12:47:30.914Z: Workers have started successfully.
Jan 09, 2022 12:47:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-09T12:47:30.947Z: Workers have started successfully.
Jan 09, 2022 4:00:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-09T16:00:37.453Z: Cancel request is committed for workflow job: 2022-01-09_04_45_34-13620770890621873228.
Jan 09, 2022 4:00:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-09T16:00:37.528Z: Cleaning up.
Jan 09, 2022 4:00:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-09T16:00:37.599Z: Stopping **** pool...
Jan 09, 2022 4:00:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-09T16:00:37.651Z: Stopping **** pool...
Jan 09, 2022 4:02:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-09T16:02:59.197Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Jan 09, 2022 4:02:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-09T16:02:59.236Z: Worker pool stopped.
Jan 09, 2022 4:03:04 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2022-01-09_04_45_34-13620770890621873228 finished with status CANCELLED.
Load test results for test (ID): 268b1372-c261-4e4c-9154-7396d7688cd4 and timestamp: 2022-01-09T12:45:29.116000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11553.163
dataflow_v2_java11_total_bytes_count             1.63210408E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220109124334
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:21470cf6ac2a11570c9e965019fca4a3334847485e2c986a2b77a13b8f1ec5ce
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220109124334]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:21470cf6ac2a11570c9e965019fca4a3334847485e2c986a2b77a13b8f1ec5ce]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220109124334] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:21470cf6ac2a11570c9e965019fca4a3334847485e2c986a2b77a13b8f1ec5ce])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:21470cf6ac2a11570c9e965019fca4a3334847485e2c986a2b77a13b8f1ec5ce
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:21470cf6ac2a11570c9e965019fca4a3334847485e2c986a2b77a13b8f1ec5ce
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:21470cf6ac2a11570c9e965019fca4a3334847485e2c986a2b77a13b8f1ec5ce].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings

Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.

BUILD FAILED in 3h 19m 47s
109 actionable tasks: 72 executed, 33 from cache, 4 up-to-date

Publishing build scan...
https://gradle.com/s/h4u7ylchxk52w

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #205

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/205/display/redirect?page=changes>

Changes:

[Alexey Romanenko] [adhoc] Fix BigTableIO description

[noreply] [BEAM-13015] Remove dead code now that all instances have migrated to

[noreply] [BEAM-13386] Add RLock support for cloudpickle (#16250)

[danthev] Fix overflow


------------------------------------------
[...truncated 49.30 KB...]
5f73317e9c84: Preparing
5dced2c18203: Preparing
c6511c243ebf: Preparing
3bb5258f46d2: Preparing
832e177bb500: Preparing
f9e18e59a565: Preparing
26a504e63be4: Preparing
8bf42db0de72: Preparing
31892cc314cb: Preparing
11936051f93b: Preparing
feb4bb1c7f43: Waiting
5f73317e9c84: Waiting
40a438875f1a: Waiting
5dced2c18203: Waiting
5c89e53604b4: Waiting
26a504e63be4: Waiting
c6511c243ebf: Waiting
8bf42db0de72: Waiting
4e6ede92f183: Waiting
31892cc314cb: Waiting
f9e18e59a565: Waiting
3bb5258f46d2: Waiting
c5c90b910af3: Waiting
832e177bb500: Waiting
11936051f93b: Waiting
da3429be8715: Pushed
2fdee7c17392: Pushed
a66adcd28cba: Pushed
acdd8856e218: Pushed
feb4bb1c7f43: Pushed
77e5c10da981: Pushed
4e6ede92f183: Pushed
5c89e53604b4: Pushed
c5c90b910af3: Pushed
40a438875f1a: Pushed
832e177bb500: Layer already exists
3bb5258f46d2: Layer already exists
5dced2c18203: Pushed
f9e18e59a565: Layer already exists
26a504e63be4: Layer already exists
8bf42db0de72: Layer already exists
31892cc314cb: Layer already exists
11936051f93b: Layer already exists
c6511c243ebf: Pushed
5f73317e9c84: Pushed
20220108124333: digest: sha256:778efe755b70f21de23933fbc0772b54ab528f99893fab28dc90a2a79f0b0f49 size: 4520

> Task :sdks:java:testing:load-tests:run
Jan 08, 2022 12:45:18 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Jan 08, 2022 12:45:18 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 203 files. Enable logging at DEBUG level to see which files will be staged.
Jan 08, 2022 12:45:19 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Jan 08, 2022 12:45:19 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Jan 08, 2022 12:45:21 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 203 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Jan 08, 2022 12:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 203 files cached, 0 files newly uploaded in 0 seconds
Jan 08, 2022 12:45:22 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Jan 08, 2022 12:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <113691 bytes, hash 61d14fb95e9bb2607751fb988bd43c314371eb02718da255f9e20dff8224af63> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-YdFPuV6bsmB3UfuYi9Q8MUNx6wJxjaJV-eIN_4Ikr2M.pb
Jan 08, 2022 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Jan 08, 2022 12:45:24 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7f5b9db, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@507d64aa, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@37045b48, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@60b34931, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4aa21f9d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@71c17a57, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@640ab13c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1e0a864d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@440e3ce6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3e67f5f2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@fd53053, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4527f70a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@707b1a44, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7132a9dc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@57435801, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2da66a44, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@527fc8e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@61bfc9bf, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2c7106d9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@329bad59]
Jan 08, 2022 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Jan 08, 2022 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Jan 08, 2022 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Jan 08, 2022 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Jan 08, 2022 12:45:24 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5854a18, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@d5556bf, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@791c12e3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@b112b13, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@24eb65e3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ac3f6f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1abebef3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@18f55704, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@67cefd84, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5fbe155, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6add8e3f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58a2b917, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@48904d5a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12bbfc54, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1491344a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@59b65dce, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1386313f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5e922647, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@433c6abb, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@288f173f]
Jan 08, 2022 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Jan 08, 2022 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Jan 08, 2022 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Jan 08, 2022 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Jan 08, 2022 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Jan 08, 2022 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Jan 08, 2022 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Jan 08, 2022 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Jan 08, 2022 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Jan 08, 2022 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Jan 08, 2022 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Jan 08, 2022 12:45:24 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.37.0-SNAPSHOT
Jan 08, 2022 12:45:25 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-01-08_04_45_24-5049803997360451121?project=apache-beam-testing
Jan 08, 2022 12:45:25 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2022-01-08_04_45_24-5049803997360451121
Jan 08, 2022 12:45:25 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2022-01-08_04_45_24-5049803997360451121
Jan 08, 2022 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2022-01-08T12:45:33.683Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-01-hzso. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Jan 08, 2022 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-08T12:45:39.437Z: Worker configuration: e2-standard-2 in us-central1-f.
Jan 08, 2022 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-08T12:45:40.058Z: Expanding SplittableParDo operations into optimizable parts.
Jan 08, 2022 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-08T12:45:40.112Z: Expanding CollectionToSingleton operations into optimizable parts.
Jan 08, 2022 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-08T12:45:40.200Z: Expanding CoGroupByKey operations into optimizable parts.
Jan 08, 2022 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-08T12:45:40.256Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Jan 08, 2022 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-08T12:45:40.299Z: Expanding GroupByKey operations into streaming Read/Write steps
Jan 08, 2022 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-08T12:45:40.355Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jan 08, 2022 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-08T12:45:40.452Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jan 08, 2022 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-08T12:45:40.478Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Jan 08, 2022 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-08T12:45:40.510Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Jan 08, 2022 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-08T12:45:40.542Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Jan 08, 2022 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-08T12:45:40.568Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Jan 08, 2022 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-08T12:45:40.597Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jan 08, 2022 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-08T12:45:40.620Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jan 08, 2022 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-08T12:45:40.651Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jan 08, 2022 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-08T12:45:40.688Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Jan 08, 2022 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-08T12:45:40.729Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Jan 08, 2022 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-08T12:45:40.777Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Jan 08, 2022 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-08T12:45:40.811Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Jan 08, 2022 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-08T12:45:40.835Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jan 08, 2022 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-08T12:45:40.865Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jan 08, 2022 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-08T12:45:40.897Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jan 08, 2022 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-08T12:45:40.919Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Jan 08, 2022 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-08T12:45:40.947Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Jan 08, 2022 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-08T12:45:40.981Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Jan 08, 2022 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-08T12:45:41.015Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Jan 08, 2022 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-08T12:45:41.048Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Jan 08, 2022 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-08T12:45:41.077Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Jan 08, 2022 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-08T12:45:41.121Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Jan 08, 2022 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-08T12:45:41.155Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Jan 08, 2022 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-08T12:45:41.532Z: Starting 5 ****s in us-central1-f...
Jan 08, 2022 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-08T12:45:49.267Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jan 08, 2022 12:46:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-08T12:46:31.705Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Jan 08, 2022 12:47:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-08T12:47:26.836Z: Workers have started successfully.
Jan 08, 2022 12:47:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-08T12:47:26.861Z: Workers have started successfully.
Jan 08, 2022 4:00:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-08T16:00:27.604Z: Cancel request is committed for workflow job: 2022-01-08_04_45_24-5049803997360451121.
Jan 08, 2022 4:00:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-08T16:00:28.485Z: Cleaning up.
Jan 08, 2022 4:00:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-08T16:00:28.554Z: Stopping **** pool...
Jan 08, 2022 4:00:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-08T16:00:28.615Z: Stopping **** pool...
Jan 08, 2022 4:02:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-08T16:02:53.930Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Jan 08, 2022 4:02:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-08T16:02:53.973Z: Worker pool stopped.
Jan 08, 2022 4:02:59 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2022-01-08_04_45_24-5049803997360451121 finished with status CANCELLED.
Load test results for test (ID): 485e827d-dfe8-4cf5-8114-013bd14af19c and timestamp: 2022-01-08T12:45:19.244000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11544.857
dataflow_v2_java11_total_bytes_count             1.90950472E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220108124333
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:778efe755b70f21de23933fbc0772b54ab528f99893fab28dc90a2a79f0b0f49
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220108124333]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:778efe755b70f21de23933fbc0772b54ab528f99893fab28dc90a2a79f0b0f49]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220108124333] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:778efe755b70f21de23933fbc0772b54ab528f99893fab28dc90a2a79f0b0f49])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:778efe755b70f21de23933fbc0772b54ab528f99893fab28dc90a2a79f0b0f49
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:778efe755b70f21de23933fbc0772b54ab528f99893fab28dc90a2a79f0b0f49
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:778efe755b70f21de23933fbc0772b54ab528f99893fab28dc90a2a79f0b0f49].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings

Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.

BUILD FAILED in 3h 19m 44s
109 actionable tasks: 72 executed, 33 from cache, 4 up-to-date

Publishing build scan...
https://gradle.com/s/ziymue32vbx2i

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #204

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/204/display/redirect?page=changes>

Changes:

[heejong] [BEAM-13091] Generate missing staged names from hash for Dataflow runner

[heejong] add test

[arietis27] [BEAM-13604] NPE while getting null from BigDecimal column

[noreply] Fixed empty labels treated as wildcard when matching cache files

[noreply] [BEAM-13570] Remove erroneous compileClasspath dependency. (#16438)

[noreply] [BEAM-13015] Plumb through process wide and bundle cache through the

[noreply] [BEAM-13015] Cache the state backed iterable used for large GBK results.

[noreply] Fix formatting/alignment (#16443)

[noreply] Merge pull request #16183 from [BEAM-13427] [Playground]  show logs for

[noreply] [BEAM-10277] re-write encoding position tests to declare schema protos

[noreply] Update local_env_tests.yml (#16444)

[noreply] [BEAM-13574] Filesystem abstraction Rename support (#16428)

[noreply] [BEAM-13597] Setup Go in github actions (#16446)

[noreply] Merge pull request #16161 from [BEAM-12164] Add Spanner Partition

[noreply] Merge pull request #16203 from [BEAM-12164] Add Spanner Change Stream


------------------------------------------
[...truncated 72.66 KB...]
SEVERE: 2022-01-07T15:31:25.287Z: Staged package avro-1.8.2-91SggwzmelqfpnpU7BXRA-8V4chQ17Jvr3tkfu3cgtM.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/avro-1.8.2-91SggwzmelqfpnpU7BXRA-8V4chQ17Jvr3tkfu3cgtM.jar' is inaccessible.
Jan 07, 2022 3:31:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-07T15:31:25.952Z: Staged package beam-vendor-bytebuddy-1_11_0-0.1-SneHjzQgCpxxLkvyN8dugqO_sTzwXJ2995kqFu4l2i4.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-bytebuddy-1_11_0-0.1-SneHjzQgCpxxLkvyN8dugqO_sTzwXJ2995kqFu4l2i4.jar' is inaccessible.
Jan 07, 2022 3:31:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-07T15:31:26.008Z: Staged package beam-vendor-guava-26_0-jre-0.1-8quizfXTrmcvJyOXi3zWDFtP5EofgqjksY5sIbsBqws.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-guava-26_0-jre-0.1-8quizfXTrmcvJyOXi3zWDFtP5EofgqjksY5sIbsBqws.jar' is inaccessible.
Jan 07, 2022 3:31:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-07T15:31:26.115Z: Staged package checker-compat-qual-2.5.5-EdE0skXpysxHRRTS1mtbhhj4A5oUZc3FW7wLNOAAi3o.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/checker-compat-qual-2.5.5-EdE0skXpysxHRRTS1mtbhhj4A5oUZc3FW7wLNOAAi3o.jar' is inaccessible.
Jan 07, 2022 3:31:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-07T15:31:26.289Z: Staged package commons-lang3-3.9-3i4dzc8--ReozoWGYaBnJqmpRPKOM61_ngi-pE3DwjA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/commons-lang3-3.9-3i4dzc8--ReozoWGYaBnJqmpRPKOM61_ngi-pE3DwjA.jar' is inaccessible.
Jan 07, 2022 3:31:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-07T15:31:26.329Z: Staged package commons-logging-1.2-2t3qHqC-D1aXirMAa4rJKDSv7vvZt-TmMW_KV98PpjY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/commons-logging-1.2-2t3qHqC-D1aXirMAa4rJKDSv7vvZt-TmMW_KV98PpjY.jar' is inaccessible.
Jan 07, 2022 3:31:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-07T15:31:26.372Z: Staged package commons-math3-3.6.1-HlbXsFjSi2Wr0la4RY44hbZ0wdWI-kPNfRy7nH7yswg.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/commons-math3-3.6.1-HlbXsFjSi2Wr0la4RY44hbZ0wdWI-kPNfRy7nH7yswg.jar' is inaccessible.
Jan 07, 2022 3:31:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-07T15:31:27.553Z: Staged package hamcrest-2.1-upOy46ViMiukMvChtTrdzFXLGIJTMZoCDtd_gk5pIFA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/hamcrest-2.1-upOy46ViMiukMvChtTrdzFXLGIJTMZoCDtd_gk5pIFA.jar' is inaccessible.
Jan 07, 2022 3:31:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-07T15:31:27.589Z: Staged package hamcrest-core-2.1-4JEJ5UoonYhQa5v-yYfd0Zn0IXyUZBMmaDUbmk8Avuk.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/hamcrest-core-2.1-4JEJ5UoonYhQa5v-yYfd0Zn0IXyUZBMmaDUbmk8Avuk.jar' is inaccessible.
Jan 07, 2022 3:31:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-07T15:31:27.718Z: Staged package j2objc-annotations-1.3-Ia8wySJnvWEiwOC00gzMtmQaN-r5VsZUDsRx1YTmSns.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/j2objc-annotations-1.3-Ia8wySJnvWEiwOC00gzMtmQaN-r5VsZUDsRx1YTmSns.jar' is inaccessible.
Jan 07, 2022 3:31:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-07T15:31:27.817Z: Staged package jackson-core-asl-1.9.13-RAqctcqVshX5U9OiCmsaENofCbUpqd3qX4pJBd2rT1o.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/jackson-core-asl-1.9.13-RAqctcqVshX5U9OiCmsaENofCbUpqd3qX4pJBd2rT1o.jar' is inaccessible.
Jan 07, 2022 3:31:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-07T15:31:27.960Z: Staged package jackson-mapper-asl-1.9.13-dOegenby7breKTEqWi68z6AZEovAIezjhW12GX6b4MI.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/jackson-mapper-asl-1.9.13-dOegenby7breKTEqWi68z6AZEovAIezjhW12GX6b4MI.jar' is inaccessible.
Jan 07, 2022 3:31:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-07T15:31:28.032Z: Staged package javax.annotation-api-1.3.2-4EulGVvNVV3JVlD3zGFNFR5LzVLSmhC4qiGX86uJq5s.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/javax.annotation-api-1.3.2-4EulGVvNVV3JVlD3zGFNFR5LzVLSmhC4qiGX86uJq5s.jar' is inaccessible.
Jan 07, 2022 3:31:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-07T15:31:28.103Z: Staged package json-20200518-D_MObagvFa-vDna9bplqerVWaFEWaoP5c8ZKmg-UbS0.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/json-20200518-D_MObagvFa-vDna9bplqerVWaFEWaoP5c8ZKmg-UbS0.jar' is inaccessible.
Jan 07, 2022 3:31:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-07T15:31:28.146Z: Staged package jsr305-3.0.2-dmrSoHg_JoeWLIrXTO7MOKKLn3Ki0IXuQ4t4E-ko0Mc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/jsr305-3.0.2-dmrSoHg_JoeWLIrXTO7MOKKLn3Ki0IXuQ4t4E-ko0Mc.jar' is inaccessible.
Jan 07, 2022 3:31:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2022-01-07T15:31:29.115Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jan 07, 2022 3:34:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2022-01-07T15:34:28.791Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jan 07, 2022 3:37:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-07T15:37:25.292Z: Staged package annotations-4.1.1.4-unNOHoTAnWFa9qCdMwNLTwRC-Hct7BIO-zdthqVlrhU.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/annotations-4.1.1.4-unNOHoTAnWFa9qCdMwNLTwRC-Hct7BIO-zdthqVlrhU.jar' is inaccessible.
Jan 07, 2022 3:37:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-07T15:37:25.457Z: Staged package avro-1.8.2-91SggwzmelqfpnpU7BXRA-8V4chQ17Jvr3tkfu3cgtM.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/avro-1.8.2-91SggwzmelqfpnpU7BXRA-8V4chQ17Jvr3tkfu3cgtM.jar' is inaccessible.
Jan 07, 2022 3:37:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-07T15:37:26.419Z: Staged package beam-vendor-bytebuddy-1_11_0-0.1-SneHjzQgCpxxLkvyN8dugqO_sTzwXJ2995kqFu4l2i4.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-bytebuddy-1_11_0-0.1-SneHjzQgCpxxLkvyN8dugqO_sTzwXJ2995kqFu4l2i4.jar' is inaccessible.
Jan 07, 2022 3:37:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-07T15:37:26.481Z: Staged package beam-vendor-guava-26_0-jre-0.1-8quizfXTrmcvJyOXi3zWDFtP5EofgqjksY5sIbsBqws.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-guava-26_0-jre-0.1-8quizfXTrmcvJyOXi3zWDFtP5EofgqjksY5sIbsBqws.jar' is inaccessible.
Jan 07, 2022 3:37:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-07T15:37:26.570Z: Staged package checker-compat-qual-2.5.5-EdE0skXpysxHRRTS1mtbhhj4A5oUZc3FW7wLNOAAi3o.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/checker-compat-qual-2.5.5-EdE0skXpysxHRRTS1mtbhhj4A5oUZc3FW7wLNOAAi3o.jar' is inaccessible.
Jan 07, 2022 3:37:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-07T15:37:27.156Z: Staged package commons-lang3-3.9-3i4dzc8--ReozoWGYaBnJqmpRPKOM61_ngi-pE3DwjA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/commons-lang3-3.9-3i4dzc8--ReozoWGYaBnJqmpRPKOM61_ngi-pE3DwjA.jar' is inaccessible.
Jan 07, 2022 3:37:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-07T15:37:27.201Z: Staged package commons-logging-1.2-2t3qHqC-D1aXirMAa4rJKDSv7vvZt-TmMW_KV98PpjY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/commons-logging-1.2-2t3qHqC-D1aXirMAa4rJKDSv7vvZt-TmMW_KV98PpjY.jar' is inaccessible.
Jan 07, 2022 3:37:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-07T15:37:27.268Z: Staged package commons-math3-3.6.1-HlbXsFjSi2Wr0la4RY44hbZ0wdWI-kPNfRy7nH7yswg.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/commons-math3-3.6.1-HlbXsFjSi2Wr0la4RY44hbZ0wdWI-kPNfRy7nH7yswg.jar' is inaccessible.
Jan 07, 2022 3:37:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-07T15:37:28.755Z: Staged package hamcrest-2.1-upOy46ViMiukMvChtTrdzFXLGIJTMZoCDtd_gk5pIFA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/hamcrest-2.1-upOy46ViMiukMvChtTrdzFXLGIJTMZoCDtd_gk5pIFA.jar' is inaccessible.
Jan 07, 2022 3:37:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-07T15:37:28.820Z: Staged package hamcrest-core-2.1-4JEJ5UoonYhQa5v-yYfd0Zn0IXyUZBMmaDUbmk8Avuk.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/hamcrest-core-2.1-4JEJ5UoonYhQa5v-yYfd0Zn0IXyUZBMmaDUbmk8Avuk.jar' is inaccessible.
Jan 07, 2022 3:37:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-07T15:37:28.916Z: Staged package j2objc-annotations-1.3-Ia8wySJnvWEiwOC00gzMtmQaN-r5VsZUDsRx1YTmSns.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/j2objc-annotations-1.3-Ia8wySJnvWEiwOC00gzMtmQaN-r5VsZUDsRx1YTmSns.jar' is inaccessible.
Jan 07, 2022 3:37:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-07T15:37:29.016Z: Staged package jackson-core-asl-1.9.13-RAqctcqVshX5U9OiCmsaENofCbUpqd3qX4pJBd2rT1o.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/jackson-core-asl-1.9.13-RAqctcqVshX5U9OiCmsaENofCbUpqd3qX4pJBd2rT1o.jar' is inaccessible.
Jan 07, 2022 3:37:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-07T15:37:29.360Z: Staged package jackson-mapper-asl-1.9.13-dOegenby7breKTEqWi68z6AZEovAIezjhW12GX6b4MI.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/jackson-mapper-asl-1.9.13-dOegenby7breKTEqWi68z6AZEovAIezjhW12GX6b4MI.jar' is inaccessible.
Jan 07, 2022 3:37:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-07T15:37:29.609Z: Staged package javax.annotation-api-1.3.2-4EulGVvNVV3JVlD3zGFNFR5LzVLSmhC4qiGX86uJq5s.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/javax.annotation-api-1.3.2-4EulGVvNVV3JVlD3zGFNFR5LzVLSmhC4qiGX86uJq5s.jar' is inaccessible.
Jan 07, 2022 3:37:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-07T15:37:29.854Z: Staged package json-20200518-D_MObagvFa-vDna9bplqerVWaFEWaoP5c8ZKmg-UbS0.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/json-20200518-D_MObagvFa-vDna9bplqerVWaFEWaoP5c8ZKmg-UbS0.jar' is inaccessible.
Jan 07, 2022 3:37:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-07T15:37:29.978Z: Staged package jsr305-3.0.2-dmrSoHg_JoeWLIrXTO7MOKKLn3Ki0IXuQ4t4E-ko0Mc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/jsr305-3.0.2-dmrSoHg_JoeWLIrXTO7MOKKLn3Ki0IXuQ4t4E-ko0Mc.jar' is inaccessible.
Jan 07, 2022 3:37:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2022-01-07T15:37:32.581Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jan 07, 2022 3:40:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2022-01-07T15:40:29.293Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jan 07, 2022 3:43:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-07T15:43:25.127Z: Staged package annotations-4.1.1.4-unNOHoTAnWFa9qCdMwNLTwRC-Hct7BIO-zdthqVlrhU.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/annotations-4.1.1.4-unNOHoTAnWFa9qCdMwNLTwRC-Hct7BIO-zdthqVlrhU.jar' is inaccessible.
Jan 07, 2022 3:43:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-07T15:43:25.238Z: Staged package avro-1.8.2-91SggwzmelqfpnpU7BXRA-8V4chQ17Jvr3tkfu3cgtM.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/avro-1.8.2-91SggwzmelqfpnpU7BXRA-8V4chQ17Jvr3tkfu3cgtM.jar' is inaccessible.
Jan 07, 2022 3:43:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-07T15:43:25.711Z: Staged package beam-vendor-bytebuddy-1_11_0-0.1-SneHjzQgCpxxLkvyN8dugqO_sTzwXJ2995kqFu4l2i4.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-bytebuddy-1_11_0-0.1-SneHjzQgCpxxLkvyN8dugqO_sTzwXJ2995kqFu4l2i4.jar' is inaccessible.
Jan 07, 2022 3:43:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-07T15:43:25.760Z: Staged package beam-vendor-guava-26_0-jre-0.1-8quizfXTrmcvJyOXi3zWDFtP5EofgqjksY5sIbsBqws.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-guava-26_0-jre-0.1-8quizfXTrmcvJyOXi3zWDFtP5EofgqjksY5sIbsBqws.jar' is inaccessible.
Jan 07, 2022 3:43:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-07T15:43:25.817Z: Staged package checker-compat-qual-2.5.5-EdE0skXpysxHRRTS1mtbhhj4A5oUZc3FW7wLNOAAi3o.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/checker-compat-qual-2.5.5-EdE0skXpysxHRRTS1mtbhhj4A5oUZc3FW7wLNOAAi3o.jar' is inaccessible.
Jan 07, 2022 3:43:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-07T15:43:25.960Z: Staged package commons-lang3-3.9-3i4dzc8--ReozoWGYaBnJqmpRPKOM61_ngi-pE3DwjA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/commons-lang3-3.9-3i4dzc8--ReozoWGYaBnJqmpRPKOM61_ngi-pE3DwjA.jar' is inaccessible.
Jan 07, 2022 3:43:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-07T15:43:26.008Z: Staged package commons-logging-1.2-2t3qHqC-D1aXirMAa4rJKDSv7vvZt-TmMW_KV98PpjY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/commons-logging-1.2-2t3qHqC-D1aXirMAa4rJKDSv7vvZt-TmMW_KV98PpjY.jar' is inaccessible.
Jan 07, 2022 3:43:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-07T15:43:26.077Z: Staged package commons-math3-3.6.1-HlbXsFjSi2Wr0la4RY44hbZ0wdWI-kPNfRy7nH7yswg.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/commons-math3-3.6.1-HlbXsFjSi2Wr0la4RY44hbZ0wdWI-kPNfRy7nH7yswg.jar' is inaccessible.
Jan 07, 2022 3:43:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-07T15:43:27.241Z: Staged package hamcrest-2.1-upOy46ViMiukMvChtTrdzFXLGIJTMZoCDtd_gk5pIFA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/hamcrest-2.1-upOy46ViMiukMvChtTrdzFXLGIJTMZoCDtd_gk5pIFA.jar' is inaccessible.
Jan 07, 2022 3:43:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-07T15:43:27.286Z: Staged package hamcrest-core-2.1-4JEJ5UoonYhQa5v-yYfd0Zn0IXyUZBMmaDUbmk8Avuk.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/hamcrest-core-2.1-4JEJ5UoonYhQa5v-yYfd0Zn0IXyUZBMmaDUbmk8Avuk.jar' is inaccessible.
Jan 07, 2022 3:43:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-07T15:43:27.393Z: Staged package j2objc-annotations-1.3-Ia8wySJnvWEiwOC00gzMtmQaN-r5VsZUDsRx1YTmSns.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/j2objc-annotations-1.3-Ia8wySJnvWEiwOC00gzMtmQaN-r5VsZUDsRx1YTmSns.jar' is inaccessible.
Jan 07, 2022 3:43:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-07T15:43:27.462Z: Staged package jackson-core-asl-1.9.13-RAqctcqVshX5U9OiCmsaENofCbUpqd3qX4pJBd2rT1o.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/jackson-core-asl-1.9.13-RAqctcqVshX5U9OiCmsaENofCbUpqd3qX4pJBd2rT1o.jar' is inaccessible.
Jan 07, 2022 3:43:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-07T15:43:27.540Z: Staged package jackson-mapper-asl-1.9.13-dOegenby7breKTEqWi68z6AZEovAIezjhW12GX6b4MI.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/jackson-mapper-asl-1.9.13-dOegenby7breKTEqWi68z6AZEovAIezjhW12GX6b4MI.jar' is inaccessible.
Jan 07, 2022 3:43:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-07T15:43:27.597Z: Staged package javax.annotation-api-1.3.2-4EulGVvNVV3JVlD3zGFNFR5LzVLSmhC4qiGX86uJq5s.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/javax.annotation-api-1.3.2-4EulGVvNVV3JVlD3zGFNFR5LzVLSmhC4qiGX86uJq5s.jar' is inaccessible.
Jan 07, 2022 3:43:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-07T15:43:27.655Z: Staged package json-20200518-D_MObagvFa-vDna9bplqerVWaFEWaoP5c8ZKmg-UbS0.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/json-20200518-D_MObagvFa-vDna9bplqerVWaFEWaoP5c8ZKmg-UbS0.jar' is inaccessible.
Jan 07, 2022 3:43:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-07T15:43:27.713Z: Staged package jsr305-3.0.2-dmrSoHg_JoeWLIrXTO7MOKKLn3Ki0IXuQ4t4E-ko0Mc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/jsr305-3.0.2-dmrSoHg_JoeWLIrXTO7MOKKLn3Ki0IXuQ4t4E-ko0Mc.jar' is inaccessible.
Jan 07, 2022 3:43:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2022-01-07T15:43:28.675Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jan 07, 2022 3:46:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2022-01-07T15:46:29.751Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jan 07, 2022 3:49:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-07T15:49:25.297Z: Staged package annotations-4.1.1.4-unNOHoTAnWFa9qCdMwNLTwRC-Hct7BIO-zdthqVlrhU.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/annotations-4.1.1.4-unNOHoTAnWFa9qCdMwNLTwRC-Hct7BIO-zdthqVlrhU.jar' is inaccessible.
Jan 07, 2022 3:49:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-07T15:49:25.461Z: Staged package avro-1.8.2-91SggwzmelqfpnpU7BXRA-8V4chQ17Jvr3tkfu3cgtM.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/avro-1.8.2-91SggwzmelqfpnpU7BXRA-8V4chQ17Jvr3tkfu3cgtM.jar' is inaccessible.
Jan 07, 2022 3:49:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-07T15:49:26.100Z: Staged package beam-vendor-bytebuddy-1_11_0-0.1-SneHjzQgCpxxLkvyN8dugqO_sTzwXJ2995kqFu4l2i4.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-bytebuddy-1_11_0-0.1-SneHjzQgCpxxLkvyN8dugqO_sTzwXJ2995kqFu4l2i4.jar' is inaccessible.
Jan 07, 2022 3:49:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-07T15:49:26.201Z: Staged package beam-vendor-guava-26_0-jre-0.1-8quizfXTrmcvJyOXi3zWDFtP5EofgqjksY5sIbsBqws.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-guava-26_0-jre-0.1-8quizfXTrmcvJyOXi3zWDFtP5EofgqjksY5sIbsBqws.jar' is inaccessible.
Jan 07, 2022 3:49:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-07T15:49:26.277Z: Staged package checker-compat-qual-2.5.5-EdE0skXpysxHRRTS1mtbhhj4A5oUZc3FW7wLNOAAi3o.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/checker-compat-qual-2.5.5-EdE0skXpysxHRRTS1mtbhhj4A5oUZc3FW7wLNOAAi3o.jar' is inaccessible.
Jan 07, 2022 3:49:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-07T15:49:26.541Z: Staged package commons-lang3-3.9-3i4dzc8--ReozoWGYaBnJqmpRPKOM61_ngi-pE3DwjA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/commons-lang3-3.9-3i4dzc8--ReozoWGYaBnJqmpRPKOM61_ngi-pE3DwjA.jar' is inaccessible.
Jan 07, 2022 3:49:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-07T15:49:26.593Z: Staged package commons-logging-1.2-2t3qHqC-D1aXirMAa4rJKDSv7vvZt-TmMW_KV98PpjY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/commons-logging-1.2-2t3qHqC-D1aXirMAa4rJKDSv7vvZt-TmMW_KV98PpjY.jar' is inaccessible.
Jan 07, 2022 3:49:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-07T15:49:26.638Z: Staged package commons-math3-3.6.1-HlbXsFjSi2Wr0la4RY44hbZ0wdWI-kPNfRy7nH7yswg.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/commons-math3-3.6.1-HlbXsFjSi2Wr0la4RY44hbZ0wdWI-kPNfRy7nH7yswg.jar' is inaccessible.
Jan 07, 2022 3:49:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-07T15:49:28.080Z: Staged package hamcrest-2.1-upOy46ViMiukMvChtTrdzFXLGIJTMZoCDtd_gk5pIFA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/hamcrest-2.1-upOy46ViMiukMvChtTrdzFXLGIJTMZoCDtd_gk5pIFA.jar' is inaccessible.
Jan 07, 2022 3:49:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-07T15:49:28.117Z: Staged package hamcrest-core-2.1-4JEJ5UoonYhQa5v-yYfd0Zn0IXyUZBMmaDUbmk8Avuk.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/hamcrest-core-2.1-4JEJ5UoonYhQa5v-yYfd0Zn0IXyUZBMmaDUbmk8Avuk.jar' is inaccessible.
Jan 07, 2022 3:49:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-07T15:49:28.226Z: Staged package j2objc-annotations-1.3-Ia8wySJnvWEiwOC00gzMtmQaN-r5VsZUDsRx1YTmSns.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/j2objc-annotations-1.3-Ia8wySJnvWEiwOC00gzMtmQaN-r5VsZUDsRx1YTmSns.jar' is inaccessible.
Jan 07, 2022 3:49:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-07T15:49:28.316Z: Staged package jackson-core-asl-1.9.13-RAqctcqVshX5U9OiCmsaENofCbUpqd3qX4pJBd2rT1o.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/jackson-core-asl-1.9.13-RAqctcqVshX5U9OiCmsaENofCbUpqd3qX4pJBd2rT1o.jar' is inaccessible.
Jan 07, 2022 3:49:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-07T15:49:28.421Z: Staged package jackson-mapper-asl-1.9.13-dOegenby7breKTEqWi68z6AZEovAIezjhW12GX6b4MI.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/jackson-mapper-asl-1.9.13-dOegenby7breKTEqWi68z6AZEovAIezjhW12GX6b4MI.jar' is inaccessible.
Jan 07, 2022 3:49:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-07T15:49:28.498Z: Staged package javax.annotation-api-1.3.2-4EulGVvNVV3JVlD3zGFNFR5LzVLSmhC4qiGX86uJq5s.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/javax.annotation-api-1.3.2-4EulGVvNVV3JVlD3zGFNFR5LzVLSmhC4qiGX86uJq5s.jar' is inaccessible.
Jan 07, 2022 3:49:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-07T15:49:28.577Z: Staged package json-20200518-D_MObagvFa-vDna9bplqerVWaFEWaoP5c8ZKmg-UbS0.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/json-20200518-D_MObagvFa-vDna9bplqerVWaFEWaoP5c8ZKmg-UbS0.jar' is inaccessible.
Jan 07, 2022 3:49:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-07T15:49:28.623Z: Staged package jsr305-3.0.2-dmrSoHg_JoeWLIrXTO7MOKKLn3Ki0IXuQ4t4E-ko0Mc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/jsr305-3.0.2-dmrSoHg_JoeWLIrXTO7MOKKLn3Ki0IXuQ4t4E-ko0Mc.jar' is inaccessible.
Jan 07, 2022 3:49:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2022-01-07T15:49:29.579Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jan 07, 2022 3:52:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2022-01-07T15:52:28.668Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jan 07, 2022 3:55:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-07T15:55:25.178Z: Staged package annotations-4.1.1.4-unNOHoTAnWFa9qCdMwNLTwRC-Hct7BIO-zdthqVlrhU.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/annotations-4.1.1.4-unNOHoTAnWFa9qCdMwNLTwRC-Hct7BIO-zdthqVlrhU.jar' is inaccessible.
Jan 07, 2022 3:55:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-07T15:55:25.290Z: Staged package avro-1.8.2-91SggwzmelqfpnpU7BXRA-8V4chQ17Jvr3tkfu3cgtM.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/avro-1.8.2-91SggwzmelqfpnpU7BXRA-8V4chQ17Jvr3tkfu3cgtM.jar' is inaccessible.
Jan 07, 2022 3:55:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-07T15:55:25.852Z: Staged package beam-vendor-bytebuddy-1_11_0-0.1-SneHjzQgCpxxLkvyN8dugqO_sTzwXJ2995kqFu4l2i4.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-bytebuddy-1_11_0-0.1-SneHjzQgCpxxLkvyN8dugqO_sTzwXJ2995kqFu4l2i4.jar' is inaccessible.
Jan 07, 2022 3:55:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-07T15:55:25.942Z: Staged package beam-vendor-guava-26_0-jre-0.1-8quizfXTrmcvJyOXi3zWDFtP5EofgqjksY5sIbsBqws.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-guava-26_0-jre-0.1-8quizfXTrmcvJyOXi3zWDFtP5EofgqjksY5sIbsBqws.jar' is inaccessible.
Jan 07, 2022 3:55:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-07T15:55:26.003Z: Staged package checker-compat-qual-2.5.5-EdE0skXpysxHRRTS1mtbhhj4A5oUZc3FW7wLNOAAi3o.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/checker-compat-qual-2.5.5-EdE0skXpysxHRRTS1mtbhhj4A5oUZc3FW7wLNOAAi3o.jar' is inaccessible.
Jan 07, 2022 3:55:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-07T15:55:26.171Z: Staged package commons-lang3-3.9-3i4dzc8--ReozoWGYaBnJqmpRPKOM61_ngi-pE3DwjA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/commons-lang3-3.9-3i4dzc8--ReozoWGYaBnJqmpRPKOM61_ngi-pE3DwjA.jar' is inaccessible.
Jan 07, 2022 3:55:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-07T15:55:26.220Z: Staged package commons-logging-1.2-2t3qHqC-D1aXirMAa4rJKDSv7vvZt-TmMW_KV98PpjY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/commons-logging-1.2-2t3qHqC-D1aXirMAa4rJKDSv7vvZt-TmMW_KV98PpjY.jar' is inaccessible.
Jan 07, 2022 3:55:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-07T15:55:26.260Z: Staged package commons-math3-3.6.1-HlbXsFjSi2Wr0la4RY44hbZ0wdWI-kPNfRy7nH7yswg.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/commons-math3-3.6.1-HlbXsFjSi2Wr0la4RY44hbZ0wdWI-kPNfRy7nH7yswg.jar' is inaccessible.
Jan 07, 2022 3:55:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-07T15:55:27.424Z: Staged package hamcrest-2.1-upOy46ViMiukMvChtTrdzFXLGIJTMZoCDtd_gk5pIFA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/hamcrest-2.1-upOy46ViMiukMvChtTrdzFXLGIJTMZoCDtd_gk5pIFA.jar' is inaccessible.
Jan 07, 2022 3:55:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-07T15:55:27.463Z: Staged package hamcrest-core-2.1-4JEJ5UoonYhQa5v-yYfd0Zn0IXyUZBMmaDUbmk8Avuk.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/hamcrest-core-2.1-4JEJ5UoonYhQa5v-yYfd0Zn0IXyUZBMmaDUbmk8Avuk.jar' is inaccessible.
Jan 07, 2022 3:55:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-07T15:55:27.540Z: Staged package j2objc-annotations-1.3-Ia8wySJnvWEiwOC00gzMtmQaN-r5VsZUDsRx1YTmSns.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/j2objc-annotations-1.3-Ia8wySJnvWEiwOC00gzMtmQaN-r5VsZUDsRx1YTmSns.jar' is inaccessible.
Jan 07, 2022 3:55:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-07T15:55:27.624Z: Staged package jackson-core-asl-1.9.13-RAqctcqVshX5U9OiCmsaENofCbUpqd3qX4pJBd2rT1o.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/jackson-core-asl-1.9.13-RAqctcqVshX5U9OiCmsaENofCbUpqd3qX4pJBd2rT1o.jar' is inaccessible.
Jan 07, 2022 3:55:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-07T15:55:27.750Z: Staged package jackson-mapper-asl-1.9.13-dOegenby7breKTEqWi68z6AZEovAIezjhW12GX6b4MI.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/jackson-mapper-asl-1.9.13-dOegenby7breKTEqWi68z6AZEovAIezjhW12GX6b4MI.jar' is inaccessible.
Jan 07, 2022 3:55:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-07T15:55:27.822Z: Staged package javax.annotation-api-1.3.2-4EulGVvNVV3JVlD3zGFNFR5LzVLSmhC4qiGX86uJq5s.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/javax.annotation-api-1.3.2-4EulGVvNVV3JVlD3zGFNFR5LzVLSmhC4qiGX86uJq5s.jar' is inaccessible.
Jan 07, 2022 3:55:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-07T15:55:27.891Z: Staged package json-20200518-D_MObagvFa-vDna9bplqerVWaFEWaoP5c8ZKmg-UbS0.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/json-20200518-D_MObagvFa-vDna9bplqerVWaFEWaoP5c8ZKmg-UbS0.jar' is inaccessible.
Jan 07, 2022 3:55:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-07T15:55:27.930Z: Staged package jsr305-3.0.2-dmrSoHg_JoeWLIrXTO7MOKKLn3Ki0IXuQ4t4E-ko0Mc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/jsr305-3.0.2-dmrSoHg_JoeWLIrXTO7MOKKLn3Ki0IXuQ4t4E-ko0Mc.jar' is inaccessible.
Jan 07, 2022 3:55:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2022-01-07T15:55:28.825Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jan 07, 2022 3:58:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2022-01-07T15:58:28.051Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jan 07, 2022 4:00:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-07T16:00:38.587Z: Cancel request is committed for workflow job: 2022-01-07_04_46_10-1307797473656761051.
Jan 07, 2022 4:00:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-07T16:00:38.616Z: Cleaning up.
Jan 07, 2022 4:00:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-07T16:00:38.729Z: Stopping **** pool...
Jan 07, 2022 4:00:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-07T16:00:38.774Z: Stopping **** pool...
Jan 07, 2022 4:03:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-07T16:03:08.023Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Jan 07, 2022 4:03:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-07T16:03:08.059Z: Worker pool stopped.
Jan 07, 2022 4:03:16 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2022-01-07_04_46_10-1307797473656761051 finished with status CANCELLED.
Load test results for test (ID): bce6d2ba-68fd-497d-9bce-739be181233d and timestamp: 2022-01-07T12:46:04.841000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11518.205
dataflow_v2_java11_total_bytes_count             1.94104724E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220107124334
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:a821f866feb3e6e5eba153a7bc47c3f79432b832f6e971a4b93562ad0000e1ef
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220107124334]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:a821f866feb3e6e5eba153a7bc47c3f79432b832f6e971a4b93562ad0000e1ef]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220107124334] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:a821f866feb3e6e5eba153a7bc47c3f79432b832f6e971a4b93562ad0000e1ef])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:a821f866feb3e6e5eba153a7bc47c3f79432b832f6e971a4b93562ad0000e1ef
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:a821f866feb3e6e5eba153a7bc47c3f79432b832f6e971a4b93562ad0000e1ef
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:a821f866feb3e6e5eba153a7bc47c3f79432b832f6e971a4b93562ad0000e1ef].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:37b32517fc226bf927301945f30602de44aa7b6ba8fef662169084699aaa409d
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:37b32517fc226bf927301945f30602de44aa7b6ba8fef662169084699aaa409d
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:37b32517fc226bf927301945f30602de44aa7b6ba8fef662169084699aaa409d].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings

Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.

BUILD FAILED in 3h 20m
109 actionable tasks: 73 executed, 32 from cache, 4 up-to-date

Publishing build scan...
https://gradle.com/s/h57a7gdzssfa4

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure


---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #203

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/203/display/redirect?page=changes>

Changes:

[aydar.zaynutdinov] [BEAM-13565][Playground]

[relax] update bom

[zyichi] Upgrade python library versions in base_image_requirements.txt

[noreply] [BEAM-13567] Consolidate runner flag definition. (#16426)

[noreply] [BEAM-13601] Don't cache Row types for a schema. (#16427)

[noreply] [BEAM-13430] Re-enable checkerframework (#16429)

[noreply] [BEAM-13430] Ensure that testRuntimeMigration depends on "default"

[noreply] Merge pull request #16277 from [BEAM-13124][Playground] Create readiness

[noreply] Merge pull request #16314 from [BEAM-13260][Playground]Implement setup

[noreply] Merge pull request #16383 from [BEAM-13566][Playground] Add logic of

[noreply] Merge pull request #16365 from [BEAM-13559][Playground] Remove tag in

[noreply] Merge pull request #16360 from [BEAM-13546][Playground] Update nginx

[noreply] Merge pull request #16192 from [BEAM-13395] [Playground] Tag katas

[noreply] Merge pull request #16254 from [BEAM-13249][Playground] Security – Mock

[ningkang0957] [BEAM-12879] Prevented missing permission from failing GCS I/O

[noreply] Merge pull request #16347: fix: move connector to use v1 BigQuery

[noreply] [BEAM-13603] Fix bug in apache_beam.utils.Shared (#16437)

[noreply] [BEAM-10345] Add an import guard to support recent google-cloud-spanner


------------------------------------------
[...truncated 51.85 KB...]
4c5e589408a3: Pushed
624025d2ed82: Pushed
4afc3431b8f8: Pushed
43cd40da8f13: Pushed
7d7a8ba090e9: Pushed
b566a0eabf87: Pushed
3bb5258f46d2: Layer already exists
832e177bb500: Layer already exists
26a504e63be4: Layer already exists
f9e18e59a565: Layer already exists
8bf42db0de72: Layer already exists
31892cc314cb: Layer already exists
11936051f93b: Layer already exists
79ddb0a130ec: Pushed
1924bfd0f3af: Pushed
20220106124353: digest: sha256:7f52ed10080141871f8f1b9dadc6532c506db584bffa32184de9a3797118e358 size: 4520

> Task :sdks:java:testing:load-tests:run
Jan 06, 2022 12:46:22 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Jan 06, 2022 12:46:22 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 203 files. Enable logging at DEBUG level to see which files will be staged.
Jan 06, 2022 12:46:23 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Jan 06, 2022 12:46:23 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Jan 06, 2022 12:46:26 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 203 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Jan 06, 2022 12:46:27 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 203 files cached, 0 files newly uploaded in 0 seconds
Jan 06, 2022 12:46:27 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Jan 06, 2022 12:46:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <113691 bytes, hash 6301f6c8c50de146b3a0c72d195b67f15dfa9917b926c0f5639a261f70315053> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-YwH2yMUN4UazoMctGVtn8V36mRe5JsD1Y5omH3AxUFM.pb
Jan 06, 2022 12:46:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Jan 06, 2022 12:46:29 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7f5b9db, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@507d64aa, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@37045b48, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@60b34931, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4aa21f9d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@71c17a57, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@640ab13c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1e0a864d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@440e3ce6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3e67f5f2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@fd53053, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4527f70a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@707b1a44, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7132a9dc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@57435801, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2da66a44, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@527fc8e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@61bfc9bf, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2c7106d9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@329bad59]
Jan 06, 2022 12:46:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Jan 06, 2022 12:46:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Jan 06, 2022 12:46:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Jan 06, 2022 12:46:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Jan 06, 2022 12:46:29 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5854a18, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@d5556bf, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@791c12e3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@b112b13, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@24eb65e3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ac3f6f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1abebef3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@18f55704, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@67cefd84, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5fbe155, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6add8e3f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58a2b917, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@48904d5a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12bbfc54, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1491344a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@59b65dce, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1386313f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5e922647, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@433c6abb, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@288f173f]
Jan 06, 2022 12:46:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Jan 06, 2022 12:46:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Jan 06, 2022 12:46:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Jan 06, 2022 12:46:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Jan 06, 2022 12:46:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Jan 06, 2022 12:46:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Jan 06, 2022 12:46:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Jan 06, 2022 12:46:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Jan 06, 2022 12:46:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Jan 06, 2022 12:46:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Jan 06, 2022 12:46:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Jan 06, 2022 12:46:29 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.37.0-SNAPSHOT
Jan 06, 2022 12:46:30 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-01-06_04_46_29-15093586055417001318?project=apache-beam-testing
Jan 06, 2022 12:46:30 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2022-01-06_04_46_29-15093586055417001318
Jan 06, 2022 12:46:30 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2022-01-06_04_46_29-15093586055417001318
Jan 06, 2022 12:46:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2022-01-06T12:46:35.741Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-01-gj7z. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Jan 06, 2022 12:46:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-06T12:46:40.836Z: Worker configuration: e2-standard-2 in us-central1-b.
Jan 06, 2022 12:46:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-06T12:46:41.587Z: Expanding SplittableParDo operations into optimizable parts.
Jan 06, 2022 12:46:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-06T12:46:41.623Z: Expanding CollectionToSingleton operations into optimizable parts.
Jan 06, 2022 12:46:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-06T12:46:41.691Z: Expanding CoGroupByKey operations into optimizable parts.
Jan 06, 2022 12:46:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-06T12:46:41.759Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Jan 06, 2022 12:46:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-06T12:46:41.790Z: Expanding GroupByKey operations into streaming Read/Write steps
Jan 06, 2022 12:46:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-06T12:46:41.848Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jan 06, 2022 12:46:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-06T12:46:41.943Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jan 06, 2022 12:46:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-06T12:46:41.963Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Jan 06, 2022 12:46:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-06T12:46:41.991Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Jan 06, 2022 12:46:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-06T12:46:42.026Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Jan 06, 2022 12:46:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-06T12:46:42.055Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Jan 06, 2022 12:46:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-06T12:46:42.092Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jan 06, 2022 12:46:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-06T12:46:42.125Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jan 06, 2022 12:46:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-06T12:46:42.160Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jan 06, 2022 12:46:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-06T12:46:42.189Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Jan 06, 2022 12:46:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-06T12:46:42.228Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Jan 06, 2022 12:46:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-06T12:46:42.254Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Jan 06, 2022 12:46:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-06T12:46:42.293Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Jan 06, 2022 12:46:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-06T12:46:42.321Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jan 06, 2022 12:46:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-06T12:46:42.357Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jan 06, 2022 12:46:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-06T12:46:42.391Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jan 06, 2022 12:46:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-06T12:46:42.428Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Jan 06, 2022 12:46:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-06T12:46:42.457Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Jan 06, 2022 12:46:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-06T12:46:42.489Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Jan 06, 2022 12:46:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-06T12:46:42.517Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Jan 06, 2022 12:46:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-06T12:46:42.540Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Jan 06, 2022 12:46:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-06T12:46:42.576Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Jan 06, 2022 12:46:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-06T12:46:42.600Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Jan 06, 2022 12:46:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-06T12:46:42.623Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Jan 06, 2022 12:46:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-06T12:46:42.978Z: Starting 5 ****s in us-central1-b...
Jan 06, 2022 12:47:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-06T12:47:14.485Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jan 06, 2022 12:47:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-06T12:47:27.332Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Jan 06, 2022 12:48:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-06T12:48:25.062Z: Workers have started successfully.
Jan 06, 2022 12:48:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-06T12:48:25.084Z: Workers have started successfully.
Jan 06, 2022 4:00:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-06T16:00:37.181Z: Cancel request is committed for workflow job: 2022-01-06_04_46_29-15093586055417001318.
Jan 06, 2022 4:00:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-06T16:00:37.255Z: Cleaning up.
Jan 06, 2022 4:00:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-06T16:00:37.346Z: Stopping **** pool...
Jan 06, 2022 4:00:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-06T16:00:37.388Z: Stopping **** pool...
Jan 06, 2022 4:03:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-06T16:03:11.288Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Jan 06, 2022 4:03:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-06T16:03:11.326Z: Worker pool stopped.
Jan 06, 2022 4:03:16 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2022-01-06_04_46_29-15093586055417001318 finished with status CANCELLED.
Load test results for test (ID): 467bee2d-801a-4407-af20-dc76119bc6fc and timestamp: 2022-01-06T12:46:23.194000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11480.679
dataflow_v2_java11_total_bytes_count             2.30969047E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220106124353
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:7f52ed10080141871f8f1b9dadc6532c506db584bffa32184de9a3797118e358
Deleted: sha256:4f0c8dd62010c7b740a495c7a45f82b0ef2322fd895d07567835fce856aa6183
Deleted: sha256:ac027c8ca90cb6281d43f5ed50bf44ef763f1782c8fdc8fcdb558b01191695a7
Deleted: sha256:0e5d4f704fa527c144ea31b7b44b97c9ad8184c999e3ee044866c3536d5f0127
Deleted: sha256:1fa1782c4f13b990085c19d17183b4f621db092b29bc60a7fcd7c25abb7794fa
Deleted: sha256:530a6f1aceeab2304bd089494782de85462cd25e8d602fb719354106c80c103d
Deleted: sha256:cddbd957eff53019d5c937092dfbec47b8a7be7428c24e0ca8305f99ee5e8f86
Deleted: sha256:fd4030329332b6e389dce3d8c019f63cef53b5bd2b1b104849eb0a1167e35544
Deleted: sha256:5ee896ac8a97283acd5634f6167da6e021edbaa12e1581038fb3f8642e25a854
Deleted: sha256:511a69fed95c551d40b04579be5f7325ecea875774bc596885dc771aedcdbff2
Deleted: sha256:bb71266021c9c917764351ba822df3000eedc1225c0505e9f3a6fc28d2941463
Deleted: sha256:b5c3a61b873a77324676327948339ef48a810bae8f77ece828c6d51de5514ca1
Deleted: sha256:ff910c9fb6c8ce2f5ede334b4b5fd78dfb6772c85dcb52bbb90dbd7a1fdd2ca1
Deleted: sha256:ddecaf6aa8de6c035efeaf30adffdc3950f1be71579b205914695531f6f2894e
Deleted: sha256:292f8c4dc0defbc6b0bac2d2db94463e656043c753401fc051ea5a88ebea8b93
Deleted: sha256:51a20a8377bdaf0cb7b3893e079d88b024f495dd2516cf1811677c757637637a
Deleted: sha256:77376927f2061a6532a530950d08c52798abe1d41cefda0adf51bce95e5731e6
Deleted: sha256:524d0d40f838f2bd23bbe0db116df95f8674769ba895c480e4aa6bde63c8ea52
Deleted: sha256:57640528df0d00e646a0f691e310f897c95901080ef19506c6dd54b905e26c4f
Deleted: sha256:9144ac81ac9609827cd864d16535a97de0e57ccce7ab9c63650865f56f18b77d
Deleted: sha256:0e08e3df55736c3f54c05f4496e325c9c5304e92b8ce36993c54c958e250b63d
Deleted: sha256:61c7cf51ce09490d95f2a89b85e8dff99ebe80f2218c88f664878b4e85c3becb
Deleted: sha256:30f67045b612b9d892e1f0a3658160535fdda8afe338d7d9a86e1f991efd8567
Deleted: sha256:ac7a5c607a07ae3785929b60cf2576495fdbe93e71c46ab2b09befb51087239c
Deleted: sha256:ef049b5a0c9532dd72f54d95d17b111df6ed2eb2c06c8d7e5a54e1ca70a85df4
Deleted: sha256:f1ccd0a8861b3e7d39a944b894045dd6d03121329f339512b2caf0f8d5f4f057
Deleted: sha256:668ac4accb8e4840fcefc537ad2658324a0bd8f58a99d5b7595a2093b77f038f
Deleted: sha256:d7b4193782ff7eccf75676e67ddafa094a1f7cb86b3eab5257c4be45ce772784
Deleted: sha256:23b0a8d2a04f9a6cf29004c79b21aa16cafcf5dfb0696a4fc00bec6a1039582d
Deleted: sha256:11a66b97b7b8e6c3fee30d3b5db72136ce85a4192ef9946790882b079e7bfd73
Deleted: sha256:b9d3614407d0ef9a58f1c8fe787cbb0a04bc4eb97789c128d99d48a627eeb024
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220106124353]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:7f52ed10080141871f8f1b9dadc6532c506db584bffa32184de9a3797118e358]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220106124353] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:7f52ed10080141871f8f1b9dadc6532c506db584bffa32184de9a3797118e358])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:7f52ed10080141871f8f1b9dadc6532c506db584bffa32184de9a3797118e358
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:7f52ed10080141871f8f1b9dadc6532c506db584bffa32184de9a3797118e358
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:7f52ed10080141871f8f1b9dadc6532c506db584bffa32184de9a3797118e358].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings

Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.

BUILD FAILED in 3h 19m 51s
109 actionable tasks: 75 executed, 30 from cache, 4 up-to-date

Publishing build scan...
https://gradle.com/s/74xt2rkwmsfeu

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #202

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/202/display/redirect?page=changes>

Changes:

[johnjcasey] [BEAM-12391] update avro sink to close the opened file handle, instead

[Robert Bradshaw] [BEAM-13482] Python fully qualified name external transforms.

[Robert Bradshaw] Add Python expansion service entry point.

[Kyle Weaver] [BEAM-13569] Change Spark dependencies to implementation.

[Kyle Weaver] remove redundant dependency

[wuren] [BEAM-13591] Bump log4j2 version to 2.17.1

[Steve Niemitz] [BEAM-13459] Update CHANGES.md, add note about artifact caching for

[relax] Add Flink runner support for OrderedListState. This version reads the

[noreply] Merge pull request #16404: [BEAM-13586] Fix NPE in

[zyichi] Fix sdk_container_builder too many values to unpack error

[noreply] [BEAM-13480] Sickbay PubSubIntegrationTest.test_streaming_data_only on

[Kyle Weaver] remove redundant testImplementation dependencies

[noreply] [BEAM-13430] Swap to use "mainClass" instead of "main" since it was

[zyichi] Fix remaining failing perf IT tests.

[noreply] [BEAM-13430] Replace deprecated "appendix" with "archiveAppendix"

[noreply] [BEAM-13015] Add jamm as a java agent to the Java SDK harness container

[noreply] [BEAM-13430] Partially revert

[noreply] Merge pull request #15863 from [BEAM-13184] Autosharding for

[noreply] [BEAM-11936] Enable FloatingPointAssertionWithinEpsilon errorprone check

[noreply] [BEAM-11936] Enable LockNotBeforeTry errorprone check (#16259)

[noreply] [BEAM-11936] Enable errorprone unused checks (#16262)

[noreply] Add Nexmark Query 14 (#16337)

[noreply] [BEAM-13015] Migrate all user state and side implementations to support

[noreply] [BEAM-13015] Use 20% of memory when the maximum has been configured.


------------------------------------------
[...truncated 44.98 KB...]
a38a8df97f6a: Pushed
c440b437d45a: Pushed
57d5327a42da: Pushed
bc62d2a7ce2e: Pushed
7cadca06c395: Pushed
3bb5258f46d2: Layer already exists
832e177bb500: Layer already exists
f9e18e59a565: Layer already exists
c0662f28aced: Pushed
8bf42db0de72: Layer already exists
26a504e63be4: Layer already exists
31892cc314cb: Layer already exists
11936051f93b: Layer already exists
b2578895a203: Pushed
2814bc0217ea: Pushed
20220105124332: digest: sha256:7137665db55d01d50172fb0d4768f2e6cecb79cbdb098517f0d09d97ed136594 size: 4520

> Task :sdks:java:testing:load-tests:run
Jan 05, 2022 12:45:15 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Jan 05, 2022 12:45:16 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 203 files. Enable logging at DEBUG level to see which files will be staged.
Jan 05, 2022 12:45:17 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Jan 05, 2022 12:45:17 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Jan 05, 2022 12:45:19 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 203 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Jan 05, 2022 12:45:19 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 203 files cached, 0 files newly uploaded in 0 seconds
Jan 05, 2022 12:45:19 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Jan 05, 2022 12:45:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <113691 bytes, hash a934baf319f10e53779cc9bfdcdccc462b838ae8f63b3e6abe0b516f090b9865> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-qTS68xnxDlN3nMm_3NzMRiuDiuj2Oz5qvgtRbwkLmGU.pb
Jan 05, 2022 12:45:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Jan 05, 2022 12:45:21 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@48a0c8aa, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6192a5d5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3722c145, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4cbc2e3b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2975a9e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@765ffb14, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@57562473, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7a360554, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@424de326, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4bc33720, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2dd0f797, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@67064bdc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4a7fd0c9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@18578491, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3291b443, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@671c4166, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@53d13cd4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77865933, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@480ad82c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4d18b73a]
Jan 05, 2022 12:45:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Jan 05, 2022 12:45:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Jan 05, 2022 12:45:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Jan 05, 2022 12:45:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Jan 05, 2022 12:45:21 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@553bc36c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@380e1909, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2d5ef498, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@95eb320, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2f521c4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4afbb6c2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@10db6131, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6c6017b9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4730e0f0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@506a1372, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1b7332a7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77c233af, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@37b56ac7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@37c41ec0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@35a0e495, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5399f6c5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4fe64d23, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58437801, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6af5bbd0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@76464795]
Jan 05, 2022 12:45:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Jan 05, 2022 12:45:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Jan 05, 2022 12:45:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Jan 05, 2022 12:45:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Jan 05, 2022 12:45:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Jan 05, 2022 12:45:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Jan 05, 2022 12:45:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Jan 05, 2022 12:45:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Jan 05, 2022 12:45:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Jan 05, 2022 12:45:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Jan 05, 2022 12:45:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Jan 05, 2022 12:45:21 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.37.0-SNAPSHOT
Jan 05, 2022 12:45:22 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-01-05_04_45_21-6676539346797392784?project=apache-beam-testing
Jan 05, 2022 12:45:22 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2022-01-05_04_45_21-6676539346797392784
Jan 05, 2022 12:45:22 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2022-01-05_04_45_21-6676539346797392784
Jan 05, 2022 12:45:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2022-01-05T12:45:28.967Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-01-z6cy. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Jan 05, 2022 12:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-05T12:45:32.578Z: Worker configuration: e2-standard-2 in us-central1-f.
Jan 05, 2022 12:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-05T12:45:33.056Z: Expanding SplittableParDo operations into optimizable parts.
Jan 05, 2022 12:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-05T12:45:33.093Z: Expanding CollectionToSingleton operations into optimizable parts.
Jan 05, 2022 12:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-05T12:45:33.162Z: Expanding CoGroupByKey operations into optimizable parts.
Jan 05, 2022 12:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-05T12:45:33.222Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Jan 05, 2022 12:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-05T12:45:33.247Z: Expanding GroupByKey operations into streaming Read/Write steps
Jan 05, 2022 12:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-05T12:45:33.293Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jan 05, 2022 12:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-05T12:45:33.390Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jan 05, 2022 12:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-05T12:45:33.422Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Jan 05, 2022 12:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-05T12:45:33.450Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Jan 05, 2022 12:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-05T12:45:33.487Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Jan 05, 2022 12:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-05T12:45:33.513Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Jan 05, 2022 12:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-05T12:45:33.538Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jan 05, 2022 12:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-05T12:45:33.565Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jan 05, 2022 12:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-05T12:45:33.598Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jan 05, 2022 12:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-05T12:45:33.646Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Jan 05, 2022 12:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-05T12:45:33.674Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Jan 05, 2022 12:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-05T12:45:33.699Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Jan 05, 2022 12:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-05T12:45:33.730Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Jan 05, 2022 12:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-05T12:45:33.764Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jan 05, 2022 12:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-05T12:45:33.791Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jan 05, 2022 12:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-05T12:45:33.816Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jan 05, 2022 12:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-05T12:45:33.844Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Jan 05, 2022 12:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-05T12:45:33.889Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Jan 05, 2022 12:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-05T12:45:33.916Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Jan 05, 2022 12:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-05T12:45:33.942Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Jan 05, 2022 12:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-05T12:45:33.966Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Jan 05, 2022 12:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-05T12:45:33.999Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Jan 05, 2022 12:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-05T12:45:34.028Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Jan 05, 2022 12:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-05T12:45:34.052Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Jan 05, 2022 12:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-05T12:45:34.418Z: Starting 5 ****s in us-central1-f...
Jan 05, 2022 12:46:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-05T12:46:05.938Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jan 05, 2022 12:46:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-05T12:46:19.489Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Jan 05, 2022 12:47:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-05T12:47:17.842Z: Workers have started successfully.
Jan 05, 2022 12:47:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-05T12:47:17.866Z: Workers have started successfully.
Jan 05, 2022 4:00:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-05T16:00:39.869Z: Cancel request is committed for workflow job: 2022-01-05_04_45_21-6676539346797392784.
Jan 05, 2022 4:00:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-05T16:00:39.929Z: Cleaning up.
Jan 05, 2022 4:00:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-05T16:00:39.992Z: Stopping **** pool...
Jan 05, 2022 4:00:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-05T16:00:40.039Z: Stopping **** pool...
Jan 05, 2022 4:03:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-05T16:03:10.078Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Jan 05, 2022 4:03:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-05T16:03:10.118Z: Worker pool stopped.
Jan 05, 2022 4:03:16 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2022-01-05_04_45_21-6676539346797392784 finished with status CANCELLED.
Load test results for test (ID): 7d9982c9-e062-4434-8d46-088216e681a0 and timestamp: 2022-01-05T12:45:16.697000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11560.481
dataflow_v2_java11_total_bytes_count             2.39643879E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220105124332
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:7137665db55d01d50172fb0d4768f2e6cecb79cbdb098517f0d09d97ed136594
Deleted: sha256:0e827a4d980825e26c5c2d56e97857d739ed2083b8b0bae68a00ab6c2c3ee4e5
Deleted: sha256:a2bc780c0d88c20578f8478923934034ee2b8db57268b617dbcff046f35390c3
Deleted: sha256:ceedaf99168457ed04f9ca6f4368bb6a85f6d29b5bed7f4fbe922d1b90d6a93b
Deleted: sha256:2eb017db841e379f44308ebeace411f3f2146ad4f4e979cf6bef3e8b06c8639f
Deleted: sha256:0de3629b3f4ba3e6fb1f8b546717c6a78d25234647807d6b82ca5a6a2f3465bd
Deleted: sha256:de8eac2c5c4221c075ede7b88932157bcf403249f0fc3ea6bce7ea530daa921c
Deleted: sha256:7bec4ca7b92d6695cf4bb4919cafb33e9d7255424178b6a18f1d9547df02f778
Deleted: sha256:b8f000818eda283484bf53065768916386eeb0a1cfe405af7be7d812647d526d
Deleted: sha256:7aca4e5bee946b1e3ca042223e082d039fc71de7f45a7497d38873c05e46ef1c
Deleted: sha256:aee8816cfe97a735614a447381e4f04412baf27339e3b89c77fca7f258cceacc
Deleted: sha256:f986bd77d4926c81c0d51e2a54b8732f9dd911dd052aa497812afade5a445325
Deleted: sha256:eac72d85b2309e63174950be2fc74df8e708c141bc3a388c06aefa4799682af5
Deleted: sha256:f288495dac0edc1c231f30a6159b7ffa5d6699b25ac500ac2467d3676b3ddd59
Deleted: sha256:79355896f4cd65c1cdddb9e18c30ea145f08d9e89d56c7b6d73166c5edb7f542
Deleted: sha256:93c438fb125a42390afaeaf99ba53574c670bcc2b1bc6b5dac037f39705a5d30
Deleted: sha256:0ae842051a97172e946be6f46cbab586397f6af0d3ad3230ed5cd75396ec4965
Deleted: sha256:c1a4d7ce85f46b3853b9c218b8eea9ff68f32152ca132758213cef52c8a83127
Deleted: sha256:e76d1dea2fb20793674124805ef68677075c732b6bf3275a11c385e973c17014
Deleted: sha256:69467249d4830dbd28046693c5ed269dcc7ca60410d413e6d14548902a3135f0
Deleted: sha256:7792e9df2c213d348e2542ed66c17cde09a08b1db4fda4088a087d8c17898227
Deleted: sha256:3eed3985600bb8b28d1895d7a94c5895739a6d0e7f34dc520fcb06030e80f0b1
Deleted: sha256:f0c858696712ad971724a21e5f6f47df7e0bd3b862f9923b0701b40ad9bceb12
Deleted: sha256:a7b22ea0277298f7d43635c744a9a85e0c09b624e390707259c0e565292d2c72
Deleted: sha256:8b862980342be1a308902c7a1885ceb58b40e36623935ced063bc37da6472b05
Deleted: sha256:895e62a38d150202e15385318db8db8f8e44afa4348d28c883b6cde0045d26ec
Deleted: sha256:a910b330e8b70842f24a7ce6373a4ca19733cfb78912decb0e3abd92bbf27889
Deleted: sha256:de3c5ae05298cbae335e35b5c92661b7727c76b500402d8745f4f2f8fedbe09f
Deleted: sha256:22de5b1eb69cf2f296e0b5282de0331f459cf2a1441cbfc1c8b9b5fafabc83f3
Deleted: sha256:7b6fd1572752f26347ba56276ed613b0bd46dc7e5f2fecab6a5116e553eb8eab
Deleted: sha256:e53624c39170eacc9d4d86446bf7d93350f65ea7b31295441710b0c483e7af3c
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220105124332]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:7137665db55d01d50172fb0d4768f2e6cecb79cbdb098517f0d09d97ed136594]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220105124332] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:7137665db55d01d50172fb0d4768f2e6cecb79cbdb098517f0d09d97ed136594])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:7137665db55d01d50172fb0d4768f2e6cecb79cbdb098517f0d09d97ed136594
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:7137665db55d01d50172fb0d4768f2e6cecb79cbdb098517f0d09d97ed136594
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:7137665db55d01d50172fb0d4768f2e6cecb79cbdb098517f0d09d97ed136594].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings

Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.

BUILD FAILED in 3h 20m 2s
89 actionable tasks: 52 executed, 33 from cache, 4 up-to-date

Publishing build scan...
https://gradle.com/s/23ltmn2wictwa

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #201

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/201/display/redirect?page=changes>

Changes:

[mmack] [BEAM-13509] Stop sharing SQS client between readers of same source.

[mmack] [BEAM-13587] Attempt to load AWS region from default provider chain in


------------------------------------------
[...truncated 44.17 KB...]
3d39fcb7121f: Preparing
cfd8f0905ca0: Preparing
ecb9d918359f: Preparing
9969548bfc57: Preparing
c3d139bb1290: Preparing
9ed35a672ef7: Preparing
3b6a23a082e5: Preparing
bed8de9ba397: Preparing
3bb5258f46d2: Preparing
832e177bb500: Preparing
3bb5258f46d2: Waiting
f9e18e59a565: Preparing
26a504e63be4: Preparing
832e177bb500: Waiting
8bf42db0de72: Preparing
31892cc314cb: Preparing
26a504e63be4: Waiting
8bf42db0de72: Waiting
11936051f93b: Preparing
11936051f93b: Waiting
31892cc314cb: Waiting
9969548bfc57: Waiting
3b6a23a082e5: Waiting
bed8de9ba397: Waiting
ecb9d918359f: Waiting
cfd8f0905ca0: Waiting
e1874121dd0a: Pushed
3d39fcb7121f: Pushed
4fc88b0bbcec: Pushed
7040f79cdc38: Pushed
cfd8f0905ca0: Pushed
6d04881c087e: Pushed
9969548bfc57: Pushed
c3d139bb1290: Pushed
3b6a23a082e5: Pushed
3bb5258f46d2: Layer already exists
bed8de9ba397: Pushed
832e177bb500: Layer already exists
f9e18e59a565: Layer already exists
8bf42db0de72: Layer already exists
ecb9d918359f: Pushed
26a504e63be4: Layer already exists
11936051f93b: Layer already exists
31892cc314cb: Layer already exists
9ed35a672ef7: Pushed
20220104124340: digest: sha256:11440593d05b58c555056a779165a75590dae6404ce10696b92ab5de6927ed10 size: 4311

> Task :sdks:java:testing:load-tests:run
Jan 04, 2022 12:45:53 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Jan 04, 2022 12:45:53 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 203 files. Enable logging at DEBUG level to see which files will be staged.
Jan 04, 2022 12:45:54 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Jan 04, 2022 12:45:54 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Jan 04, 2022 12:45:56 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 203 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Jan 04, 2022 12:45:57 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 203 files cached, 0 files newly uploaded in 0 seconds
Jan 04, 2022 12:45:57 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Jan 04, 2022 12:45:57 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <113691 bytes, hash 76afdf79b87ee40efcfb8518451e84fd1d988cfc1ec422fa47936d06b0aa21d4> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-dq_febh-5A78-4UYRR6E_R2YjPwexCL6R5NtBrCqIdQ.pb
Jan 04, 2022 12:45:58 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Jan 04, 2022 12:45:58 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@33634f04, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4993febc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7601bc96, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@48a0c8aa, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6192a5d5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3722c145, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4cbc2e3b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2975a9e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@765ffb14, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@57562473, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7a360554, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@424de326, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4bc33720, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2dd0f797, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@67064bdc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4a7fd0c9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@18578491, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3291b443, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@671c4166, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@53d13cd4]
Jan 04, 2022 12:45:58 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Jan 04, 2022 12:45:58 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Jan 04, 2022 12:45:58 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Jan 04, 2022 12:45:58 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Jan 04, 2022 12:45:58 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1f1e58ca, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@57f847af, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ed34ef5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@553bc36c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@380e1909, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2d5ef498, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@95eb320, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2f521c4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4afbb6c2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@10db6131, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6c6017b9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4730e0f0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@506a1372, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1b7332a7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77c233af, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@37b56ac7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@37c41ec0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@35a0e495, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5399f6c5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4fe64d23]
Jan 04, 2022 12:45:58 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Jan 04, 2022 12:45:58 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Jan 04, 2022 12:45:58 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Jan 04, 2022 12:45:58 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Jan 04, 2022 12:45:58 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Jan 04, 2022 12:45:58 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Jan 04, 2022 12:45:58 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Jan 04, 2022 12:45:58 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Jan 04, 2022 12:45:58 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Jan 04, 2022 12:45:58 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Jan 04, 2022 12:45:58 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Jan 04, 2022 12:45:59 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.36.0-SNAPSHOT
Jan 04, 2022 12:46:00 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-01-04_04_45_59-6945613796395576760?project=apache-beam-testing
Jan 04, 2022 12:46:00 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2022-01-04_04_45_59-6945613796395576760
Jan 04, 2022 12:46:00 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2022-01-04_04_45_59-6945613796395576760
Jan 04, 2022 12:46:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2022-01-04T12:46:07.084Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-01-n2tk. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Jan 04, 2022 12:46:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-04T12:46:12.572Z: Worker configuration: e2-standard-2 in us-central1-f.
Jan 04, 2022 12:46:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-04T12:46:13.204Z: Expanding SplittableParDo operations into optimizable parts.
Jan 04, 2022 12:46:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-04T12:46:13.237Z: Expanding CollectionToSingleton operations into optimizable parts.
Jan 04, 2022 12:46:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-04T12:46:13.298Z: Expanding CoGroupByKey operations into optimizable parts.
Jan 04, 2022 12:46:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-04T12:46:13.388Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Jan 04, 2022 12:46:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-04T12:46:13.406Z: Expanding GroupByKey operations into streaming Read/Write steps
Jan 04, 2022 12:46:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-04T12:46:13.470Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jan 04, 2022 12:46:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-04T12:46:13.604Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jan 04, 2022 12:46:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-04T12:46:13.648Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Jan 04, 2022 12:46:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-04T12:46:13.695Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Jan 04, 2022 12:46:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-04T12:46:13.736Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Jan 04, 2022 12:46:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-04T12:46:13.774Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Jan 04, 2022 12:46:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-04T12:46:13.797Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jan 04, 2022 12:46:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-04T12:46:13.836Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jan 04, 2022 12:46:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-04T12:46:13.902Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jan 04, 2022 12:46:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-04T12:46:13.937Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Jan 04, 2022 12:46:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-04T12:46:13.972Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Jan 04, 2022 12:46:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-04T12:46:14.006Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Jan 04, 2022 12:46:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-04T12:46:14.051Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Jan 04, 2022 12:46:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-04T12:46:14.078Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jan 04, 2022 12:46:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-04T12:46:14.108Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jan 04, 2022 12:46:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-04T12:46:14.134Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jan 04, 2022 12:46:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-04T12:46:14.167Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Jan 04, 2022 12:46:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-04T12:46:14.212Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Jan 04, 2022 12:46:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-04T12:46:14.246Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Jan 04, 2022 12:46:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-04T12:46:14.270Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Jan 04, 2022 12:46:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-04T12:46:14.294Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Jan 04, 2022 12:46:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-04T12:46:14.325Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Jan 04, 2022 12:46:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-04T12:46:14.360Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Jan 04, 2022 12:46:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-04T12:46:14.389Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Jan 04, 2022 12:46:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-04T12:46:15.083Z: Starting 5 ****s in us-central1-f...
Jan 04, 2022 12:46:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-04T12:46:37.350Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jan 04, 2022 12:47:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-04T12:47:02.627Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Jan 04, 2022 12:47:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-04T12:47:59.109Z: Workers have started successfully.
Jan 04, 2022 12:47:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-04T12:47:59.135Z: Workers have started successfully.
Jan 04, 2022 4:00:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-04T16:00:35.198Z: Cancel request is committed for workflow job: 2022-01-04_04_45_59-6945613796395576760.
Jan 04, 2022 4:00:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-04T16:00:35.295Z: Cleaning up.
Jan 04, 2022 4:00:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-04T16:00:35.372Z: Stopping **** pool...
Jan 04, 2022 4:00:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-04T16:00:35.425Z: Stopping **** pool...
Jan 04, 2022 4:02:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-04T16:02:51.554Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Jan 04, 2022 4:02:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-04T16:02:51.588Z: Worker pool stopped.
Jan 04, 2022 4:02:57 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2022-01-04_04_45_59-6945613796395576760 finished with status CANCELLED.
Load test results for test (ID): 47e51832-f5cf-400c-ae48-f6bbcc6c458f and timestamp: 2022-01-04T12:45:53.883000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11526.081
dataflow_v2_java11_total_bytes_count             2.64100198E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220104124340
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:11440593d05b58c555056a779165a75590dae6404ce10696b92ab5de6927ed10
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220104124340]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:11440593d05b58c555056a779165a75590dae6404ce10696b92ab5de6927ed10]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220104124340] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:11440593d05b58c555056a779165a75590dae6404ce10696b92ab5de6927ed10])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:11440593d05b58c555056a779165a75590dae6404ce10696b92ab5de6927ed10
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:11440593d05b58c555056a779165a75590dae6404ce10696b92ab5de6927ed10
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:11440593d05b58c555056a779165a75590dae6404ce10696b92ab5de6927ed10].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings

Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.

BUILD FAILED in 3h 19m 38s
89 actionable tasks: 52 executed, 33 from cache, 4 up-to-date

Publishing build scan...
https://gradle.com/s/wuqby3xz6dyns

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #200

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/200/display/redirect>

Changes:


------------------------------------------
[...truncated 44.13 KB...]
9c08266a8aba: Preparing
377ecb17b8f1: Preparing
01f796727ffe: Preparing
ddb9abc2a84b: Preparing
0af330436921: Preparing
377ecb17b8f1: Waiting
01f796727ffe: Waiting
111b68a388aa: Preparing
ddb9abc2a84b: Waiting
f49adac5f6e1: Preparing
111b68a388aa: Waiting
407cbcefd7c1: Preparing
0af330436921: Waiting
f49adac5f6e1: Waiting
3bb5258f46d2: Preparing
832e177bb500: Preparing
f9e18e59a565: Preparing
26a504e63be4: Preparing
3bb5258f46d2: Waiting
407cbcefd7c1: Waiting
8bf42db0de72: Preparing
31892cc314cb: Preparing
26a504e63be4: Waiting
832e177bb500: Waiting
11936051f93b: Preparing
f9e18e59a565: Waiting
37c283b36270: Pushed
c69cc6eeddd0: Pushed
9c08266a8aba: Pushed
d76da2a02430: Pushed
377ecb17b8f1: Pushed
08a31831c26c: Pushed
ddb9abc2a84b: Pushed
0af330436921: Pushed
3bb5258f46d2: Layer already exists
832e177bb500: Layer already exists
f9e18e59a565: Layer already exists
26a504e63be4: Layer already exists
407cbcefd7c1: Pushed
f49adac5f6e1: Pushed
01f796727ffe: Pushed
8bf42db0de72: Layer already exists
31892cc314cb: Layer already exists
11936051f93b: Layer already exists
111b68a388aa: Pushed
20220103124334: digest: sha256:5c4801e630e008392fca95d09a29febc044bef86325e63ab06d801c3521caa8a size: 4311

> Task :sdks:java:testing:load-tests:run
Jan 03, 2022 12:45:17 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Jan 03, 2022 12:45:17 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 203 files. Enable logging at DEBUG level to see which files will be staged.
Jan 03, 2022 12:45:18 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Jan 03, 2022 12:45:18 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Jan 03, 2022 12:45:20 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 203 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Jan 03, 2022 12:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 203 files cached, 0 files newly uploaded in 1 seconds
Jan 03, 2022 12:45:22 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Jan 03, 2022 12:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <113691 bytes, hash fa094f55c8dd8d82ce23e78305adb82261940beec6d9dfdab5605f134da6b11c> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline--glPVcjdjYLOI-eDBa24ImGUC-7G2d_atWBfE02msRw.pb
Jan 03, 2022 12:45:23 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Jan 03, 2022 12:45:24 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@33634f04, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4993febc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7601bc96, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@48a0c8aa, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6192a5d5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3722c145, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4cbc2e3b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2975a9e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@765ffb14, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@57562473, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7a360554, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@424de326, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4bc33720, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2dd0f797, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@67064bdc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4a7fd0c9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@18578491, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3291b443, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@671c4166, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@53d13cd4]
Jan 03, 2022 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Jan 03, 2022 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Jan 03, 2022 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Jan 03, 2022 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Jan 03, 2022 12:45:24 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1f1e58ca, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@57f847af, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ed34ef5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@553bc36c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@380e1909, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2d5ef498, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@95eb320, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2f521c4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4afbb6c2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@10db6131, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6c6017b9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4730e0f0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@506a1372, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1b7332a7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77c233af, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@37b56ac7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@37c41ec0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@35a0e495, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5399f6c5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4fe64d23]
Jan 03, 2022 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Jan 03, 2022 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Jan 03, 2022 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Jan 03, 2022 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Jan 03, 2022 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Jan 03, 2022 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Jan 03, 2022 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Jan 03, 2022 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Jan 03, 2022 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Jan 03, 2022 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Jan 03, 2022 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Jan 03, 2022 12:45:24 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.36.0-SNAPSHOT
Jan 03, 2022 12:45:25 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-01-03_04_45_24-3301662855178043808?project=apache-beam-testing
Jan 03, 2022 12:45:25 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2022-01-03_04_45_24-3301662855178043808
Jan 03, 2022 12:45:25 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2022-01-03_04_45_24-3301662855178043808
Jan 03, 2022 12:45:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2022-01-03T12:45:31.400Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-01-piq. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Jan 03, 2022 12:45:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-03T12:45:35.491Z: Worker configuration: e2-standard-2 in us-central1-f.
Jan 03, 2022 12:45:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-03T12:45:36.173Z: Expanding SplittableParDo operations into optimizable parts.
Jan 03, 2022 12:45:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-03T12:45:36.205Z: Expanding CollectionToSingleton operations into optimizable parts.
Jan 03, 2022 12:45:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-03T12:45:36.261Z: Expanding CoGroupByKey operations into optimizable parts.
Jan 03, 2022 12:45:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-03T12:45:36.320Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Jan 03, 2022 12:45:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-03T12:45:36.350Z: Expanding GroupByKey operations into streaming Read/Write steps
Jan 03, 2022 12:45:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-03T12:45:36.420Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jan 03, 2022 12:45:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-03T12:45:36.513Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jan 03, 2022 12:45:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-03T12:45:36.560Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Jan 03, 2022 12:45:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-03T12:45:36.591Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Jan 03, 2022 12:45:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-03T12:45:36.624Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Jan 03, 2022 12:45:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-03T12:45:36.658Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Jan 03, 2022 12:45:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-03T12:45:36.690Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jan 03, 2022 12:45:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-03T12:45:36.715Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jan 03, 2022 12:45:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-03T12:45:36.738Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jan 03, 2022 12:45:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-03T12:45:36.788Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Jan 03, 2022 12:45:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-03T12:45:36.814Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Jan 03, 2022 12:45:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-03T12:45:36.846Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Jan 03, 2022 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-03T12:45:36.877Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Jan 03, 2022 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-03T12:45:36.907Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jan 03, 2022 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-03T12:45:36.931Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jan 03, 2022 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-03T12:45:36.965Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jan 03, 2022 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-03T12:45:36.998Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Jan 03, 2022 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-03T12:45:37.031Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Jan 03, 2022 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-03T12:45:37.056Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Jan 03, 2022 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-03T12:45:37.087Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Jan 03, 2022 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-03T12:45:37.141Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Jan 03, 2022 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-03T12:45:37.173Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Jan 03, 2022 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-03T12:45:37.206Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Jan 03, 2022 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-03T12:45:37.233Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Jan 03, 2022 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-03T12:45:37.709Z: Starting 5 ****s in us-central1-f...
Jan 03, 2022 12:46:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-03T12:46:05.936Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jan 03, 2022 12:46:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-03T12:46:31.550Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Jan 03, 2022 12:47:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-03T12:47:26.301Z: Workers have started successfully.
Jan 03, 2022 12:47:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-03T12:47:26.328Z: Workers have started successfully.
Jan 03, 2022 4:00:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-03T16:00:35.036Z: Cancel request is committed for workflow job: 2022-01-03_04_45_24-3301662855178043808.
Jan 03, 2022 4:00:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-03T16:00:35.088Z: Cleaning up.
Jan 03, 2022 4:00:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-03T16:00:35.150Z: Stopping **** pool...
Jan 03, 2022 4:00:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-03T16:00:35.193Z: Stopping **** pool...
Jan 03, 2022 4:02:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-03T16:02:57.207Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Jan 03, 2022 4:02:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-03T16:02:57.244Z: Worker pool stopped.
Jan 03, 2022 4:03:03 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2022-01-03_04_45_24-3301662855178043808 finished with status CANCELLED.
Load test results for test (ID): 24d069d5-f680-4866-b469-ae572d5588ba and timestamp: 2022-01-03T12:45:17.811000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11548.556
dataflow_v2_java11_total_bytes_count             2.11090174E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220103124334
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:5c4801e630e008392fca95d09a29febc044bef86325e63ab06d801c3521caa8a
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220103124334]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:5c4801e630e008392fca95d09a29febc044bef86325e63ab06d801c3521caa8a]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220103124334] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:5c4801e630e008392fca95d09a29febc044bef86325e63ab06d801c3521caa8a])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:5c4801e630e008392fca95d09a29febc044bef86325e63ab06d801c3521caa8a
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:5c4801e630e008392fca95d09a29febc044bef86325e63ab06d801c3521caa8a
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:5c4801e630e008392fca95d09a29febc044bef86325e63ab06d801c3521caa8a].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings

Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.

BUILD FAILED in 3h 19m 48s
89 actionable tasks: 52 executed, 33 from cache, 4 up-to-date

Publishing build scan...
https://gradle.com/s/r6pvf2kehh5sm

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 - Build # 199 - Aborted!

Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 - Build # 199 - Aborted:

Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/199/ to view the results.

Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #198

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/198/display/redirect?page=changes>

Changes:

[noreply] [BEAM-13541] Fix spotbugs issue with DLS in tail (#16407)

[noreply] [BEAM-13588] Add missing module for PVR tests and also sickbay known

[noreply] [BEAM-13575] Sickbay test that is flaky to restore precommit test signal

[noreply] [BEAM-12092] Bump jedis to version 3.8.0 (#16403)


------------------------------------------
[...truncated 44.69 KB...]
31892cc314cb: Preparing
11936051f93b: Preparing
31892cc314cb: Waiting
11936051f93b: Waiting
9efaf169ab33: Pushed
09ee4e8c9a7a: Pushed
adaa8aefeb93: Pushed
0fa3bfc97b21: Pushed
ce84906a8d75: Pushed
da52bf965b1e: Pushed
71afe8bec404: Pushed
876244654db6: Pushed
3bb5258f46d2: Layer already exists
832e177bb500: Layer already exists
f9e18e59a565: Layer already exists
26a504e63be4: Layer already exists
b3625b798ce3: Pushed
8bf42db0de72: Layer already exists
31892cc314cb: Layer already exists
11936051f93b: Layer already exists
fd6ec0e53c9a: Pushed
f4e94253fbf6: Pushed
da2b13271c97: Pushed
20220101124331: digest: sha256:5448f5165a3efe69bdb1b7278c1c1e2a1ab61b67f49f412c9707c4402e724f9b size: 4311

> Task :sdks:java:testing:load-tests:run
Jan 01, 2022 12:45:15 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Jan 01, 2022 12:45:16 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 203 files. Enable logging at DEBUG level to see which files will be staged.
Jan 01, 2022 12:45:16 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Jan 01, 2022 12:45:16 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Jan 01, 2022 12:45:18 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 203 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Jan 01, 2022 12:45:19 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 203 files cached, 0 files newly uploaded in 0 seconds
Jan 01, 2022 12:45:19 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Jan 01, 2022 12:45:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <113691 bytes, hash f4c201c582559cf176342a2737864b67d6f0279d30ce7162996a65249d5642a2> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-9MIBxYJVnPF2NConN4ZLZ9bwJ50wznFimWplJJ1WQqI.pb
Jan 01, 2022 12:45:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Jan 01, 2022 12:45:21 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7601bc96, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@48a0c8aa, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6192a5d5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3722c145, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4cbc2e3b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2975a9e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@765ffb14, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@57562473, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7a360554, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@424de326, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4bc33720, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2dd0f797, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@67064bdc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4a7fd0c9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@18578491, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3291b443, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@671c4166, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@53d13cd4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77865933, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@480ad82c]
Jan 01, 2022 12:45:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Jan 01, 2022 12:45:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Jan 01, 2022 12:45:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Jan 01, 2022 12:45:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Jan 01, 2022 12:45:21 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ed34ef5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@553bc36c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@380e1909, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2d5ef498, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@95eb320, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2f521c4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4afbb6c2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@10db6131, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6c6017b9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4730e0f0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@506a1372, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1b7332a7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77c233af, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@37b56ac7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@37c41ec0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@35a0e495, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5399f6c5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4fe64d23, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58437801, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6af5bbd0]
Jan 01, 2022 12:45:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Jan 01, 2022 12:45:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Jan 01, 2022 12:45:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Jan 01, 2022 12:45:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Jan 01, 2022 12:45:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Jan 01, 2022 12:45:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Jan 01, 2022 12:45:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Jan 01, 2022 12:45:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Jan 01, 2022 12:45:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Jan 01, 2022 12:45:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Jan 01, 2022 12:45:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Jan 01, 2022 12:45:21 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.36.0-SNAPSHOT
Jan 01, 2022 12:45:22 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-01-01_04_45_21-16412038114127075776?project=apache-beam-testing
Jan 01, 2022 12:45:22 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2022-01-01_04_45_21-16412038114127075776
Jan 01, 2022 12:45:22 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2022-01-01_04_45_21-16412038114127075776
Jan 01, 2022 12:45:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2022-01-01T12:45:28.911Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-01-nwc4. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Jan 01, 2022 12:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-01T12:45:33.711Z: Worker configuration: e2-standard-2 in us-central1-f.
Jan 01, 2022 12:45:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-01T12:45:34.503Z: Expanding SplittableParDo operations into optimizable parts.
Jan 01, 2022 12:45:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-01T12:45:34.533Z: Expanding CollectionToSingleton operations into optimizable parts.
Jan 01, 2022 12:45:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-01T12:45:34.601Z: Expanding CoGroupByKey operations into optimizable parts.
Jan 01, 2022 12:45:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-01T12:45:34.674Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Jan 01, 2022 12:45:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-01T12:45:34.713Z: Expanding GroupByKey operations into streaming Read/Write steps
Jan 01, 2022 12:45:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-01T12:45:34.782Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jan 01, 2022 12:45:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-01T12:45:34.882Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jan 01, 2022 12:45:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-01T12:45:34.930Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Jan 01, 2022 12:45:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-01T12:45:34.961Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Jan 01, 2022 12:45:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-01T12:45:34.994Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Jan 01, 2022 12:45:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-01T12:45:35.021Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Jan 01, 2022 12:45:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-01T12:45:35.043Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jan 01, 2022 12:45:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-01T12:45:35.070Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jan 01, 2022 12:45:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-01T12:45:35.104Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jan 01, 2022 12:45:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-01T12:45:35.138Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Jan 01, 2022 12:45:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-01T12:45:35.171Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Jan 01, 2022 12:45:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-01T12:45:35.206Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Jan 01, 2022 12:45:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-01T12:45:35.245Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Jan 01, 2022 12:45:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-01T12:45:35.280Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jan 01, 2022 12:45:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-01T12:45:35.310Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jan 01, 2022 12:45:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-01T12:45:35.342Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jan 01, 2022 12:45:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-01T12:45:35.375Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Jan 01, 2022 12:45:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-01T12:45:35.409Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Jan 01, 2022 12:45:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-01T12:45:35.454Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Jan 01, 2022 12:45:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-01T12:45:35.489Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Jan 01, 2022 12:45:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-01T12:45:35.519Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Jan 01, 2022 12:45:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-01T12:45:35.553Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Jan 01, 2022 12:45:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-01T12:45:35.588Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Jan 01, 2022 12:45:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-01T12:45:35.621Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Jan 01, 2022 12:45:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-01T12:45:36.048Z: Starting 5 ****s in us-central1-f...
Jan 01, 2022 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-01T12:45:47.653Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jan 01, 2022 12:46:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-01T12:46:26.897Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Jan 01, 2022 12:47:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-01T12:47:22.288Z: Workers have started successfully.
Jan 01, 2022 12:47:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-01T12:47:22.316Z: Workers have started successfully.
Jan 01, 2022 1:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-01T13:45:38.432Z: Staged package jackson-dataformat-cbor-2.13.0-87i8Y8vQiEl2BVWwSbyVD2k9G-KE9nB4qPOXblapNm4.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/jackson-dataformat-cbor-2.13.0-87i8Y8vQiEl2BVWwSbyVD2k9G-KE9nB4qPOXblapNm4.jar' is inaccessible.
Jan 01, 2022 1:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2022-01-01T13:45:39.541Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jan 01, 2022 1:48:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2022-01-01T13:48:39.675Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jan 01, 2022 1:51:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-01T13:51:38.751Z: Staged package jackson-dataformat-cbor-2.13.0-87i8Y8vQiEl2BVWwSbyVD2k9G-KE9nB4qPOXblapNm4.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/jackson-dataformat-cbor-2.13.0-87i8Y8vQiEl2BVWwSbyVD2k9G-KE9nB4qPOXblapNm4.jar' is inaccessible.
Jan 01, 2022 1:51:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2022-01-01T13:51:39.906Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jan 01, 2022 1:54:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2022-01-01T13:54:39.497Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jan 01, 2022 1:57:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-01T13:57:38.630Z: Staged package jackson-dataformat-cbor-2.13.0-87i8Y8vQiEl2BVWwSbyVD2k9G-KE9nB4qPOXblapNm4.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/jackson-dataformat-cbor-2.13.0-87i8Y8vQiEl2BVWwSbyVD2k9G-KE9nB4qPOXblapNm4.jar' is inaccessible.
Jan 01, 2022 1:57:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2022-01-01T13:57:39.794Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jan 01, 2022 2:00:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2022-01-01T14:00:39.488Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jan 01, 2022 2:03:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-01-01T14:03:38.599Z: Staged package jackson-dataformat-cbor-2.13.0-87i8Y8vQiEl2BVWwSbyVD2k9G-KE9nB4qPOXblapNm4.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/jackson-dataformat-cbor-2.13.0-87i8Y8vQiEl2BVWwSbyVD2k9G-KE9nB4qPOXblapNm4.jar' is inaccessible.
Jan 01, 2022 2:03:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2022-01-01T14:03:39.574Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jan 01, 2022 4:00:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-01T16:00:31.897Z: Cancel request is committed for workflow job: 2022-01-01_04_45_21-16412038114127075776.
Jan 01, 2022 4:00:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-01T16:00:31.946Z: Cleaning up.
Jan 01, 2022 4:00:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-01T16:00:32.058Z: Stopping **** pool...
Jan 01, 2022 4:00:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-01T16:00:32.114Z: Stopping **** pool...
Jan 01, 2022 4:02:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-01T16:02:59.295Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Jan 01, 2022 4:02:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-01-01T16:02:59.327Z: Worker pool stopped.
Jan 01, 2022 4:03:06 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2022-01-01_04_45_21-16412038114127075776 finished with status CANCELLED.
Load test results for test (ID): 5f434bb9-4a16-460c-9420-6672f7023320 and timestamp: 2022-01-01T12:45:16.445000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11562.529
dataflow_v2_java11_total_bytes_count             1.98463947E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220101124331
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:5448f5165a3efe69bdb1b7278c1c1e2a1ab61b67f49f412c9707c4402e724f9b
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220101124331]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:5448f5165a3efe69bdb1b7278c1c1e2a1ab61b67f49f412c9707c4402e724f9b]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220101124331] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:5448f5165a3efe69bdb1b7278c1c1e2a1ab61b67f49f412c9707c4402e724f9b])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:5448f5165a3efe69bdb1b7278c1c1e2a1ab61b67f49f412c9707c4402e724f9b
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:5448f5165a3efe69bdb1b7278c1c1e2a1ab61b67f49f412c9707c4402e724f9b
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:5448f5165a3efe69bdb1b7278c1c1e2a1ab61b67f49f412c9707c4402e724f9b].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings

Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.

BUILD FAILED in 3h 19m 51s
89 actionable tasks: 52 executed, 33 from cache, 4 up-to-date

Publishing build scan...
https://gradle.com/s/ikjufpq4rbezs

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #197

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/197/display/redirect?page=changes>

Changes:

[stranniknm] [BEAM-13555] playground - quick fix for safari

[mmack] [BEAM-13009] Fix tests to ensure absence of duplicates only per request

[mmack] [adhoc] Use predefined min/max combiners in SQS reader to gather stats

[zyichi] Fix class not found in perf integration tests.

[noreply] Update

[noreply] [BEAM-13430] Fix class not found error for example integration tests and

[noreply] [BEAM-13430] Re-enable dependency analysis for modules. (#16395)

[noreply] [BEAM-13484][Playground] Improve Terraform Scripts For Deploy

[noreply] [BEAM-13541] More intelligent caching of CoGBK values. (#16354)

[noreply] Merge pull request #15769 from [BEAM-13031] [Playground] Code editor -


------------------------------------------
[...truncated 99.45 KB...]
Dec 31, 2021 1:09:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-31T13:09:41.976Z: Staged package google-api-services-pubsub-v1-rev20211012-1.32.1-tlbE3NRPfq1Tei5jlklIHI3_Lr4AjKiDD53olYlLfVE.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-api-services-pubsub-v1-rev20211012-1.32.1-tlbE3NRPfq1Tei5jlklIHI3_Lr4AjKiDD53olYlLfVE.jar' is inaccessible.
Dec 31, 2021 1:09:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-31T13:09:42.048Z: Staged package google-api-services-storage-v1-rev20211018-1.32.1-2gWjhhV23GKwelCtsAd9ctESRIU6uPJIsOmeUs_0hi8.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-api-services-storage-v1-rev20211018-1.32.1-2gWjhhV23GKwelCtsAd9ctESRIU6uPJIsOmeUs_0hi8.jar' is inaccessible.
Dec 31, 2021 1:09:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-31T13:09:42.100Z: Staged package google-auth-library-credentials-1.2.1-nCGtJZrqUXMZnlzIo91vLcP41fMbfGg44IcF2O_Y5qA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-auth-library-credentials-1.2.1-nCGtJZrqUXMZnlzIo91vLcP41fMbfGg44IcF2O_Y5qA.jar' is inaccessible.
Dec 31, 2021 1:09:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-31T13:09:42.144Z: Staged package google-auth-library-oauth2-http-1.2.1-ymQyaBiipFjTUxnvljub7iaq-AcvCqv2UqS-k06SUV0.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-auth-library-oauth2-http-1.2.1-ymQyaBiipFjTUxnvljub7iaq-AcvCqv2UqS-k06SUV0.jar' is inaccessible.
Dec 31, 2021 1:09:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-31T13:09:42.190Z: Staged package google-cloud-bigquery-2.3.3-h5H0Z_KojTQDv3i8rjpgEY4RPPFWSFRnGLhqpvwVOmc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-cloud-bigquery-2.3.3-h5H0Z_KojTQDv3i8rjpgEY4RPPFWSFRnGLhqpvwVOmc.jar' is inaccessible.
Dec 31, 2021 1:09:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-31T13:09:42.233Z: Staged package google-cloud-bigquerystorage-2.4.2-5YVJ7WVXPG5s8grIJNjOCJLzuAXvvV7hycFi9NxJ_Ac.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-cloud-bigquerystorage-2.4.2-5YVJ7WVXPG5s8grIJNjOCJLzuAXvvV7hycFi9NxJ_Ac.jar' is inaccessible.
Dec 31, 2021 1:09:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-31T13:09:42.270Z: Staged package google-cloud-bigtable-2.2.0-KABz6OcxVcBW19P_ZXYkYmoNylNzEKunlIxSVP6ryik.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-cloud-bigtable-2.2.0-KABz6OcxVcBW19P_ZXYkYmoNylNzEKunlIxSVP6ryik.jar' is inaccessible.
Dec 31, 2021 1:09:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-31T13:09:42.312Z: Staged package google-cloud-core-2.2.0-bg2jk3pGS8qiBzYFrbcz7Adg9ZCCcWQWfGjbckAfX_g.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-cloud-core-2.2.0-bg2jk3pGS8qiBzYFrbcz7Adg9ZCCcWQWfGjbckAfX_g.jar' is inaccessible.
Dec 31, 2021 1:09:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-31T13:09:42.356Z: Staged package google-cloud-core-grpc-2.2.0-25EdFn1IvmF3iXOnwS1gTe1otKU1ONZ8yntYNSwEM1A.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-cloud-core-grpc-2.2.0-25EdFn1IvmF3iXOnwS1gTe1otKU1ONZ8yntYNSwEM1A.jar' is inaccessible.
Dec 31, 2021 1:09:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-31T13:09:42.398Z: Staged package google-cloud-core-http-2.2.0-m-jbHJU73e-M-rojYZiTQYKX-gb3ZWoffl24bynuFpk.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-cloud-core-http-2.2.0-m-jbHJU73e-M-rojYZiTQYKX-gb3ZWoffl24bynuFpk.jar' is inaccessible.
Dec 31, 2021 1:09:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-31T13:09:42.436Z: Staged package google-cloud-firestore-3.0.6-0H7ybq6QzTpRy9fPhvcrT77cj81RRbcyO9SXDS7ozxk.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-cloud-firestore-3.0.6-0H7ybq6QzTpRy9fPhvcrT77cj81RRbcyO9SXDS7ozxk.jar' is inaccessible.
Dec 31, 2021 1:09:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-31T13:09:42.480Z: Staged package google-cloud-pubsub-1.114.7-woRLOVDIiIpw8CLOmWlYCs2gKrDJr_HFlXd_UHco4Hs.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-cloud-pubsub-1.114.7-woRLOVDIiIpw8CLOmWlYCs2gKrDJr_HFlXd_UHco4Hs.jar' is inaccessible.
Dec 31, 2021 1:09:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-31T13:09:42.560Z: Staged package google-cloud-spanner-6.14.0-tQ8OcjcjMFVRBiw6mkwxf2e43T9agLgRzEBanWqjlvc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-cloud-spanner-6.14.0-tQ8OcjcjMFVRBiw6mkwxf2e43T9agLgRzEBanWqjlvc.jar' is inaccessible.
Dec 31, 2021 1:09:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-31T13:09:42.622Z: Staged package google-http-client-1.40.1-BpXaPzhHg4yT7ID8hldkBl5hADVfKxmOZjz_JIwGcmM.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-http-client-1.40.1-BpXaPzhHg4yT7ID8hldkBl5hADVfKxmOZjz_JIwGcmM.jar' is inaccessible.
Dec 31, 2021 1:09:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-31T13:09:42.664Z: Staged package google-http-client-apache-v2-1.40.1-cs8kwbGkAltMXtc7wVYlUFJEZNA-uRLq2kM6-mjL1Kc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-http-client-apache-v2-1.40.1-cs8kwbGkAltMXtc7wVYlUFJEZNA-uRLq2kM6-mjL1Kc.jar' is inaccessible.
Dec 31, 2021 1:09:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-31T13:09:42.725Z: Staged package google-http-client-appengine-1.40.1-NF6cBtmubRxM6pmMKfLCGa9VpaudAxDIh7imCPupxNk.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-http-client-appengine-1.40.1-NF6cBtmubRxM6pmMKfLCGa9VpaudAxDIh7imCPupxNk.jar' is inaccessible.
Dec 31, 2021 1:09:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-31T13:09:42.763Z: Staged package google-http-client-gson-1.40.1-4LLTtTR1cge1KtAXql5h0N-TN73CsLiiP2bGxKNiqDE.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-http-client-gson-1.40.1-4LLTtTR1cge1KtAXql5h0N-TN73CsLiiP2bGxKNiqDE.jar' is inaccessible.
Dec 31, 2021 1:09:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-31T13:09:42.795Z: Staged package google-http-client-jackson2-1.40.1-Z6N6POQYvgaGZyJkrLI07CaXlh8rylsLMY0DfYqo9Kg.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-http-client-jackson2-1.40.1-Z6N6POQYvgaGZyJkrLI07CaXlh8rylsLMY0DfYqo9Kg.jar' is inaccessible.
Dec 31, 2021 1:09:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-31T13:09:42.832Z: Staged package google-http-client-protobuf-1.40.1-TtuwwZAI-ILzLP08Iu0cYugu62RsDofESZAOrus5EFU.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-http-client-protobuf-1.40.1-TtuwwZAI-ILzLP08Iu0cYugu62RsDofESZAOrus5EFU.jar' is inaccessible.
Dec 31, 2021 1:09:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-31T13:09:42.869Z: Staged package google-oauth-client-1.32.1-BjA55K1f9S_AagXi8aDjR87ky_Eeqbea63qbRdfG2_I.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-oauth-client-1.32.1-BjA55K1f9S_AagXi8aDjR87ky_Eeqbea63qbRdfG2_I.jar' is inaccessible.
Dec 31, 2021 1:09:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-31T13:09:42.911Z: Staged package grpc-alts-1.41.0-buUDV4vPyIeTwOEME8gfxOF1vd15_qcfFK0lKyG_RkI.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/grpc-alts-1.41.0-buUDV4vPyIeTwOEME8gfxOF1vd15_qcfFK0lKyG_RkI.jar' is inaccessible.
Dec 31, 2021 1:09:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-31T13:09:42.947Z: Staged package grpc-api-1.41.0-aCBqGaxOsCYRSqPna6z64TGcWLvghzVNK5Ble6brd0g.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/grpc-api-1.41.0-aCBqGaxOsCYRSqPna6z64TGcWLvghzVNK5Ble6brd0g.jar' is inaccessible.
Dec 31, 2021 1:09:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-31T13:09:42.991Z: Staged package grpc-auth-1.41.0-XXIg-0qjPOJgMrqBiBimY3HRI2979J4ZCElMIJBOiW4.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/grpc-auth-1.41.0-XXIg-0qjPOJgMrqBiBimY3HRI2979J4ZCElMIJBOiW4.jar' is inaccessible.
Dec 31, 2021 1:09:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-31T13:09:43.062Z: Staged package grpc-context-1.41.0-DjVVMFHhq-1W33SPICLwdYBznLUyUGaeNSxLBdTu9iQ.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/grpc-context-1.41.0-DjVVMFHhq-1W33SPICLwdYBznLUyUGaeNSxLBdTu9iQ.jar' is inaccessible.
Dec 31, 2021 1:09:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-31T13:09:43.103Z: Staged package grpc-core-1.41.0-DKE1JbG9j-inB3fVeARoifvpuFLIn5hpkMYmQHSW0hI.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/grpc-core-1.41.0-DKE1JbG9j-inB3fVeARoifvpuFLIn5hpkMYmQHSW0hI.jar' is inaccessible.
Dec 31, 2021 1:09:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-31T13:09:43.148Z: Staged package grpc-gcp-1.1.0-zKq--BAvLo2Cks8hjNm7iNASsOWFeFMqwc55hcc1YEY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/grpc-gcp-1.1.0-zKq--BAvLo2Cks8hjNm7iNASsOWFeFMqwc55hcc1YEY.jar' is inaccessible.
Dec 31, 2021 1:09:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-31T13:09:43.188Z: Staged package grpc-google-cloud-bigquerystorage-v1-2.4.2-rcyu5coRb3EhnZYcqVFKVTBwhLq_0TtMhHMWm0lJ__w.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/grpc-google-cloud-bigquerystorage-v1-2.4.2-rcyu5coRb3EhnZYcqVFKVTBwhLq_0TtMhHMWm0lJ__w.jar' is inaccessible.
Dec 31, 2021 1:09:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-31T13:09:43.229Z: Staged package grpc-google-cloud-bigquerystorage-v1beta1-0.128.2-ete21VWpi5GseyDR_FnEThEmAJf71-MqB14Ln51KLvo.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/grpc-google-cloud-bigquerystorage-v1beta1-0.128.2-ete21VWpi5GseyDR_FnEThEmAJf71-MqB14Ln51KLvo.jar' is inaccessible.
Dec 31, 2021 1:09:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-31T13:09:43.265Z: Staged package grpc-google-cloud-bigquerystorage-v1beta2-0.128.2-B4C0HfWYubAE_53ivzw1sJKsKD4I5NHY_wMvwDqUOpc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/grpc-google-cloud-bigquerystorage-v1beta2-0.128.2-B4C0HfWYubAE_53ivzw1sJKsKD4I5NHY_wMvwDqUOpc.jar' is inaccessible.
Dec 31, 2021 1:09:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-31T13:09:43.311Z: Staged package grpc-google-cloud-bigtable-admin-v2-2.2.0-LkV5bnp7fgLAzTBX7LgKIvs7DlKD7G0Q01lin_rH8ac.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/grpc-google-cloud-bigtable-admin-v2-2.2.0-LkV5bnp7fgLAzTBX7LgKIvs7DlKD7G0Q01lin_rH8ac.jar' is inaccessible.
Dec 31, 2021 1:09:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-31T13:09:43.394Z: Staged package grpc-google-cloud-bigtable-v2-2.2.0-qgo1zO_IHIy4Dmca1kB7VggPE7yqRbbiCTZPenixFlw.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/grpc-google-cloud-bigtable-v2-2.2.0-qgo1zO_IHIy4Dmca1kB7VggPE7yqRbbiCTZPenixFlw.jar' is inaccessible.
Dec 31, 2021 1:09:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-31T13:09:43.433Z: Staged package grpc-google-cloud-pubsub-v1-1.96.7-tvXiMnIk_9ysKh_V9bdga6iTlt80SFH6JmDfEHfNURA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/grpc-google-cloud-pubsub-v1-1.96.7-tvXiMnIk_9ysKh_V9bdga6iTlt80SFH6JmDfEHfNURA.jar' is inaccessible.
Dec 31, 2021 1:09:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-31T13:09:43.477Z: Staged package grpc-google-cloud-spanner-admin-database-v1-6.14.0-N3sEkaCh7OdyURtjtvlCmBDKIfy88Q5BnFF76usTO6M.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/grpc-google-cloud-spanner-admin-database-v1-6.14.0-N3sEkaCh7OdyURtjtvlCmBDKIfy88Q5BnFF76usTO6M.jar' is inaccessible.
Dec 31, 2021 1:09:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-31T13:09:43.531Z: Staged package grpc-google-cloud-spanner-admin-instance-v1-6.14.0-ZsNtJIROHYhwMWP8en-Xz2Yurf-b1E5euM7Jg1cPe5k.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/grpc-google-cloud-spanner-admin-instance-v1-6.14.0-ZsNtJIROHYhwMWP8en-Xz2Yurf-b1E5euM7Jg1cPe5k.jar' is inaccessible.
Dec 31, 2021 1:09:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-31T13:09:43.567Z: Staged package grpc-google-cloud-spanner-v1-6.14.0-qVyAMCy3RymhF9rEgQ_4t3QI1S9iq3eM6w_nyYb4pg0.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/grpc-google-cloud-spanner-v1-6.14.0-qVyAMCy3RymhF9rEgQ_4t3QI1S9iq3eM6w_nyYb4pg0.jar' is inaccessible.
Dec 31, 2021 1:09:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-31T13:09:43.603Z: Staged package grpc-google-cloud-storage-v2-2.0.1-alpha-lJsdHAHrV8TzJsKX-hrfXVKVS7MsFS0C6BBA6Sf7sIc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/grpc-google-cloud-storage-v2-2.0.1-alpha-lJsdHAHrV8TzJsKX-hrfXVKVS7MsFS0C6BBA6Sf7sIc.jar' is inaccessible.
Dec 31, 2021 1:09:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-31T13:09:43.646Z: Staged package grpc-google-common-protos-2.6.0-NPLxjU4ENdg2phomClAqI2VfxW_qfHU28jEvsuXycbs.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/grpc-google-common-protos-2.6.0-NPLxjU4ENdg2phomClAqI2VfxW_qfHU28jEvsuXycbs.jar' is inaccessible.
Dec 31, 2021 1:09:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-31T13:09:43.688Z: Staged package grpc-grpclb-1.41.0-KzVoSsMCeWN_1GtLlTN_sXQP2s-JtSi9-oj-lmuX8Jw.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/grpc-grpclb-1.41.0-KzVoSsMCeWN_1GtLlTN_sXQP2s-JtSi9-oj-lmuX8Jw.jar' is inaccessible.
Dec 31, 2021 1:09:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-31T13:09:43.728Z: Staged package grpc-netty-1.41.0-FukMMrsnufcSkq76WASP0ibUqpSw0FrV2kCc-D12PJo.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/grpc-netty-1.41.0-FukMMrsnufcSkq76WASP0ibUqpSw0FrV2kCc-D12PJo.jar' is inaccessible.
Dec 31, 2021 1:09:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-31T13:09:43.771Z: Staged package grpc-netty-shaded-1.41.0-lJ8OF6tIvAkRhUWA-Zva4PJSFIJmwcD5Z0e1r6dAZUk.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/grpc-netty-shaded-1.41.0-lJ8OF6tIvAkRhUWA-Zva4PJSFIJmwcD5Z0e1r6dAZUk.jar' is inaccessible.
Dec 31, 2021 1:09:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-31T13:09:43.809Z: Staged package grpc-protobuf-1.41.0-Q4P84ABZaYAiAyfh9sB-L4VbcQjRuaYCd5495YRneZY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/grpc-protobuf-1.41.0-Q4P84ABZaYAiAyfh9sB-L4VbcQjRuaYCd5495YRneZY.jar' is inaccessible.
Dec 31, 2021 1:09:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-31T13:09:43.871Z: Staged package grpc-stub-1.41.0-FBU9bDpjIwB1rwrfBLYKTFjcQBMkLll5y_P17xO-uLU.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/grpc-stub-1.41.0-FBU9bDpjIwB1rwrfBLYKTFjcQBMkLll5y_P17xO-uLU.jar' is inaccessible.
Dec 31, 2021 1:09:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-31T13:09:43.950Z: Staged package guava-31.0.1-jre-1b6U1l6HvSGfsxk60VF7qlWjuI_JHSHPc1gmq1rwh7k.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/guava-31.0.1-jre-1b6U1l6HvSGfsxk60VF7qlWjuI_JHSHPc1gmq1rwh7k.jar' is inaccessible.
Dec 31, 2021 1:09:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-31T13:09:44.042Z: Staged package httpclient-4.5.13-b-kCalZsalABYIzz_DIZZkH2weXhmG0QN8zb1fMe90M.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/httpclient-4.5.13-b-kCalZsalABYIzz_DIZZkH2weXhmG0QN8zb1fMe90M.jar' is inaccessible.
Dec 31, 2021 1:09:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-31T13:09:44.079Z: Staged package httpcore-4.4.14--VYgnkUMsdDFF3bfvSPlPp3Y25oSmO1itwvwlEumOyg.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/httpcore-4.4.14--VYgnkUMsdDFF3bfvSPlPp3Y25oSmO1itwvwlEumOyg.jar' is inaccessible.
Dec 31, 2021 1:09:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-31T13:09:44.217Z: Staged package jackson-annotations-2.13.0-gflyTYhD6LCPj2wGCeeisDDQDDSGHErH4gmacjUEfW8.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/jackson-annotations-2.13.0-gflyTYhD6LCPj2wGCeeisDDQDDSGHErH4gmacjUEfW8.jar' is inaccessible.
Dec 31, 2021 1:09:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-31T13:09:44.245Z: Staged package jackson-core-2.13.0-NIvFmzSN8ugHs1bx1i0q-0GpdAczKKvHc-sJMrhV0sg.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/jackson-core-2.13.0-NIvFmzSN8ugHs1bx1i0q-0GpdAczKKvHc-sJMrhV0sg.jar' is inaccessible.
Dec 31, 2021 1:09:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-31T13:09:44.320Z: Staged package jackson-databind-2.13.0-nIJtJxdiaHd63Pl-HG4gUcfjOnqqLDcMLoxgd_2do_Q.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/jackson-databind-2.13.0-nIJtJxdiaHd63Pl-HG4gUcfjOnqqLDcMLoxgd_2do_Q.jar' is inaccessible.
Dec 31, 2021 1:09:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-31T13:09:44.378Z: Staged package jackson-dataformat-yaml-2.13.0-qrbF97bznPcp-z7JnadDcAu6rQdd6kHQZE6N8EyPnqM.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/jackson-dataformat-yaml-2.13.0-qrbF97bznPcp-z7JnadDcAu6rQdd6kHQZE6N8EyPnqM.jar' is inaccessible.
Dec 31, 2021 1:09:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-31T13:09:44.725Z: Staged package metrics-core-3.1.2-JFuipmqbxxDOTbFHERJud7y05tlu9-YiZZKA88kMu1w.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/metrics-core-3.1.2-JFuipmqbxxDOTbFHERJud7y05tlu9-YiZZKA88kMu1w.jar' is inaccessible.
Dec 31, 2021 1:09:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-31T13:09:45.115Z: Staged package proto-google-cloud-bigquerystorage-v1-2.4.2-I8k8jupfFMXmd3oCRu97y-SArCaAGuc9S9lAKYz6aNY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/proto-google-cloud-bigquerystorage-v1-2.4.2-I8k8jupfFMXmd3oCRu97y-SArCaAGuc9S9lAKYz6aNY.jar' is inaccessible.
Dec 31, 2021 1:09:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-31T13:09:45.154Z: Staged package proto-google-cloud-bigquerystorage-v1beta1-0.128.2-rY58mb0Q3U3wXkzhrxUkoFiELXER0H_ZeBM56mk09gY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/proto-google-cloud-bigquerystorage-v1beta1-0.128.2-rY58mb0Q3U3wXkzhrxUkoFiELXER0H_ZeBM56mk09gY.jar' is inaccessible.
Dec 31, 2021 1:09:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-31T13:09:45.186Z: Staged package proto-google-cloud-bigquerystorage-v1beta2-0.128.2-QS2p7NzODxTEWnvg5_0_0qQkS3IcYzxDPom7NrGk2PU.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/proto-google-cloud-bigquerystorage-v1beta2-0.128.2-QS2p7NzODxTEWnvg5_0_0qQkS3IcYzxDPom7NrGk2PU.jar' is inaccessible.
Dec 31, 2021 1:09:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-31T13:09:45.225Z: Staged package proto-google-cloud-bigtable-admin-v2-2.2.0-roMg0XgAMQ2FIQF6CZxvH9TaB4kHQlkbpPnfZ33Qgh8.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/proto-google-cloud-bigtable-admin-v2-2.2.0-roMg0XgAMQ2FIQF6CZxvH9TaB4kHQlkbpPnfZ33Qgh8.jar' is inaccessible.
Dec 31, 2021 1:09:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-31T13:09:45.271Z: Staged package proto-google-cloud-bigtable-v2-2.2.0-eQuZNGvDK3rktI8lIFj3RceyFRnCQZQ7kzsVL_jefpk.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/proto-google-cloud-bigtable-v2-2.2.0-eQuZNGvDK3rktI8lIFj3RceyFRnCQZQ7kzsVL_jefpk.jar' is inaccessible.
Dec 31, 2021 1:09:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-31T13:09:45.315Z: Staged package proto-google-cloud-datastore-v1-0.92.3-GAooTTW0iEb04NomkkOPT5sN-SkixSjTSTIbKBgWRmU.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/proto-google-cloud-datastore-v1-0.92.3-GAooTTW0iEb04NomkkOPT5sN-SkixSjTSTIbKBgWRmU.jar' is inaccessible.
Dec 31, 2021 1:09:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-31T13:09:45.360Z: Staged package proto-google-cloud-firestore-bundle-v1-3.0.6-hMoeG-RDGv3XTTCFZdGmw_KrJY-8n-l4umeZu-mJuH0.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/proto-google-cloud-firestore-bundle-v1-3.0.6-hMoeG-RDGv3XTTCFZdGmw_KrJY-8n-l4umeZu-mJuH0.jar' is inaccessible.
Dec 31, 2021 1:09:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-31T13:09:45.394Z: Staged package proto-google-cloud-firestore-v1-3.0.6--VBOSAcR_FEKBkitnXBJzdARz49V4U2Go2hDVvYVyn8.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/proto-google-cloud-firestore-v1-3.0.6--VBOSAcR_FEKBkitnXBJzdARz49V4U2Go2hDVvYVyn8.jar' is inaccessible.
Dec 31, 2021 1:09:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-31T13:09:45.440Z: Staged package proto-google-cloud-pubsub-v1-1.96.7-ELakveR57_uj4HA8dVD0JE-uvJ2vQ5t62sVSbSQLqkA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/proto-google-cloud-pubsub-v1-1.96.7-ELakveR57_uj4HA8dVD0JE-uvJ2vQ5t62sVSbSQLqkA.jar' is inaccessible.
Dec 31, 2021 1:09:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-31T13:09:45.495Z: Staged package proto-google-cloud-spanner-admin-database-v1-6.14.0-svBhrFylwOEpV1RV3tyxpohFIg7ZauQu7ILOheRZHyE.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/proto-google-cloud-spanner-admin-database-v1-6.14.0-svBhrFylwOEpV1RV3tyxpohFIg7ZauQu7ILOheRZHyE.jar' is inaccessible.
Dec 31, 2021 1:09:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-31T13:09:45.540Z: Staged package proto-google-cloud-spanner-admin-instance-v1-6.14.0-6ZF4J-m2Y8168Yzax45VkVJ1tPALrotea0sjlS7mLsc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/proto-google-cloud-spanner-admin-instance-v1-6.14.0-6ZF4J-m2Y8168Yzax45VkVJ1tPALrotea0sjlS7mLsc.jar' is inaccessible.
Dec 31, 2021 1:09:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-31T13:09:45.579Z: Staged package proto-google-cloud-spanner-v1-6.14.0-ygyOMRjsWchGubRL9kRWduJyW5sn-6MSNv_IbwPn6Bo.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/proto-google-cloud-spanner-v1-6.14.0-ygyOMRjsWchGubRL9kRWduJyW5sn-6MSNv_IbwPn6Bo.jar' is inaccessible.
Dec 31, 2021 1:09:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-31T13:09:45.611Z: Staged package proto-google-cloud-storage-v2-2.0.1-alpha-hkn1Zre08VdXlAcFbsybY0a1fEj8QkQMcZ8KH3SE20Q.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/proto-google-cloud-storage-v2-2.0.1-alpha-hkn1Zre08VdXlAcFbsybY0a1fEj8QkQMcZ8KH3SE20Q.jar' is inaccessible.
Dec 31, 2021 1:09:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-31T13:09:45.648Z: Staged package proto-google-common-protos-2.6.0-6_Zu7qKrSyUfjKuUmLKZMDB5AvbypHYNyp9VWCtJCjM.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/proto-google-common-protos-2.6.0-6_Zu7qKrSyUfjKuUmLKZMDB5AvbypHYNyp9VWCtJCjM.jar' is inaccessible.
Dec 31, 2021 1:09:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-31T13:09:45.692Z: Staged package proto-google-iam-v1-1.1.6-RDXWTySL7P9f8jyZL9WO8aLT9sN8SWxcV_HoEQET4l8.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/proto-google-iam-v1-1.1.6-RDXWTySL7P9f8jyZL9WO8aLT9sN8SWxcV_HoEQET4l8.jar' is inaccessible.
Dec 31, 2021 1:09:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-31T13:09:45.737Z: Staged package protobuf-java-3.18.1-2wmd4uALeiExmzP6x8QPbYG6kTyyw7xBwbUatSrQreA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/protobuf-java-3.18.1-2wmd4uALeiExmzP6x8QPbYG6kTyyw7xBwbUatSrQreA.jar' is inaccessible.
Dec 31, 2021 1:09:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-31T13:09:45.782Z: Staged package protobuf-java-util-3.18.1-Cxo2gtdJGmkoEno6pqyzGm8EZhdIRfwbKVUcw4KeK_k.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/protobuf-java-util-3.18.1-Cxo2gtdJGmkoEno6pqyzGm8EZhdIRfwbKVUcw4KeK_k.jar' is inaccessible.
Dec 31, 2021 1:09:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-31T13:09:45.871Z: Staged package snakeyaml-1.28-NURqFCFDXUXkxqwN47U3hSfVzCRGwHGD4kRHcwzh__o.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/snakeyaml-1.28-NURqFCFDXUXkxqwN47U3hSfVzCRGwHGD4kRHcwzh__o.jar' is inaccessible.
Dec 31, 2021 1:09:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-12-31T13:09:46.033Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Dec 31, 2021 3:51:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-31T15:51:43.495Z: Staged package slf4j-api-1.7.30-zboHlk0btAoHYUhcax6ML4_Z6x0ZxTkorA1_lRAQXFc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/slf4j-api-1.7.30-zboHlk0btAoHYUhcax6ML4_Z6x0ZxTkorA1_lRAQXFc.jar' is inaccessible.
Dec 31, 2021 3:51:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-31T15:51:43.533Z: Staged package slf4j-jdk14-1.7.30-4PnbBJN49kZ5QXcUVJlSMyhft633LEZ-ZdryXmc6y6g.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/slf4j-jdk14-1.7.30-4PnbBJN49kZ5QXcUVJlSMyhft633LEZ-ZdryXmc6y6g.jar' is inaccessible.
Dec 31, 2021 3:51:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-31T15:51:43.679Z: Staged package xz-1.5-hvMPqHdfo6Ys2znR7XimAZFkwQWIZASNQsvuJE4m6EA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/xz-1.5-hvMPqHdfo6Ys2znR7XimAZFkwQWIZASNQsvuJE4m6EA.jar' is inaccessible.
Dec 31, 2021 3:51:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-12-31T15:51:43.768Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Dec 31, 2021 3:54:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-12-31T15:54:43.801Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Dec 31, 2021 3:57:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-31T15:57:43.595Z: Staged package slf4j-api-1.7.30-zboHlk0btAoHYUhcax6ML4_Z6x0ZxTkorA1_lRAQXFc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/slf4j-api-1.7.30-zboHlk0btAoHYUhcax6ML4_Z6x0ZxTkorA1_lRAQXFc.jar' is inaccessible.
Dec 31, 2021 3:57:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-31T15:57:43.631Z: Staged package slf4j-jdk14-1.7.30-4PnbBJN49kZ5QXcUVJlSMyhft633LEZ-ZdryXmc6y6g.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/slf4j-jdk14-1.7.30-4PnbBJN49kZ5QXcUVJlSMyhft633LEZ-ZdryXmc6y6g.jar' is inaccessible.
Dec 31, 2021 3:57:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-31T15:57:43.785Z: Staged package xz-1.5-hvMPqHdfo6Ys2znR7XimAZFkwQWIZASNQsvuJE4m6EA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/xz-1.5-hvMPqHdfo6Ys2znR7XimAZFkwQWIZASNQsvuJE4m6EA.jar' is inaccessible.
Dec 31, 2021 3:57:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-12-31T15:57:43.840Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Dec 31, 2021 4:00:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-31T16:00:35.892Z: Cancel request is committed for workflow job: 2021-12-31_04_45_26-9839654050190384555.
Dec 31, 2021 4:00:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-31T16:00:35.925Z: Cleaning up.
Dec 31, 2021 4:00:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-31T16:00:35.996Z: Stopping **** pool...
Dec 31, 2021 4:00:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-31T16:00:36.041Z: Stopping **** pool...
Dec 31, 2021 4:02:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-31T16:02:56.246Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Dec 31, 2021 4:02:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-31T16:02:56.280Z: Worker pool stopped.
Dec 31, 2021 4:03:02 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-12-31_04_45_26-9839654050190384555 finished with status CANCELLED.
Load test results for test (ID): f069d659-a75c-413e-ba89-1bcc4152f76d and timestamp: 2021-12-31T12:45:21.184000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11535.943
dataflow_v2_java11_total_bytes_count             2.26421956E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211231124330
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:1c0bb3cc854516926cb8435a412a0c55892191237abf60aa7984c1df29803623
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211231124330]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:1c0bb3cc854516926cb8435a412a0c55892191237abf60aa7984c1df29803623]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211231124330] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:1c0bb3cc854516926cb8435a412a0c55892191237abf60aa7984c1df29803623])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:1c0bb3cc854516926cb8435a412a0c55892191237abf60aa7984c1df29803623
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:1c0bb3cc854516926cb8435a412a0c55892191237abf60aa7984c1df29803623
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:1c0bb3cc854516926cb8435a412a0c55892191237abf60aa7984c1df29803623].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:fac13068a9736398cc394221d973238172e98fb2089fe8b887697d60acb48351
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:fac13068a9736398cc394221d973238172e98fb2089fe8b887697d60acb48351
ERROR: (gcloud.container.images.delete) Not found: response: {'status': '404', 'content-length': '168', 'x-xss-protection': '0', 'transfer-encoding': 'chunked', 'server': 'Docker Registry', '-content-encoding': 'gzip', 'docker-distribution-api-version': 'registry/2.0', 'cache-control': 'private', 'date': 'Fri, 31 Dec 2021 16:03:06 GMT', 'x-frame-options': 'SAMEORIGIN', 'content-type': 'application/json'}
Failed to compute blob liveness for manifest: 'sha256:fac13068a9736398cc394221d973238172e98fb2089fe8b887697d60acb48351': None

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages FAILED

FAILURE: Build completed with 2 failures.

1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
==============================================================================

2: Task failed with an exception.
-----------
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src/runners/google-cloud-dataflow-java/build.gradle'> line: 291

* What went wrong:
Execution failed for task ':runners:google-cloud-dataflow-java:cleanUpDockerImages'.
> Process 'command './scripts/cleanup_untagged_gcr_images.sh'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
==============================================================================

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings

Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.

BUILD FAILED in 3h 19m 48s
89 actionable tasks: 52 executed, 33 from cache, 4 up-to-date

Publishing build scan...
https://gradle.com/s/45cb27nhwkqom

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #196

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/196/display/redirect?page=changes>

Changes:

[Steve Niemitz] [BEAM-13488] Work around bug in classpath detection

[noreply] [BEAM-13578] Validate that repeatable PipelineOptions need to be

[noreply] [BEAM-13430] Update comment over eclipse plugin. (#16387)

[zyichi] [BEAM-13430] Fix broken ULR validates runner tests

[noreply] Update venv creation commands in release scripts. (#16381)

[Steve Niemitz] [BEAM-13459] Cache artifacts across job runs on python+dataflow

[noreply] [BEAM-12777] Stable URL for current API docs (#16327)

[noreply] [BEAM-13539] correct resource override dependencies (#16392)

[Kyle Weaver] [BEAM-13581] Remove previous job name for Flink PVR precommit.

[emilyye] Moving to 2.37.0-SNAPSHOT on master branch.

[noreply] Release website update for Beam 2.35.0 (#16115)

[noreply] [BEAM-13430] Re-enable checkerframework for the project excluding the


------------------------------------------
[...truncated 49.37 KB...]
66f373940980: Preparing
cfcce9963fe0: Preparing
c52059f2ee5d: Preparing
723c528c5f78: Preparing
5f2203c3e31a: Preparing
333f47bdfc73: Preparing
3bb5258f46d2: Preparing
832e177bb500: Preparing
f9e18e59a565: Preparing
26a504e63be4: Preparing
8bf42db0de72: Preparing
31892cc314cb: Preparing
66f373940980: Waiting
11936051f93b: Preparing
cfcce9963fe0: Waiting
723c528c5f78: Waiting
26a504e63be4: Waiting
5f2203c3e31a: Waiting
8bf42db0de72: Waiting
333f47bdfc73: Waiting
31892cc314cb: Waiting
3bb5258f46d2: Waiting
11936051f93b: Waiting
c7f603f0058e: Waiting
c52059f2ee5d: Waiting
f9e18e59a565: Waiting
4f8762388591: Pushed
1a7e19cf6e3b: Pushed
9e5d7637c93f: Pushed
96424d3eac6c: Pushed
c7f603f0058e: Pushed
00f54578a665: Pushed
cfcce9963fe0: Pushed
c52059f2ee5d: Pushed
5f2203c3e31a: Pushed
3bb5258f46d2: Layer already exists
f9e18e59a565: Layer already exists
832e177bb500: Layer already exists
66f373940980: Pushed
8bf42db0de72: Layer already exists
26a504e63be4: Layer already exists
31892cc314cb: Layer already exists
11936051f93b: Layer already exists
333f47bdfc73: Pushed
723c528c5f78: Pushed
20211230124337: digest: sha256:d33024c0898b0bb10b7ff51d23ee24caeec181b344c0d9cbd0e4f05a2db98020 size: 4311

> Task :sdks:java:testing:load-tests:run
Dec 30, 2021 12:45:34 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Dec 30, 2021 12:45:34 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 203 files. Enable logging at DEBUG level to see which files will be staged.
Dec 30, 2021 12:45:35 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Dec 30, 2021 12:45:35 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Dec 30, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 203 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Dec 30, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 203 files cached, 0 files newly uploaded in 0 seconds
Dec 30, 2021 12:45:39 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Dec 30, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <113691 bytes, hash cf758e32b63019061a6d96d4d81202a9d026b7716ad88eb773fbd518ec0c0e5d> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-z3WOMrYwGQYabZbU2BICqdAmt3Fq2I63c_vVGOwMDl0.pb
Dec 30, 2021 12:45:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Dec 30, 2021 12:45:41 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@cda6019, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@797c3c3b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4012d5bc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4375b013, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1cf0cacc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4f5b08d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@529c2a9a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c87fdf2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26bbe604, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@fe34b86, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c98781a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3f736a16, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4601203a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@53abfc07, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2c8c16c0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@80bfa9d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@47c40b56, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4b039c6d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7f5b9db, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@507d64aa]
Dec 30, 2021 12:45:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Dec 30, 2021 12:45:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Dec 30, 2021 12:45:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Dec 30, 2021 12:45:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Dec 30, 2021 12:45:41 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@34aeacd1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@54067fdc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4098dd77, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43aeb5e0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2274160, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@65383667, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@63cd2cd2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@557a84fe, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6deee370, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@49c17ba4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43e065f2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@423c5404, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5853ca50, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1a0d96a5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5a02bfe3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7a3e5cd3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c79088e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4a37191a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5854a18, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@d5556bf]
Dec 30, 2021 12:45:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Dec 30, 2021 12:45:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Dec 30, 2021 12:45:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Dec 30, 2021 12:45:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Dec 30, 2021 12:45:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Dec 30, 2021 12:45:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Dec 30, 2021 12:45:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Dec 30, 2021 12:45:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Dec 30, 2021 12:45:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Dec 30, 2021 12:45:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Dec 30, 2021 12:45:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Dec 30, 2021 12:45:41 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.37.0-SNAPSHOT
Dec 30, 2021 12:45:42 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-12-30_04_45_41-14461742849939712139?project=apache-beam-testing
Dec 30, 2021 12:45:42 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-12-30_04_45_41-14461742849939712139
Dec 30, 2021 12:45:42 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-12-30_04_45_41-14461742849939712139
Dec 30, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-12-30T12:45:49.203Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-12-f8zw. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Dec 30, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-30T12:45:54.834Z: Worker configuration: e2-standard-2 in us-central1-f.
Dec 30, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-30T12:45:55.476Z: Expanding SplittableParDo operations into optimizable parts.
Dec 30, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-30T12:45:55.509Z: Expanding CollectionToSingleton operations into optimizable parts.
Dec 30, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-30T12:45:55.579Z: Expanding CoGroupByKey operations into optimizable parts.
Dec 30, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-30T12:45:55.670Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Dec 30, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-30T12:45:55.701Z: Expanding GroupByKey operations into streaming Read/Write steps
Dec 30, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-30T12:45:55.762Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Dec 30, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-30T12:45:55.873Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Dec 30, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-30T12:45:55.907Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Dec 30, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-30T12:45:55.941Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Dec 30, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-30T12:45:55.983Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Dec 30, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-30T12:45:56.010Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Dec 30, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-30T12:45:56.033Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Dec 30, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-30T12:45:56.061Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Dec 30, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-30T12:45:56.088Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Dec 30, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-30T12:45:56.147Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Dec 30, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-30T12:45:56.181Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Dec 30, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-30T12:45:56.217Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Dec 30, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-30T12:45:56.265Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Dec 30, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-30T12:45:56.299Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Dec 30, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-30T12:45:56.323Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Dec 30, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-30T12:45:56.350Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Dec 30, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-30T12:45:56.384Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Dec 30, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-30T12:45:56.413Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Dec 30, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-30T12:45:56.438Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Dec 30, 2021 12:45:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-30T12:45:56.484Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Dec 30, 2021 12:45:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-30T12:45:56.545Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Dec 30, 2021 12:45:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-30T12:45:56.666Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Dec 30, 2021 12:45:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-30T12:45:56.727Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Dec 30, 2021 12:45:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-30T12:45:56.805Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Dec 30, 2021 12:45:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-30T12:45:57.426Z: Starting 5 ****s in us-central1-f...
Dec 30, 2021 12:46:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-30T12:46:07.696Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Dec 30, 2021 12:46:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-30T12:46:47.057Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Dec 30, 2021 12:47:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-30T12:47:41.179Z: Workers have started successfully.
Dec 30, 2021 12:47:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-30T12:47:41.215Z: Workers have started successfully.
Dec 30, 2021 4:00:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-30T16:00:39.363Z: Cancel request is committed for workflow job: 2021-12-30_04_45_41-14461742849939712139.
Dec 30, 2021 4:00:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-30T16:00:39.444Z: Cleaning up.
Dec 30, 2021 4:00:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-30T16:00:39.512Z: Stopping **** pool...
Dec 30, 2021 4:00:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-30T16:00:39.556Z: Stopping **** pool...
Dec 30, 2021 4:03:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-30T16:03:05.380Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Dec 30, 2021 4:03:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-30T16:03:05.407Z: Worker pool stopped.
Dec 30, 2021 4:03:10 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-12-30_04_45_41-14461742849939712139 finished with status CANCELLED.
Load test results for test (ID): ec339e52-3ad6-427f-a5f4-330f277c2c74 and timestamp: 2021-12-30T12:45:34.941000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11514.611
dataflow_v2_java11_total_bytes_count             2.58986545E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211230124337
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:d33024c0898b0bb10b7ff51d23ee24caeec181b344c0d9cbd0e4f05a2db98020
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211230124337]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:d33024c0898b0bb10b7ff51d23ee24caeec181b344c0d9cbd0e4f05a2db98020]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211230124337] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:d33024c0898b0bb10b7ff51d23ee24caeec181b344c0d9cbd0e4f05a2db98020])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:d33024c0898b0bb10b7ff51d23ee24caeec181b344c0d9cbd0e4f05a2db98020
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:d33024c0898b0bb10b7ff51d23ee24caeec181b344c0d9cbd0e4f05a2db98020
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:d33024c0898b0bb10b7ff51d23ee24caeec181b344c0d9cbd0e4f05a2db98020].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings

Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.

BUILD FAILED in 3h 19m 51s
109 actionable tasks: 72 executed, 33 from cache, 4 up-to-date

Publishing build scan...
https://gradle.com/s/7efeevipuddh4

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #195

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/195/display/redirect?page=changes>

Changes:

[dpcollins] [BEAM-13402] Simplify PubsubLiteSink

[Kyle Weaver] [BEAM-13521] flink pvr batch precommit

[Kyle Weaver] [BEAM-13571] Fix ClassNotFound exception in Flink tests

[Kyle Weaver] [BEAM-13498] [BEAM-13573] exclude new tests on Flink

[noreply] Exclude UsesOnWindowExpiration by category from Dataflow v2 streaming

[noreply] [BEAM-13052] Increment pubsub python version and fix breakages. (#16126)

[noreply] [BEAM-13052] Add Pub/Sub Lite xlang transforms in python (#15727)

[noreply] [BEAM-13402] Version bump Pub/Sub Lite and implement changes to ensure


------------------------------------------
[...truncated 44.10 KB...]
26a504e63be4: Preparing
8bf42db0de72: Preparing
31892cc314cb: Preparing
11936051f93b: Preparing
e55d55f036ac: Waiting
26a504e63be4: Waiting
e4aada6129fc: Waiting
3bb5258f46d2: Waiting
7d14dd19b753: Waiting
832e177bb500: Waiting
79b056163f51: Waiting
8bf42db0de72: Waiting
31892cc314cb: Waiting
11936051f93b: Waiting
f9e18e59a565: Waiting
e27740273bd6: Waiting
eb9331e0c24e: Waiting
cf874dadde9b: Waiting
5b1831a6a4ca: Pushed
e5c2291c5cbf: Pushed
4bcb9b02dcdc: Pushed
e55d55f036ac: Pushed
e548b597699e: Pushed
18a9e383e81f: Pushed
7d14dd19b753: Pushed
79b056163f51: Pushed
3bb5258f46d2: Layer already exists
832e177bb500: Layer already exists
e27740273bd6: Pushed
e4aada6129fc: Pushed
f9e18e59a565: Layer already exists
26a504e63be4: Layer already exists
31892cc314cb: Layer already exists
11936051f93b: Layer already exists
8bf42db0de72: Layer already exists
eb9331e0c24e: Pushed
cf874dadde9b: Pushed
20211229124330: digest: sha256:c20b7d271e664dd827cac2faf0eb922ad88f35f9577ebcc865ed6598b9eba905 size: 4311

> Task :sdks:java:testing:load-tests:run
Dec 29, 2021 12:45:22 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Dec 29, 2021 12:45:22 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 203 files. Enable logging at DEBUG level to see which files will be staged.
Dec 29, 2021 12:45:23 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Dec 29, 2021 12:45:23 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Dec 29, 2021 12:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 203 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Dec 29, 2021 12:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 203 files cached, 0 files newly uploaded in 0 seconds
Dec 29, 2021 12:45:25 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Dec 29, 2021 12:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <113691 bytes, hash 9646933fb259160f2eb74f74e47729f04be85b28fb01fd9eaec765609d1144be> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-lkaTP7JZFg8ut0905Hcp8EvoWyj7Af2ersdlYJ0RRL4.pb
Dec 29, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Dec 29, 2021 12:45:27 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5793b87, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12704e15, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@512575e9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6f1a16fe, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2373ad99, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@33634f04, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4993febc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7601bc96, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@48a0c8aa, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6192a5d5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3722c145, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4cbc2e3b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2975a9e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@765ffb14, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@57562473, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7a360554, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@424de326, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4bc33720, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2dd0f797, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@67064bdc]
Dec 29, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Dec 29, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Dec 29, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Dec 29, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Dec 29, 2021 12:45:27 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@65d8dff8, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@444f44c5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@303f1234, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@24d61e4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2149594a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1f1e58ca, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@57f847af, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ed34ef5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@553bc36c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@380e1909, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2d5ef498, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@95eb320, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2f521c4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4afbb6c2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@10db6131, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6c6017b9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4730e0f0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@506a1372, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1b7332a7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77c233af]
Dec 29, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Dec 29, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Dec 29, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Dec 29, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Dec 29, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Dec 29, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Dec 29, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Dec 29, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Dec 29, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Dec 29, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Dec 29, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Dec 29, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.36.0-SNAPSHOT
Dec 29, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-12-29_04_45_27-6039810460170652176?project=apache-beam-testing
Dec 29, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-12-29_04_45_27-6039810460170652176
Dec 29, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-12-29_04_45_27-6039810460170652176
Dec 29, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-12-29T12:45:36.200Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-12-j1t0. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Dec 29, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-29T12:45:41.122Z: Worker configuration: e2-standard-2 in us-central1-f.
Dec 29, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-29T12:45:41.607Z: Expanding SplittableParDo operations into optimizable parts.
Dec 29, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-29T12:45:41.628Z: Expanding CollectionToSingleton operations into optimizable parts.
Dec 29, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-29T12:45:41.693Z: Expanding CoGroupByKey operations into optimizable parts.
Dec 29, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-29T12:45:41.794Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Dec 29, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-29T12:45:41.828Z: Expanding GroupByKey operations into streaming Read/Write steps
Dec 29, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-29T12:45:41.927Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Dec 29, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-29T12:45:42.028Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Dec 29, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-29T12:45:42.055Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Dec 29, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-29T12:45:42.086Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Dec 29, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-29T12:45:42.118Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Dec 29, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-29T12:45:42.145Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Dec 29, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-29T12:45:42.176Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Dec 29, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-29T12:45:42.201Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Dec 29, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-29T12:45:42.238Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Dec 29, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-29T12:45:42.267Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Dec 29, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-29T12:45:42.299Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Dec 29, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-29T12:45:42.326Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Dec 29, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-29T12:45:42.353Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Dec 29, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-29T12:45:42.384Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Dec 29, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-29T12:45:42.416Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Dec 29, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-29T12:45:42.438Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Dec 29, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-29T12:45:42.467Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Dec 29, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-29T12:45:42.500Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Dec 29, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-29T12:45:42.526Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Dec 29, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-29T12:45:42.570Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Dec 29, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-29T12:45:42.605Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Dec 29, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-29T12:45:42.638Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Dec 29, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-29T12:45:42.672Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Dec 29, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-29T12:45:42.703Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Dec 29, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-29T12:45:43.115Z: Starting 5 ****s in us-central1-f...
Dec 29, 2021 12:46:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-29T12:46:16.383Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Dec 29, 2021 12:46:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-29T12:46:29.492Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Dec 29, 2021 12:47:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-29T12:47:23.181Z: Workers have started successfully.
Dec 29, 2021 12:47:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-29T12:47:23.209Z: Workers have started successfully.
Dec 29, 2021 1:21:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-29T13:21:44.471Z: Staged package gcsio-2.2.4-p9i4-xCmDBUow6YptIUTZYGtZZhNat4tocR78u1Q8kQ.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/gcsio-2.2.4-p9i4-xCmDBUow6YptIUTZYGtZZhNat4tocR78u1Q8kQ.jar' is inaccessible.
Dec 29, 2021 1:21:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-29T13:21:44.679Z: Staged package google-api-services-iamcredentials-v1-rev20210326-1.32.1-rzLlTs22sMZOFz3yEOfAB70ZZw2Gl99hr42n_k-f8-M.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-api-services-iamcredentials-v1-rev20210326-1.32.1-rzLlTs22sMZOFz3yEOfAB70ZZw2Gl99hr42n_k-f8-M.jar' is inaccessible.
Dec 29, 2021 1:21:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-29T13:21:46.855Z: Staged package util-2.2.4-r-7-okFosTGJSr5au-fNqmC_EgCN_DyG3X-dgI2RZFA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/util-2.2.4-r-7-okFosTGJSr5au-fNqmC_EgCN_DyG3X-dgI2RZFA.jar' is inaccessible.
Dec 29, 2021 1:21:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-12-29T13:21:46.924Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Dec 29, 2021 4:00:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-29T16:00:37.050Z: Cancel request is committed for workflow job: 2021-12-29_04_45_27-6039810460170652176.
Dec 29, 2021 4:00:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-29T16:00:37.110Z: Cleaning up.
Dec 29, 2021 4:00:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-29T16:00:37.237Z: Stopping **** pool...
Dec 29, 2021 4:00:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-29T16:00:37.443Z: Stopping **** pool...
Dec 29, 2021 4:03:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-29T16:02:59.333Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Dec 29, 2021 4:03:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-29T16:02:59.363Z: Worker pool stopped.
Dec 29, 2021 4:03:05 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-12-29_04_45_27-6039810460170652176 finished with status CANCELLED.
Load test results for test (ID): 5a1105b4-d15f-429f-89c2-53954f8eddf1 and timestamp: 2021-12-29T12:45:22.700000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11556.931
dataflow_v2_java11_total_bytes_count              2.1040439E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211229124330
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:c20b7d271e664dd827cac2faf0eb922ad88f35f9577ebcc865ed6598b9eba905
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211229124330]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:c20b7d271e664dd827cac2faf0eb922ad88f35f9577ebcc865ed6598b9eba905]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211229124330] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:c20b7d271e664dd827cac2faf0eb922ad88f35f9577ebcc865ed6598b9eba905])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:c20b7d271e664dd827cac2faf0eb922ad88f35f9577ebcc865ed6598b9eba905
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:c20b7d271e664dd827cac2faf0eb922ad88f35f9577ebcc865ed6598b9eba905
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:c20b7d271e664dd827cac2faf0eb922ad88f35f9577ebcc865ed6598b9eba905].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings

Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.

BUILD FAILED in 3h 19m 51s
89 actionable tasks: 52 executed, 33 from cache, 4 up-to-date

Publishing build scan...
https://gradle.com/s/zc2qlpvipgvqk

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #194

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/194/display/redirect?page=changes>

Changes:

[mmack] [adhoc] Forbid to import guava and others from org.testcontainers.shaded

[noreply] [BEAM-13526] Kafka.IO: make DeserializerProvider a public interface


------------------------------------------
[...truncated 44.10 KB...]
31892cc314cb: Preparing
11936051f93b: Preparing
4439fb655b0f: Waiting
da4ad5cd1a53: Waiting
3bb5258f46d2: Waiting
6f9b4de1510e: Waiting
832e177bb500: Waiting
b14acabcfd09: Waiting
11936051f93b: Waiting
f9e18e59a565: Waiting
31892cc314cb: Waiting
878ce1a685b1: Waiting
26a504e63be4: Waiting
79eca828d636: Waiting
c184ca102ae9: Waiting
8bf42db0de72: Waiting
d9f705c4b2c9: Pushed
212bb41a9a5b: Pushed
7f8966cf78fa: Pushed
658f4c3ab6dd: Pushed
b8f45c3b1d13: Pushed
878ce1a685b1: Pushed
c184ca102ae9: Pushed
da4ad5cd1a53: Pushed
3bb5258f46d2: Layer already exists
832e177bb500: Layer already exists
f9e18e59a565: Layer already exists
b14acabcfd09: Pushed
26a504e63be4: Layer already exists
4439fb655b0f: Pushed
79eca828d636: Pushed
8bf42db0de72: Layer already exists
31892cc314cb: Layer already exists
11936051f93b: Layer already exists
6f9b4de1510e: Pushed
20211228124334: digest: sha256:1ba1d5f7db2351edc08f8180258d916daa3a65f6a21de03a25e5277e0ae63dca size: 4311

> Task :sdks:java:testing:load-tests:run
Dec 28, 2021 12:45:19 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Dec 28, 2021 12:45:19 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 203 files. Enable logging at DEBUG level to see which files will be staged.
Dec 28, 2021 12:45:20 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Dec 28, 2021 12:45:20 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Dec 28, 2021 12:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 203 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Dec 28, 2021 12:45:23 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 203 files cached, 0 files newly uploaded in 0 seconds
Dec 28, 2021 12:45:23 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Dec 28, 2021 12:45:23 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <113691 bytes, hash 040ff368759e4279492ca9a43a3c3796957af8dbf72392b4a3e1fbf8fca95824> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-BA_zaHWeQnlJLKmkOjw3lpV6-Nv3I5K0o-H7-PypWCQ.pb
Dec 28, 2021 12:45:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Dec 28, 2021 12:45:25 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@15405bd6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@352ed70d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@70730db, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5793b87, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12704e15, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@512575e9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6f1a16fe, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2373ad99, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@33634f04, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4993febc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7601bc96, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@48a0c8aa, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6192a5d5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3722c145, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4cbc2e3b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2975a9e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@765ffb14, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@57562473, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7a360554, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@424de326]
Dec 28, 2021 12:45:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Dec 28, 2021 12:45:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Dec 28, 2021 12:45:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Dec 28, 2021 12:45:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Dec 28, 2021 12:45:25 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@597f0937, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7ad1caa2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6b6b3572, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@65d8dff8, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@444f44c5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@303f1234, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@24d61e4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2149594a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1f1e58ca, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@57f847af, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ed34ef5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@553bc36c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@380e1909, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2d5ef498, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@95eb320, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2f521c4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4afbb6c2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@10db6131, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6c6017b9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4730e0f0]
Dec 28, 2021 12:45:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Dec 28, 2021 12:45:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Dec 28, 2021 12:45:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Dec 28, 2021 12:45:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Dec 28, 2021 12:45:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Dec 28, 2021 12:45:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Dec 28, 2021 12:45:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Dec 28, 2021 12:45:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Dec 28, 2021 12:45:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Dec 28, 2021 12:45:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Dec 28, 2021 12:45:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Dec 28, 2021 12:45:25 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.36.0-SNAPSHOT
Dec 28, 2021 12:45:26 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-12-28_04_45_25-2016289845667715822?project=apache-beam-testing
Dec 28, 2021 12:45:26 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-12-28_04_45_25-2016289845667715822
Dec 28, 2021 12:45:26 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-12-28_04_45_25-2016289845667715822
Dec 28, 2021 12:45:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-12-28T12:45:31.111Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-12-x0om. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Dec 28, 2021 12:45:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-28T12:45:34.994Z: Worker configuration: e2-standard-2 in us-central1-f.
Dec 28, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-28T12:45:35.767Z: Expanding SplittableParDo operations into optimizable parts.
Dec 28, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-28T12:45:35.799Z: Expanding CollectionToSingleton operations into optimizable parts.
Dec 28, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-28T12:45:35.898Z: Expanding CoGroupByKey operations into optimizable parts.
Dec 28, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-28T12:45:35.999Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Dec 28, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-28T12:45:36.025Z: Expanding GroupByKey operations into streaming Read/Write steps
Dec 28, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-28T12:45:36.078Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Dec 28, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-28T12:45:36.177Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Dec 28, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-28T12:45:36.216Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Dec 28, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-28T12:45:36.253Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Dec 28, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-28T12:45:36.280Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Dec 28, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-28T12:45:36.307Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Dec 28, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-28T12:45:36.333Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Dec 28, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-28T12:45:36.358Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Dec 28, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-28T12:45:36.382Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Dec 28, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-28T12:45:36.422Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Dec 28, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-28T12:45:36.455Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Dec 28, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-28T12:45:36.485Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Dec 28, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-28T12:45:36.514Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Dec 28, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-28T12:45:36.545Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Dec 28, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-28T12:45:36.579Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Dec 28, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-28T12:45:36.611Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Dec 28, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-28T12:45:36.650Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Dec 28, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-28T12:45:36.678Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Dec 28, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-28T12:45:36.705Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Dec 28, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-28T12:45:36.743Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Dec 28, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-28T12:45:36.766Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Dec 28, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-28T12:45:36.797Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Dec 28, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-28T12:45:36.824Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Dec 28, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-28T12:45:36.854Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Dec 28, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-28T12:45:37.190Z: Starting 5 ****s in us-central1-f...
Dec 28, 2021 12:46:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-28T12:46:03.533Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Dec 28, 2021 12:46:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-28T12:46:22.959Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Dec 28, 2021 12:47:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-28T12:47:17.701Z: Workers have started successfully.
Dec 28, 2021 12:47:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-28T12:47:17.725Z: Workers have started successfully.
Dec 28, 2021 3:03:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-28T15:03:38.140Z: Staged package checker-qual-3.10.0-pNyILKaqxJbTM4Hl5esGBMRUg7AEvD6sk2jxuxfLJRI.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/checker-qual-3.10.0-pNyILKaqxJbTM4Hl5esGBMRUg7AEvD6sk2jxuxfLJRI.jar' is inaccessible.
Dec 28, 2021 3:03:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-28T15:03:40.269Z: Staged package opencensus-contrib-grpc-util-0.28.0-uRaDRuavZZMwChvCfvdCVKofJAGYhZON2PuFK4d9VfA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/opencensus-contrib-grpc-util-0.28.0-uRaDRuavZZMwChvCfvdCVKofJAGYhZON2PuFK4d9VfA.jar' is inaccessible.
Dec 28, 2021 3:03:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-28T15:03:40.802Z: Staged package zstd-jni-1.4.5-2-lzQxwUtNCahuI7cYQRb8rC2FUB60p0MPfRhczhr0YFA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/zstd-jni-1.4.5-2-lzQxwUtNCahuI7cYQRb8rC2FUB60p0MPfRhczhr0YFA.jar' is inaccessible.
Dec 28, 2021 3:03:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-12-28T15:03:40.825Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Dec 28, 2021 3:06:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-12-28T15:06:40.710Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Dec 28, 2021 4:00:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-28T16:00:33.390Z: Cancel request is committed for workflow job: 2021-12-28_04_45_25-2016289845667715822.
Dec 28, 2021 4:00:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-28T16:00:33.439Z: Cleaning up.
Dec 28, 2021 4:00:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-28T16:00:33.522Z: Stopping **** pool...
Dec 28, 2021 4:00:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-28T16:00:33.606Z: Stopping **** pool...
Dec 28, 2021 4:03:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-28T16:03:03.745Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Dec 28, 2021 4:03:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-28T16:03:03.778Z: Worker pool stopped.
Dec 28, 2021 4:03:11 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-12-28_04_45_25-2016289845667715822 finished with status CANCELLED.
Load test results for test (ID): 848dbf29-1cf7-41da-ba3f-d12511425051 and timestamp: 2021-12-28T12:45:19.917000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11562.848
dataflow_v2_java11_total_bytes_count             2.31544738E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211228124334
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:1ba1d5f7db2351edc08f8180258d916daa3a65f6a21de03a25e5277e0ae63dca
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211228124334]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:1ba1d5f7db2351edc08f8180258d916daa3a65f6a21de03a25e5277e0ae63dca]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211228124334] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:1ba1d5f7db2351edc08f8180258d916daa3a65f6a21de03a25e5277e0ae63dca])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:1ba1d5f7db2351edc08f8180258d916daa3a65f6a21de03a25e5277e0ae63dca
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:1ba1d5f7db2351edc08f8180258d916daa3a65f6a21de03a25e5277e0ae63dca
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:1ba1d5f7db2351edc08f8180258d916daa3a65f6a21de03a25e5277e0ae63dca].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings

Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.

BUILD FAILED in 3h 19m 54s
89 actionable tasks: 52 executed, 33 from cache, 4 up-to-date

Publishing build scan...
https://gradle.com/s/aygfhxg5bb6ui

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #193

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/193/display/redirect?page=changes>

Changes:

[noreply] Merge pull request #16285 from [BEAM-13492][Playground]  Update backend


------------------------------------------
[...truncated 60.32 KB...]
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:688
Dec 27, 2021 3:04:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-27T15:04:35.678Z: Staged package common-config-5.3.2-B_HQvj7POWJFHkM1OEwdiS2Y6zStZi7OXoiw-BJN7Nc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/common-config-5.3.2-B_HQvj7POWJFHkM1OEwdiS2Y6zStZi7OXoiw-BJN7Nc.jar' is inaccessible.
Dec 27, 2021 3:04:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-27T15:04:35.818Z: Staged package common-utils-5.3.2-4Rm-xdfQw0Yp9EH6ym0CZyPloNIGMAJ4b5DO_bByg9c.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/common-utils-5.3.2-4Rm-xdfQw0Yp9EH6ym0CZyPloNIGMAJ4b5DO_bByg9c.jar' is inaccessible.
Dec 27, 2021 3:04:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-27T15:04:38.028Z: Staged package kafka-avro-serializer-5.3.2-7FAoV2nngjZLbCXD3IPBOus_R_2KrM_zffAaNlVOls4.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/kafka-avro-serializer-5.3.2-7FAoV2nngjZLbCXD3IPBOus_R_2KrM_zffAaNlVOls4.jar' is inaccessible.
Dec 27, 2021 3:04:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-27T15:04:38.093Z: Staged package kafka-schema-registry-client-5.3.2-0mGQmXEF1E2hwRsR-O-mddgC_DwcNIU4Ba9SQcw2hKY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/kafka-schema-registry-client-5.3.2-0mGQmXEF1E2hwRsR-O-mddgC_DwcNIU4Ba9SQcw2hKY.jar' is inaccessible.
Dec 27, 2021 3:04:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-27T15:04:39.013Z: Staged package spring-core-4.3.18.RELEASE-qd-hKocv03STfOrJiihBIfzwVyBqE8q9UnPLPqutu2w.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/spring-core-4.3.18.RELEASE-qd-hKocv03STfOrJiihBIfzwVyBqE8q9UnPLPqutu2w.jar' is inaccessible.
Dec 27, 2021 3:04:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-27T15:04:39.060Z: Staged package spring-expression-4.3.18.RELEASE-aB3PS8pe5rtvJwR2BF3WrB6nO_rdEY2KpkFz7bel1_A.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/spring-expression-4.3.18.RELEASE-aB3PS8pe5rtvJwR2BF3WrB6nO_rdEY2KpkFz7bel1_A.jar' is inaccessible.
Dec 27, 2021 3:04:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-27T15:04:39.159Z: Staged package zkclient-0.10-JumIuLuoOMck_YNQszHui1_8WcOpwHTfEVxMOmyEOHg.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/zkclient-0.10-JumIuLuoOMck_YNQszHui1_8WcOpwHTfEVxMOmyEOHg.jar' is inaccessible.
Dec 27, 2021 3:04:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-12-27T15:04:39.231Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Dec 27, 2021 3:07:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-12-27T15:07:38.416Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Dec 27, 2021 3:10:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-27T15:10:35.671Z: Staged package common-config-5.3.2-B_HQvj7POWJFHkM1OEwdiS2Y6zStZi7OXoiw-BJN7Nc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/common-config-5.3.2-B_HQvj7POWJFHkM1OEwdiS2Y6zStZi7OXoiw-BJN7Nc.jar' is inaccessible.
Dec 27, 2021 3:10:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-27T15:10:35.724Z: Staged package common-utils-5.3.2-4Rm-xdfQw0Yp9EH6ym0CZyPloNIGMAJ4b5DO_bByg9c.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/common-utils-5.3.2-4Rm-xdfQw0Yp9EH6ym0CZyPloNIGMAJ4b5DO_bByg9c.jar' is inaccessible.
Dec 27, 2021 3:10:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-27T15:10:37.765Z: Staged package kafka-avro-serializer-5.3.2-7FAoV2nngjZLbCXD3IPBOus_R_2KrM_zffAaNlVOls4.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/kafka-avro-serializer-5.3.2-7FAoV2nngjZLbCXD3IPBOus_R_2KrM_zffAaNlVOls4.jar' is inaccessible.
Dec 27, 2021 3:10:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-27T15:10:37.841Z: Staged package kafka-schema-registry-client-5.3.2-0mGQmXEF1E2hwRsR-O-mddgC_DwcNIU4Ba9SQcw2hKY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/kafka-schema-registry-client-5.3.2-0mGQmXEF1E2hwRsR-O-mddgC_DwcNIU4Ba9SQcw2hKY.jar' is inaccessible.
Dec 27, 2021 3:10:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-27T15:10:38.704Z: Staged package spring-core-4.3.18.RELEASE-qd-hKocv03STfOrJiihBIfzwVyBqE8q9UnPLPqutu2w.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/spring-core-4.3.18.RELEASE-qd-hKocv03STfOrJiihBIfzwVyBqE8q9UnPLPqutu2w.jar' is inaccessible.
Dec 27, 2021 3:10:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-27T15:10:38.744Z: Staged package spring-expression-4.3.18.RELEASE-aB3PS8pe5rtvJwR2BF3WrB6nO_rdEY2KpkFz7bel1_A.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/spring-expression-4.3.18.RELEASE-aB3PS8pe5rtvJwR2BF3WrB6nO_rdEY2KpkFz7bel1_A.jar' is inaccessible.
Dec 27, 2021 3:10:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-27T15:10:38.849Z: Staged package zkclient-0.10-JumIuLuoOMck_YNQszHui1_8WcOpwHTfEVxMOmyEOHg.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/zkclient-0.10-JumIuLuoOMck_YNQszHui1_8WcOpwHTfEVxMOmyEOHg.jar' is inaccessible.
Dec 27, 2021 3:10:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-12-27T15:10:38.891Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Dec 27, 2021 3:13:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-12-27T15:13:38.675Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Dec 27, 2021 3:16:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-27T15:16:35.576Z: Staged package common-config-5.3.2-B_HQvj7POWJFHkM1OEwdiS2Y6zStZi7OXoiw-BJN7Nc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/common-config-5.3.2-B_HQvj7POWJFHkM1OEwdiS2Y6zStZi7OXoiw-BJN7Nc.jar' is inaccessible.
Dec 27, 2021 3:16:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-27T15:16:35.617Z: Staged package common-utils-5.3.2-4Rm-xdfQw0Yp9EH6ym0CZyPloNIGMAJ4b5DO_bByg9c.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/common-utils-5.3.2-4Rm-xdfQw0Yp9EH6ym0CZyPloNIGMAJ4b5DO_bByg9c.jar' is inaccessible.
Dec 27, 2021 3:16:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-27T15:16:37.623Z: Staged package kafka-avro-serializer-5.3.2-7FAoV2nngjZLbCXD3IPBOus_R_2KrM_zffAaNlVOls4.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/kafka-avro-serializer-5.3.2-7FAoV2nngjZLbCXD3IPBOus_R_2KrM_zffAaNlVOls4.jar' is inaccessible.
Dec 27, 2021 3:16:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-27T15:16:37.697Z: Staged package kafka-schema-registry-client-5.3.2-0mGQmXEF1E2hwRsR-O-mddgC_DwcNIU4Ba9SQcw2hKY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/kafka-schema-registry-client-5.3.2-0mGQmXEF1E2hwRsR-O-mddgC_DwcNIU4Ba9SQcw2hKY.jar' is inaccessible.
Dec 27, 2021 3:16:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-27T15:16:38.684Z: Staged package spring-core-4.3.18.RELEASE-qd-hKocv03STfOrJiihBIfzwVyBqE8q9UnPLPqutu2w.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/spring-core-4.3.18.RELEASE-qd-hKocv03STfOrJiihBIfzwVyBqE8q9UnPLPqutu2w.jar' is inaccessible.
Dec 27, 2021 3:16:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-27T15:16:38.741Z: Staged package spring-expression-4.3.18.RELEASE-aB3PS8pe5rtvJwR2BF3WrB6nO_rdEY2KpkFz7bel1_A.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/spring-expression-4.3.18.RELEASE-aB3PS8pe5rtvJwR2BF3WrB6nO_rdEY2KpkFz7bel1_A.jar' is inaccessible.
Dec 27, 2021 3:16:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-27T15:16:38.828Z: Staged package zkclient-0.10-JumIuLuoOMck_YNQszHui1_8WcOpwHTfEVxMOmyEOHg.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/zkclient-0.10-JumIuLuoOMck_YNQszHui1_8WcOpwHTfEVxMOmyEOHg.jar' is inaccessible.
Dec 27, 2021 3:16:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-12-27T15:16:38.869Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Dec 27, 2021 3:19:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-12-27T15:19:38.685Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Dec 27, 2021 3:22:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-27T15:22:35.739Z: Staged package common-config-5.3.2-B_HQvj7POWJFHkM1OEwdiS2Y6zStZi7OXoiw-BJN7Nc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/common-config-5.3.2-B_HQvj7POWJFHkM1OEwdiS2Y6zStZi7OXoiw-BJN7Nc.jar' is inaccessible.
Dec 27, 2021 3:22:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-27T15:22:35.822Z: Staged package common-utils-5.3.2-4Rm-xdfQw0Yp9EH6ym0CZyPloNIGMAJ4b5DO_bByg9c.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/common-utils-5.3.2-4Rm-xdfQw0Yp9EH6ym0CZyPloNIGMAJ4b5DO_bByg9c.jar' is inaccessible.
Dec 27, 2021 3:22:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-27T15:22:38.204Z: Staged package kafka-avro-serializer-5.3.2-7FAoV2nngjZLbCXD3IPBOus_R_2KrM_zffAaNlVOls4.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/kafka-avro-serializer-5.3.2-7FAoV2nngjZLbCXD3IPBOus_R_2KrM_zffAaNlVOls4.jar' is inaccessible.
Dec 27, 2021 3:22:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-27T15:22:38.255Z: Staged package kafka-schema-registry-client-5.3.2-0mGQmXEF1E2hwRsR-O-mddgC_DwcNIU4Ba9SQcw2hKY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/kafka-schema-registry-client-5.3.2-0mGQmXEF1E2hwRsR-O-mddgC_DwcNIU4Ba9SQcw2hKY.jar' is inaccessible.
Dec 27, 2021 3:22:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-27T15:22:39.386Z: Staged package spring-core-4.3.18.RELEASE-qd-hKocv03STfOrJiihBIfzwVyBqE8q9UnPLPqutu2w.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/spring-core-4.3.18.RELEASE-qd-hKocv03STfOrJiihBIfzwVyBqE8q9UnPLPqutu2w.jar' is inaccessible.
Dec 27, 2021 3:22:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-27T15:22:39.421Z: Staged package spring-expression-4.3.18.RELEASE-aB3PS8pe5rtvJwR2BF3WrB6nO_rdEY2KpkFz7bel1_A.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/spring-expression-4.3.18.RELEASE-aB3PS8pe5rtvJwR2BF3WrB6nO_rdEY2KpkFz7bel1_A.jar' is inaccessible.
Dec 27, 2021 3:22:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-27T15:22:39.511Z: Staged package zkclient-0.10-JumIuLuoOMck_YNQszHui1_8WcOpwHTfEVxMOmyEOHg.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/zkclient-0.10-JumIuLuoOMck_YNQszHui1_8WcOpwHTfEVxMOmyEOHg.jar' is inaccessible.
Dec 27, 2021 3:22:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-12-27T15:22:39.543Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Dec 27, 2021 3:25:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-12-27T15:25:38.844Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Dec 27, 2021 3:28:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-27T15:28:35.455Z: Staged package common-config-5.3.2-B_HQvj7POWJFHkM1OEwdiS2Y6zStZi7OXoiw-BJN7Nc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/common-config-5.3.2-B_HQvj7POWJFHkM1OEwdiS2Y6zStZi7OXoiw-BJN7Nc.jar' is inaccessible.
Dec 27, 2021 3:28:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-27T15:28:35.497Z: Staged package common-utils-5.3.2-4Rm-xdfQw0Yp9EH6ym0CZyPloNIGMAJ4b5DO_bByg9c.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/common-utils-5.3.2-4Rm-xdfQw0Yp9EH6ym0CZyPloNIGMAJ4b5DO_bByg9c.jar' is inaccessible.
Dec 27, 2021 3:28:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-27T15:28:37.484Z: Staged package kafka-avro-serializer-5.3.2-7FAoV2nngjZLbCXD3IPBOus_R_2KrM_zffAaNlVOls4.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/kafka-avro-serializer-5.3.2-7FAoV2nngjZLbCXD3IPBOus_R_2KrM_zffAaNlVOls4.jar' is inaccessible.
Dec 27, 2021 3:28:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-27T15:28:37.555Z: Staged package kafka-schema-registry-client-5.3.2-0mGQmXEF1E2hwRsR-O-mddgC_DwcNIU4Ba9SQcw2hKY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/kafka-schema-registry-client-5.3.2-0mGQmXEF1E2hwRsR-O-mddgC_DwcNIU4Ba9SQcw2hKY.jar' is inaccessible.
Dec 27, 2021 3:28:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-27T15:28:38.537Z: Staged package spring-core-4.3.18.RELEASE-qd-hKocv03STfOrJiihBIfzwVyBqE8q9UnPLPqutu2w.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/spring-core-4.3.18.RELEASE-qd-hKocv03STfOrJiihBIfzwVyBqE8q9UnPLPqutu2w.jar' is inaccessible.
Dec 27, 2021 3:28:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-27T15:28:38.573Z: Staged package spring-expression-4.3.18.RELEASE-aB3PS8pe5rtvJwR2BF3WrB6nO_rdEY2KpkFz7bel1_A.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/spring-expression-4.3.18.RELEASE-aB3PS8pe5rtvJwR2BF3WrB6nO_rdEY2KpkFz7bel1_A.jar' is inaccessible.
Dec 27, 2021 3:28:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-27T15:28:38.643Z: Staged package zkclient-0.10-JumIuLuoOMck_YNQszHui1_8WcOpwHTfEVxMOmyEOHg.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/zkclient-0.10-JumIuLuoOMck_YNQszHui1_8WcOpwHTfEVxMOmyEOHg.jar' is inaccessible.
Dec 27, 2021 3:28:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-12-27T15:28:38.681Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Dec 27, 2021 3:31:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-12-27T15:31:39.398Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Dec 27, 2021 3:34:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-27T15:34:35.462Z: Staged package common-config-5.3.2-B_HQvj7POWJFHkM1OEwdiS2Y6zStZi7OXoiw-BJN7Nc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/common-config-5.3.2-B_HQvj7POWJFHkM1OEwdiS2Y6zStZi7OXoiw-BJN7Nc.jar' is inaccessible.
Dec 27, 2021 3:34:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-27T15:34:35.491Z: Staged package common-utils-5.3.2-4Rm-xdfQw0Yp9EH6ym0CZyPloNIGMAJ4b5DO_bByg9c.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/common-utils-5.3.2-4Rm-xdfQw0Yp9EH6ym0CZyPloNIGMAJ4b5DO_bByg9c.jar' is inaccessible.
Dec 27, 2021 3:34:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-27T15:34:37.465Z: Staged package kafka-avro-serializer-5.3.2-7FAoV2nngjZLbCXD3IPBOus_R_2KrM_zffAaNlVOls4.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/kafka-avro-serializer-5.3.2-7FAoV2nngjZLbCXD3IPBOus_R_2KrM_zffAaNlVOls4.jar' is inaccessible.
Dec 27, 2021 3:34:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-27T15:34:37.516Z: Staged package kafka-schema-registry-client-5.3.2-0mGQmXEF1E2hwRsR-O-mddgC_DwcNIU4Ba9SQcw2hKY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/kafka-schema-registry-client-5.3.2-0mGQmXEF1E2hwRsR-O-mddgC_DwcNIU4Ba9SQcw2hKY.jar' is inaccessible.
Dec 27, 2021 3:34:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-27T15:34:38.363Z: Staged package spring-core-4.3.18.RELEASE-qd-hKocv03STfOrJiihBIfzwVyBqE8q9UnPLPqutu2w.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/spring-core-4.3.18.RELEASE-qd-hKocv03STfOrJiihBIfzwVyBqE8q9UnPLPqutu2w.jar' is inaccessible.
Dec 27, 2021 3:34:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-27T15:34:38.400Z: Staged package spring-expression-4.3.18.RELEASE-aB3PS8pe5rtvJwR2BF3WrB6nO_rdEY2KpkFz7bel1_A.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/spring-expression-4.3.18.RELEASE-aB3PS8pe5rtvJwR2BF3WrB6nO_rdEY2KpkFz7bel1_A.jar' is inaccessible.
Dec 27, 2021 3:34:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-27T15:34:38.494Z: Staged package zkclient-0.10-JumIuLuoOMck_YNQszHui1_8WcOpwHTfEVxMOmyEOHg.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/zkclient-0.10-JumIuLuoOMck_YNQszHui1_8WcOpwHTfEVxMOmyEOHg.jar' is inaccessible.
Dec 27, 2021 3:34:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-12-27T15:34:38.529Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Dec 27, 2021 3:37:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-12-27T15:37:38.771Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Dec 27, 2021 3:40:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-27T15:40:35.478Z: Staged package common-config-5.3.2-B_HQvj7POWJFHkM1OEwdiS2Y6zStZi7OXoiw-BJN7Nc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/common-config-5.3.2-B_HQvj7POWJFHkM1OEwdiS2Y6zStZi7OXoiw-BJN7Nc.jar' is inaccessible.
Dec 27, 2021 3:40:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-27T15:40:35.519Z: Staged package common-utils-5.3.2-4Rm-xdfQw0Yp9EH6ym0CZyPloNIGMAJ4b5DO_bByg9c.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/common-utils-5.3.2-4Rm-xdfQw0Yp9EH6ym0CZyPloNIGMAJ4b5DO_bByg9c.jar' is inaccessible.
Dec 27, 2021 3:40:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-27T15:40:37.361Z: Staged package kafka-avro-serializer-5.3.2-7FAoV2nngjZLbCXD3IPBOus_R_2KrM_zffAaNlVOls4.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/kafka-avro-serializer-5.3.2-7FAoV2nngjZLbCXD3IPBOus_R_2KrM_zffAaNlVOls4.jar' is inaccessible.
Dec 27, 2021 3:40:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-27T15:40:37.430Z: Staged package kafka-schema-registry-client-5.3.2-0mGQmXEF1E2hwRsR-O-mddgC_DwcNIU4Ba9SQcw2hKY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/kafka-schema-registry-client-5.3.2-0mGQmXEF1E2hwRsR-O-mddgC_DwcNIU4Ba9SQcw2hKY.jar' is inaccessible.
Dec 27, 2021 3:40:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-27T15:40:38.306Z: Staged package spring-core-4.3.18.RELEASE-qd-hKocv03STfOrJiihBIfzwVyBqE8q9UnPLPqutu2w.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/spring-core-4.3.18.RELEASE-qd-hKocv03STfOrJiihBIfzwVyBqE8q9UnPLPqutu2w.jar' is inaccessible.
Dec 27, 2021 3:40:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-27T15:40:38.352Z: Staged package spring-expression-4.3.18.RELEASE-aB3PS8pe5rtvJwR2BF3WrB6nO_rdEY2KpkFz7bel1_A.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/spring-expression-4.3.18.RELEASE-aB3PS8pe5rtvJwR2BF3WrB6nO_rdEY2KpkFz7bel1_A.jar' is inaccessible.
Dec 27, 2021 3:40:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-27T15:40:38.438Z: Staged package zkclient-0.10-JumIuLuoOMck_YNQszHui1_8WcOpwHTfEVxMOmyEOHg.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/zkclient-0.10-JumIuLuoOMck_YNQszHui1_8WcOpwHTfEVxMOmyEOHg.jar' is inaccessible.
Dec 27, 2021 3:40:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-12-27T15:40:38.466Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Dec 27, 2021 3:43:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-12-27T15:43:38.775Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Dec 27, 2021 3:46:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-27T15:46:35.621Z: Staged package common-config-5.3.2-B_HQvj7POWJFHkM1OEwdiS2Y6zStZi7OXoiw-BJN7Nc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/common-config-5.3.2-B_HQvj7POWJFHkM1OEwdiS2Y6zStZi7OXoiw-BJN7Nc.jar' is inaccessible.
Dec 27, 2021 3:46:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-27T15:46:35.674Z: Staged package common-utils-5.3.2-4Rm-xdfQw0Yp9EH6ym0CZyPloNIGMAJ4b5DO_bByg9c.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/common-utils-5.3.2-4Rm-xdfQw0Yp9EH6ym0CZyPloNIGMAJ4b5DO_bByg9c.jar' is inaccessible.
Dec 27, 2021 3:46:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-27T15:46:37.934Z: Staged package kafka-avro-serializer-5.3.2-7FAoV2nngjZLbCXD3IPBOus_R_2KrM_zffAaNlVOls4.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/kafka-avro-serializer-5.3.2-7FAoV2nngjZLbCXD3IPBOus_R_2KrM_zffAaNlVOls4.jar' is inaccessible.
Dec 27, 2021 3:46:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-27T15:46:38.001Z: Staged package kafka-schema-registry-client-5.3.2-0mGQmXEF1E2hwRsR-O-mddgC_DwcNIU4Ba9SQcw2hKY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/kafka-schema-registry-client-5.3.2-0mGQmXEF1E2hwRsR-O-mddgC_DwcNIU4Ba9SQcw2hKY.jar' is inaccessible.
Dec 27, 2021 3:46:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-27T15:46:38.928Z: Staged package spring-core-4.3.18.RELEASE-qd-hKocv03STfOrJiihBIfzwVyBqE8q9UnPLPqutu2w.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/spring-core-4.3.18.RELEASE-qd-hKocv03STfOrJiihBIfzwVyBqE8q9UnPLPqutu2w.jar' is inaccessible.
Dec 27, 2021 3:46:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-27T15:46:38.967Z: Staged package spring-expression-4.3.18.RELEASE-aB3PS8pe5rtvJwR2BF3WrB6nO_rdEY2KpkFz7bel1_A.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/spring-expression-4.3.18.RELEASE-aB3PS8pe5rtvJwR2BF3WrB6nO_rdEY2KpkFz7bel1_A.jar' is inaccessible.
Dec 27, 2021 3:46:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-27T15:46:39.057Z: Staged package zkclient-0.10-JumIuLuoOMck_YNQszHui1_8WcOpwHTfEVxMOmyEOHg.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/zkclient-0.10-JumIuLuoOMck_YNQszHui1_8WcOpwHTfEVxMOmyEOHg.jar' is inaccessible.
Dec 27, 2021 3:46:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-12-27T15:46:39.166Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Dec 27, 2021 3:49:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-12-27T15:49:39.128Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Dec 27, 2021 3:52:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-27T15:52:35.541Z: Staged package common-config-5.3.2-B_HQvj7POWJFHkM1OEwdiS2Y6zStZi7OXoiw-BJN7Nc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/common-config-5.3.2-B_HQvj7POWJFHkM1OEwdiS2Y6zStZi7OXoiw-BJN7Nc.jar' is inaccessible.
Dec 27, 2021 3:52:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-27T15:52:35.573Z: Staged package common-utils-5.3.2-4Rm-xdfQw0Yp9EH6ym0CZyPloNIGMAJ4b5DO_bByg9c.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/common-utils-5.3.2-4Rm-xdfQw0Yp9EH6ym0CZyPloNIGMAJ4b5DO_bByg9c.jar' is inaccessible.
Dec 27, 2021 3:52:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-27T15:52:37.493Z: Staged package kafka-avro-serializer-5.3.2-7FAoV2nngjZLbCXD3IPBOus_R_2KrM_zffAaNlVOls4.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/kafka-avro-serializer-5.3.2-7FAoV2nngjZLbCXD3IPBOus_R_2KrM_zffAaNlVOls4.jar' is inaccessible.
Dec 27, 2021 3:52:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-27T15:52:37.559Z: Staged package kafka-schema-registry-client-5.3.2-0mGQmXEF1E2hwRsR-O-mddgC_DwcNIU4Ba9SQcw2hKY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/kafka-schema-registry-client-5.3.2-0mGQmXEF1E2hwRsR-O-mddgC_DwcNIU4Ba9SQcw2hKY.jar' is inaccessible.
Dec 27, 2021 3:52:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-27T15:52:38.361Z: Staged package spring-core-4.3.18.RELEASE-qd-hKocv03STfOrJiihBIfzwVyBqE8q9UnPLPqutu2w.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/spring-core-4.3.18.RELEASE-qd-hKocv03STfOrJiihBIfzwVyBqE8q9UnPLPqutu2w.jar' is inaccessible.
Dec 27, 2021 3:52:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-27T15:52:38.410Z: Staged package spring-expression-4.3.18.RELEASE-aB3PS8pe5rtvJwR2BF3WrB6nO_rdEY2KpkFz7bel1_A.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/spring-expression-4.3.18.RELEASE-aB3PS8pe5rtvJwR2BF3WrB6nO_rdEY2KpkFz7bel1_A.jar' is inaccessible.
Dec 27, 2021 3:52:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-27T15:52:38.489Z: Staged package zkclient-0.10-JumIuLuoOMck_YNQszHui1_8WcOpwHTfEVxMOmyEOHg.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/zkclient-0.10-JumIuLuoOMck_YNQszHui1_8WcOpwHTfEVxMOmyEOHg.jar' is inaccessible.
Dec 27, 2021 3:52:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-12-27T15:52:38.551Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Dec 27, 2021 3:55:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-12-27T15:55:38.262Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Dec 27, 2021 3:58:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-27T15:58:35.562Z: Staged package common-config-5.3.2-B_HQvj7POWJFHkM1OEwdiS2Y6zStZi7OXoiw-BJN7Nc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/common-config-5.3.2-B_HQvj7POWJFHkM1OEwdiS2Y6zStZi7OXoiw-BJN7Nc.jar' is inaccessible.
Dec 27, 2021 3:58:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-27T15:58:35.614Z: Staged package common-utils-5.3.2-4Rm-xdfQw0Yp9EH6ym0CZyPloNIGMAJ4b5DO_bByg9c.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/common-utils-5.3.2-4Rm-xdfQw0Yp9EH6ym0CZyPloNIGMAJ4b5DO_bByg9c.jar' is inaccessible.
Dec 27, 2021 3:58:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-27T15:58:37.639Z: Staged package kafka-avro-serializer-5.3.2-7FAoV2nngjZLbCXD3IPBOus_R_2KrM_zffAaNlVOls4.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/kafka-avro-serializer-5.3.2-7FAoV2nngjZLbCXD3IPBOus_R_2KrM_zffAaNlVOls4.jar' is inaccessible.
Dec 27, 2021 3:58:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-27T15:58:37.693Z: Staged package kafka-schema-registry-client-5.3.2-0mGQmXEF1E2hwRsR-O-mddgC_DwcNIU4Ba9SQcw2hKY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/kafka-schema-registry-client-5.3.2-0mGQmXEF1E2hwRsR-O-mddgC_DwcNIU4Ba9SQcw2hKY.jar' is inaccessible.
Dec 27, 2021 3:58:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-27T15:58:38.511Z: Staged package spring-core-4.3.18.RELEASE-qd-hKocv03STfOrJiihBIfzwVyBqE8q9UnPLPqutu2w.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/spring-core-4.3.18.RELEASE-qd-hKocv03STfOrJiihBIfzwVyBqE8q9UnPLPqutu2w.jar' is inaccessible.
Dec 27, 2021 3:58:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-27T15:58:38.545Z: Staged package spring-expression-4.3.18.RELEASE-aB3PS8pe5rtvJwR2BF3WrB6nO_rdEY2KpkFz7bel1_A.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/spring-expression-4.3.18.RELEASE-aB3PS8pe5rtvJwR2BF3WrB6nO_rdEY2KpkFz7bel1_A.jar' is inaccessible.
Dec 27, 2021 3:58:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-27T15:58:38.646Z: Staged package zkclient-0.10-JumIuLuoOMck_YNQszHui1_8WcOpwHTfEVxMOmyEOHg.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/zkclient-0.10-JumIuLuoOMck_YNQszHui1_8WcOpwHTfEVxMOmyEOHg.jar' is inaccessible.
Dec 27, 2021 3:58:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-12-27T15:58:38.694Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Dec 27, 2021 4:00:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-27T16:00:35.833Z: Cancel request is committed for workflow job: 2021-12-27_04_46_14-6106271570807984149.
Dec 27, 2021 4:00:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-27T16:00:35.867Z: Cleaning up.
Dec 27, 2021 4:00:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-27T16:00:35.966Z: Stopping **** pool...
Dec 27, 2021 4:00:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-27T16:00:36.078Z: Stopping **** pool...
Dec 27, 2021 4:03:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-27T16:02:59.940Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Dec 27, 2021 4:03:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-27T16:02:59.979Z: Worker pool stopped.
Dec 27, 2021 4:03:06 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-12-27_04_46_14-6106271570807984149 finished with status CANCELLED.
Load test results for test (ID): 20b7f72c-923e-4d42-a367-f2ac1a1736c1 and timestamp: 2021-12-27T12:46:08.815000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11496.929
dataflow_v2_java11_total_bytes_count             2.32537889E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211227124415
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:12629fa35f20b9dc8a82682aa835377d27188fdced8f10d760bce57ac4fd4a4d
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211227124415]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:12629fa35f20b9dc8a82682aa835377d27188fdced8f10d760bce57ac4fd4a4d]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211227124415] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:12629fa35f20b9dc8a82682aa835377d27188fdced8f10d760bce57ac4fd4a4d])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:12629fa35f20b9dc8a82682aa835377d27188fdced8f10d760bce57ac4fd4a4d
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:12629fa35f20b9dc8a82682aa835377d27188fdced8f10d760bce57ac4fd4a4d
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:12629fa35f20b9dc8a82682aa835377d27188fdced8f10d760bce57ac4fd4a4d].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings

Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.

BUILD FAILED in 3h 19m 9s
89 actionable tasks: 52 executed, 33 from cache, 4 up-to-date

Publishing build scan...
https://gradle.com/s/vyuqn4royskj2

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure


---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #192

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/192/display/redirect?page=changes>

Changes:

[alexander.zhuravlev] [BEAM-13530] Fixed scrollbar in the Output section

[noreply] Merge pull request #16355 from [BEAM-13529] [Playground] [Bugfix] Fix

[noreply] Fixed grammar error in Go ParDo function comments. (#16329)


------------------------------------------
[...truncated 51.43 KB...]
Dec 26, 2021 12:45:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Dec 26, 2021 12:45:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Dec 26, 2021 12:45:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Dec 26, 2021 12:45:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Dec 26, 2021 12:45:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Dec 26, 2021 12:45:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Dec 26, 2021 12:45:34 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.36.0-SNAPSHOT
Dec 26, 2021 12:45:35 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-12-26_04_45_34-13218376411721923737?project=apache-beam-testing
Dec 26, 2021 12:45:35 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-12-26_04_45_34-13218376411721923737
Dec 26, 2021 12:45:35 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-12-26_04_45_34-13218376411721923737
Dec 26, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-12-26T12:45:42.044Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-12-bfhv. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Dec 26, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-26T12:45:50.252Z: Worker configuration: e2-standard-2 in us-central1-a.
Dec 26, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-26T12:45:50.946Z: Expanding SplittableParDo operations into optimizable parts.
Dec 26, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-26T12:45:50.976Z: Expanding CollectionToSingleton operations into optimizable parts.
Dec 26, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-26T12:45:51.041Z: Expanding CoGroupByKey operations into optimizable parts.
Dec 26, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-26T12:45:51.109Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Dec 26, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-26T12:45:51.131Z: Expanding GroupByKey operations into streaming Read/Write steps
Dec 26, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-26T12:45:51.189Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Dec 26, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-26T12:45:51.299Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Dec 26, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-26T12:45:51.330Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Dec 26, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-26T12:45:51.368Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Dec 26, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-26T12:45:51.438Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Dec 26, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-26T12:45:51.477Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Dec 26, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-26T12:45:51.507Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Dec 26, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-26T12:45:51.541Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Dec 26, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-26T12:45:51.573Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Dec 26, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-26T12:45:51.607Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Dec 26, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-26T12:45:51.649Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Dec 26, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-26T12:45:51.684Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Dec 26, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-26T12:45:51.731Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Dec 26, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-26T12:45:51.762Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Dec 26, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-26T12:45:51.796Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Dec 26, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-26T12:45:51.831Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Dec 26, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-26T12:45:51.864Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Dec 26, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-26T12:45:51.910Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Dec 26, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-26T12:45:51.943Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Dec 26, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-26T12:45:51.980Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Dec 26, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-26T12:45:52.015Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Dec 26, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-26T12:45:52.048Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Dec 26, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-26T12:45:52.078Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Dec 26, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-26T12:45:52.122Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Dec 26, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-26T12:45:52.534Z: Starting 5 ****s in us-central1-a...
Dec 26, 2021 12:46:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-26T12:45:59.495Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Dec 26, 2021 12:46:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-26T12:46:36.659Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Dec 26, 2021 12:47:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-26T12:47:32.761Z: Workers have started successfully.
Dec 26, 2021 12:47:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-26T12:47:32.781Z: Workers have started successfully.
Dec 26, 2021 1:15:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-26T13:15:53.947Z: Staged package failureaccess-1.0.1-oXHuTHNN0tqDfksWvp30Zhr6typBra8x64Tf2vk2yiY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/failureaccess-1.0.1-oXHuTHNN0tqDfksWvp30Zhr6typBra8x64Tf2vk2yiY.jar' is inaccessible.
Dec 26, 2021 1:15:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-26T13:15:55.668Z: Staged package listenablefuture-9999.0-empty-to-avoid-conflict-with-guava-s3KgN9QjCqV_vv_e8w_WEj-cDC24XQrO0AyRuXTzP5k.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava-s3KgN9QjCqV_vv_e8w_WEj-cDC24XQrO0AyRuXTzP5k.jar' is inaccessible.
Dec 26, 2021 1:15:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-26T13:15:55.998Z: Staged package netty-tcnative-boringssl-static-2.0.33.Final-q3CryMTeke7zEAdYTla2hvWe2cxEwrM38_6C8EAtOH0.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-tcnative-boringssl-static-2.0.33.Final-q3CryMTeke7zEAdYTla2hvWe2cxEwrM38_6C8EAtOH0.jar' is inaccessible.
Dec 26, 2021 1:15:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-12-26T13:15:56.654Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Dec 26, 2021 1:18:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-12-26T13:18:56.096Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Dec 26, 2021 1:21:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-26T13:21:53.873Z: Staged package failureaccess-1.0.1-oXHuTHNN0tqDfksWvp30Zhr6typBra8x64Tf2vk2yiY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/failureaccess-1.0.1-oXHuTHNN0tqDfksWvp30Zhr6typBra8x64Tf2vk2yiY.jar' is inaccessible.
Dec 26, 2021 1:21:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-26T13:21:55.537Z: Staged package listenablefuture-9999.0-empty-to-avoid-conflict-with-guava-s3KgN9QjCqV_vv_e8w_WEj-cDC24XQrO0AyRuXTzP5k.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava-s3KgN9QjCqV_vv_e8w_WEj-cDC24XQrO0AyRuXTzP5k.jar' is inaccessible.
Dec 26, 2021 1:21:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-26T13:21:55.806Z: Staged package netty-tcnative-boringssl-static-2.0.33.Final-q3CryMTeke7zEAdYTla2hvWe2cxEwrM38_6C8EAtOH0.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-tcnative-boringssl-static-2.0.33.Final-q3CryMTeke7zEAdYTla2hvWe2cxEwrM38_6C8EAtOH0.jar' is inaccessible.
Dec 26, 2021 1:21:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-12-26T13:21:56.402Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Dec 26, 2021 1:24:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-12-26T13:24:56.202Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Dec 26, 2021 1:27:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-26T13:27:53.875Z: Staged package failureaccess-1.0.1-oXHuTHNN0tqDfksWvp30Zhr6typBra8x64Tf2vk2yiY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/failureaccess-1.0.1-oXHuTHNN0tqDfksWvp30Zhr6typBra8x64Tf2vk2yiY.jar' is inaccessible.
Dec 26, 2021 1:27:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-26T13:27:55.592Z: Staged package listenablefuture-9999.0-empty-to-avoid-conflict-with-guava-s3KgN9QjCqV_vv_e8w_WEj-cDC24XQrO0AyRuXTzP5k.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava-s3KgN9QjCqV_vv_e8w_WEj-cDC24XQrO0AyRuXTzP5k.jar' is inaccessible.
Dec 26, 2021 1:27:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-26T13:27:55.822Z: Staged package netty-tcnative-boringssl-static-2.0.33.Final-q3CryMTeke7zEAdYTla2hvWe2cxEwrM38_6C8EAtOH0.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-tcnative-boringssl-static-2.0.33.Final-q3CryMTeke7zEAdYTla2hvWe2cxEwrM38_6C8EAtOH0.jar' is inaccessible.
Dec 26, 2021 1:27:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-12-26T13:27:56.453Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Dec 26, 2021 1:30:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-12-26T13:30:56.257Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Dec 26, 2021 1:33:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-26T13:33:53.851Z: Staged package failureaccess-1.0.1-oXHuTHNN0tqDfksWvp30Zhr6typBra8x64Tf2vk2yiY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/failureaccess-1.0.1-oXHuTHNN0tqDfksWvp30Zhr6typBra8x64Tf2vk2yiY.jar' is inaccessible.
Dec 26, 2021 1:33:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-26T13:33:55.452Z: Staged package listenablefuture-9999.0-empty-to-avoid-conflict-with-guava-s3KgN9QjCqV_vv_e8w_WEj-cDC24XQrO0AyRuXTzP5k.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava-s3KgN9QjCqV_vv_e8w_WEj-cDC24XQrO0AyRuXTzP5k.jar' is inaccessible.
Dec 26, 2021 1:33:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-26T13:33:55.704Z: Staged package netty-tcnative-boringssl-static-2.0.33.Final-q3CryMTeke7zEAdYTla2hvWe2cxEwrM38_6C8EAtOH0.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-tcnative-boringssl-static-2.0.33.Final-q3CryMTeke7zEAdYTla2hvWe2cxEwrM38_6C8EAtOH0.jar' is inaccessible.
Dec 26, 2021 1:33:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-12-26T13:33:56.295Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Dec 26, 2021 1:36:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-12-26T13:36:56.110Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Dec 26, 2021 1:39:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-26T13:39:53.704Z: Staged package failureaccess-1.0.1-oXHuTHNN0tqDfksWvp30Zhr6typBra8x64Tf2vk2yiY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/failureaccess-1.0.1-oXHuTHNN0tqDfksWvp30Zhr6typBra8x64Tf2vk2yiY.jar' is inaccessible.
Dec 26, 2021 1:39:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-26T13:39:55.359Z: Staged package listenablefuture-9999.0-empty-to-avoid-conflict-with-guava-s3KgN9QjCqV_vv_e8w_WEj-cDC24XQrO0AyRuXTzP5k.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava-s3KgN9QjCqV_vv_e8w_WEj-cDC24XQrO0AyRuXTzP5k.jar' is inaccessible.
Dec 26, 2021 1:39:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-26T13:39:55.622Z: Staged package netty-tcnative-boringssl-static-2.0.33.Final-q3CryMTeke7zEAdYTla2hvWe2cxEwrM38_6C8EAtOH0.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-tcnative-boringssl-static-2.0.33.Final-q3CryMTeke7zEAdYTla2hvWe2cxEwrM38_6C8EAtOH0.jar' is inaccessible.
Dec 26, 2021 1:39:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-12-26T13:39:56.208Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Dec 26, 2021 1:42:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-12-26T13:42:56.567Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Dec 26, 2021 1:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-26T13:45:53.809Z: Staged package failureaccess-1.0.1-oXHuTHNN0tqDfksWvp30Zhr6typBra8x64Tf2vk2yiY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/failureaccess-1.0.1-oXHuTHNN0tqDfksWvp30Zhr6typBra8x64Tf2vk2yiY.jar' is inaccessible.
Dec 26, 2021 1:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-26T13:45:55.346Z: Staged package listenablefuture-9999.0-empty-to-avoid-conflict-with-guava-s3KgN9QjCqV_vv_e8w_WEj-cDC24XQrO0AyRuXTzP5k.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava-s3KgN9QjCqV_vv_e8w_WEj-cDC24XQrO0AyRuXTzP5k.jar' is inaccessible.
Dec 26, 2021 1:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-26T13:45:55.593Z: Staged package netty-tcnative-boringssl-static-2.0.33.Final-q3CryMTeke7zEAdYTla2hvWe2cxEwrM38_6C8EAtOH0.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-tcnative-boringssl-static-2.0.33.Final-q3CryMTeke7zEAdYTla2hvWe2cxEwrM38_6C8EAtOH0.jar' is inaccessible.
Dec 26, 2021 1:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-12-26T13:45:56.244Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Dec 26, 2021 1:48:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-12-26T13:48:56.238Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Dec 26, 2021 1:51:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-26T13:51:53.783Z: Staged package failureaccess-1.0.1-oXHuTHNN0tqDfksWvp30Zhr6typBra8x64Tf2vk2yiY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/failureaccess-1.0.1-oXHuTHNN0tqDfksWvp30Zhr6typBra8x64Tf2vk2yiY.jar' is inaccessible.
Dec 26, 2021 1:51:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-26T13:51:55.505Z: Staged package listenablefuture-9999.0-empty-to-avoid-conflict-with-guava-s3KgN9QjCqV_vv_e8w_WEj-cDC24XQrO0AyRuXTzP5k.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava-s3KgN9QjCqV_vv_e8w_WEj-cDC24XQrO0AyRuXTzP5k.jar' is inaccessible.
Dec 26, 2021 1:51:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-26T13:51:55.744Z: Staged package netty-tcnative-boringssl-static-2.0.33.Final-q3CryMTeke7zEAdYTla2hvWe2cxEwrM38_6C8EAtOH0.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-tcnative-boringssl-static-2.0.33.Final-q3CryMTeke7zEAdYTla2hvWe2cxEwrM38_6C8EAtOH0.jar' is inaccessible.
Dec 26, 2021 1:51:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-12-26T13:51:56.328Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Dec 26, 2021 1:54:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-12-26T13:54:57.480Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Dec 26, 2021 1:57:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-26T13:57:53.806Z: Staged package failureaccess-1.0.1-oXHuTHNN0tqDfksWvp30Zhr6typBra8x64Tf2vk2yiY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/failureaccess-1.0.1-oXHuTHNN0tqDfksWvp30Zhr6typBra8x64Tf2vk2yiY.jar' is inaccessible.
Dec 26, 2021 1:57:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-26T13:57:55.444Z: Staged package listenablefuture-9999.0-empty-to-avoid-conflict-with-guava-s3KgN9QjCqV_vv_e8w_WEj-cDC24XQrO0AyRuXTzP5k.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava-s3KgN9QjCqV_vv_e8w_WEj-cDC24XQrO0AyRuXTzP5k.jar' is inaccessible.
Dec 26, 2021 1:57:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-26T13:57:55.703Z: Staged package netty-tcnative-boringssl-static-2.0.33.Final-q3CryMTeke7zEAdYTla2hvWe2cxEwrM38_6C8EAtOH0.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-tcnative-boringssl-static-2.0.33.Final-q3CryMTeke7zEAdYTla2hvWe2cxEwrM38_6C8EAtOH0.jar' is inaccessible.
Dec 26, 2021 1:57:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-12-26T13:57:56.377Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Dec 26, 2021 2:00:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-12-26T14:00:56.228Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Dec 26, 2021 2:03:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-26T14:03:53.732Z: Staged package failureaccess-1.0.1-oXHuTHNN0tqDfksWvp30Zhr6typBra8x64Tf2vk2yiY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/failureaccess-1.0.1-oXHuTHNN0tqDfksWvp30Zhr6typBra8x64Tf2vk2yiY.jar' is inaccessible.
Dec 26, 2021 2:03:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-26T14:03:55.383Z: Staged package listenablefuture-9999.0-empty-to-avoid-conflict-with-guava-s3KgN9QjCqV_vv_e8w_WEj-cDC24XQrO0AyRuXTzP5k.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava-s3KgN9QjCqV_vv_e8w_WEj-cDC24XQrO0AyRuXTzP5k.jar' is inaccessible.
Dec 26, 2021 2:03:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-26T14:03:55.661Z: Staged package netty-tcnative-boringssl-static-2.0.33.Final-q3CryMTeke7zEAdYTla2hvWe2cxEwrM38_6C8EAtOH0.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-tcnative-boringssl-static-2.0.33.Final-q3CryMTeke7zEAdYTla2hvWe2cxEwrM38_6C8EAtOH0.jar' is inaccessible.
Dec 26, 2021 2:03:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-12-26T14:03:56.315Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Dec 26, 2021 4:00:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-26T16:00:32.862Z: Cancel request is committed for workflow job: 2021-12-26_04_45_34-13218376411721923737.
Dec 26, 2021 4:00:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-26T16:00:32.896Z: Cleaning up.
Dec 26, 2021 4:00:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-26T16:00:32.970Z: Stopping **** pool...
Dec 26, 2021 4:00:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-26T16:00:33.024Z: Stopping **** pool...
Dec 26, 2021 4:03:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-26T16:02:58.158Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Dec 26, 2021 4:03:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-26T16:02:58.196Z: Worker pool stopped.
Dec 26, 2021 4:03:04 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-12-26_04_45_34-13218376411721923737 finished with status CANCELLED.
Load test results for test (ID): 3518d724-bf76-41d3-9f3a-9e62f9af6b5a and timestamp: 2021-12-26T12:45:29.709000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11541.274
dataflow_v2_java11_total_bytes_count             1.75150109E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211226124333
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:b2baa66d4b8dbce58c28f0f56cb1cfee194c29b246c3e273e1761ec87756fb90
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211226124333]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:b2baa66d4b8dbce58c28f0f56cb1cfee194c29b246c3e273e1761ec87756fb90]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211226124333] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:b2baa66d4b8dbce58c28f0f56cb1cfee194c29b246c3e273e1761ec87756fb90])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:b2baa66d4b8dbce58c28f0f56cb1cfee194c29b246c3e273e1761ec87756fb90
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:b2baa66d4b8dbce58c28f0f56cb1cfee194c29b246c3e273e1761ec87756fb90
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:b2baa66d4b8dbce58c28f0f56cb1cfee194c29b246c3e273e1761ec87756fb90].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings

Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.

BUILD FAILED in 3h 19m 50s
89 actionable tasks: 52 executed, 33 from cache, 4 up-to-date

Publishing build scan...
https://gradle.com/s/7rumlkvvskpri

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #191

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/191/display/redirect>

Changes:


------------------------------------------
[...truncated 45.89 KB...]
Dec 25, 2021 12:47:11 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 203 files. Enable logging at DEBUG level to see which files will be staged.
Dec 25, 2021 12:47:12 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Dec 25, 2021 12:47:13 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Dec 25, 2021 12:47:15 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 203 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Dec 25, 2021 12:47:16 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 203 files cached, 0 files newly uploaded in 0 seconds
Dec 25, 2021 12:47:16 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Dec 25, 2021 12:47:16 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <113691 bytes, hash 054409705b0d8499190adcbf08f46b4657022bdd9f6a91df456888cf64144c8e> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-BUQJcFsNhJkZCty_CPRrRlcCK92fapHfRWiIz2QUTI4.pb
Dec 25, 2021 12:47:18 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Dec 25, 2021 12:47:18 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@15405bd6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@352ed70d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@70730db, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5793b87, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12704e15, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@512575e9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6f1a16fe, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2373ad99, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@33634f04, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4993febc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7601bc96, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@48a0c8aa, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6192a5d5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3722c145, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4cbc2e3b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2975a9e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@765ffb14, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@57562473, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7a360554, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@424de326]
Dec 25, 2021 12:47:18 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Dec 25, 2021 12:47:18 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Dec 25, 2021 12:47:18 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Dec 25, 2021 12:47:18 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Dec 25, 2021 12:47:18 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@597f0937, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7ad1caa2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6b6b3572, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@65d8dff8, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@444f44c5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@303f1234, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@24d61e4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2149594a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1f1e58ca, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@57f847af, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ed34ef5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@553bc36c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@380e1909, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2d5ef498, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@95eb320, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2f521c4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4afbb6c2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@10db6131, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6c6017b9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4730e0f0]
Dec 25, 2021 12:47:18 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Dec 25, 2021 12:47:18 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Dec 25, 2021 12:47:18 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Dec 25, 2021 12:47:18 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Dec 25, 2021 12:47:18 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Dec 25, 2021 12:47:18 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Dec 25, 2021 12:47:18 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Dec 25, 2021 12:47:18 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Dec 25, 2021 12:47:18 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Dec 25, 2021 12:47:18 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Dec 25, 2021 12:47:18 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Dec 25, 2021 12:47:18 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.36.0-SNAPSHOT
Dec 25, 2021 12:47:19 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-12-25_04_47_18-16676523383915913724?project=apache-beam-testing
Dec 25, 2021 12:47:19 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-12-25_04_47_18-16676523383915913724
Dec 25, 2021 12:47:19 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-12-25_04_47_18-16676523383915913724
Dec 25, 2021 12:47:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-12-25T12:47:26.601Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-12-66l5. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Dec 25, 2021 12:47:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-25T12:47:31.208Z: Worker configuration: e2-standard-2 in us-central1-a.
Dec 25, 2021 12:47:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-25T12:47:31.941Z: Expanding SplittableParDo operations into optimizable parts.
Dec 25, 2021 12:47:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-25T12:47:31.971Z: Expanding CollectionToSingleton operations into optimizable parts.
Dec 25, 2021 12:47:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-25T12:47:32.026Z: Expanding CoGroupByKey operations into optimizable parts.
Dec 25, 2021 12:47:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-25T12:47:32.074Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Dec 25, 2021 12:47:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-25T12:47:32.094Z: Expanding GroupByKey operations into streaming Read/Write steps
Dec 25, 2021 12:47:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-25T12:47:32.136Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Dec 25, 2021 12:47:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-25T12:47:32.213Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Dec 25, 2021 12:47:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-25T12:47:32.235Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Dec 25, 2021 12:47:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-25T12:47:32.257Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Dec 25, 2021 12:47:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-25T12:47:32.282Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Dec 25, 2021 12:47:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-25T12:47:32.304Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Dec 25, 2021 12:47:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-25T12:47:32.331Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Dec 25, 2021 12:47:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-25T12:47:32.358Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Dec 25, 2021 12:47:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-25T12:47:32.378Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Dec 25, 2021 12:47:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-25T12:47:32.398Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Dec 25, 2021 12:47:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-25T12:47:32.419Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Dec 25, 2021 12:47:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-25T12:47:32.448Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Dec 25, 2021 12:47:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-25T12:47:32.490Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Dec 25, 2021 12:47:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-25T12:47:32.511Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Dec 25, 2021 12:47:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-25T12:47:32.532Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Dec 25, 2021 12:47:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-25T12:47:32.559Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Dec 25, 2021 12:47:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-25T12:47:32.580Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Dec 25, 2021 12:47:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-25T12:47:32.602Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Dec 25, 2021 12:47:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-25T12:47:32.639Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Dec 25, 2021 12:47:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-25T12:47:32.664Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Dec 25, 2021 12:47:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-25T12:47:32.691Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Dec 25, 2021 12:47:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-25T12:47:32.730Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Dec 25, 2021 12:47:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-25T12:47:32.772Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Dec 25, 2021 12:47:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-25T12:47:32.800Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Dec 25, 2021 12:47:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-25T12:47:33.205Z: Starting 5 ****s in us-central1-a...
Dec 25, 2021 12:48:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-25T12:47:57.842Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Dec 25, 2021 12:48:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-25T12:48:21.777Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Dec 25, 2021 12:49:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-25T12:49:17.842Z: Workers have started successfully.
Dec 25, 2021 12:49:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-25T12:49:17.869Z: Workers have started successfully.
Dec 25, 2021 2:41:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-25T14:41:33.323Z: Staged package amazon-kinesis-client-1.14.2-qtBh3OuKuLE7S26ITTJzYsoYfG-mWjqbGmv3VJPhec8.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/amazon-kinesis-client-1.14.2-qtBh3OuKuLE7S26ITTJzYsoYfG-mWjqbGmv3VJPhec8.jar' is inaccessible.
Dec 25, 2021 2:41:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-25T14:41:33.603Z: Staged package aws-java-sdk-cloudwatch-1.12.106-kUP1xZSNspLnoAnC0aAzibk9w0IXNbBNeMBESBYiZtg.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/aws-java-sdk-cloudwatch-1.12.106-kUP1xZSNspLnoAnC0aAzibk9w0IXNbBNeMBESBYiZtg.jar' is inaccessible.
Dec 25, 2021 2:41:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-25T14:41:33.705Z: Staged package aws-java-sdk-core-1.12.106-2a5g2j72YwhD9uNhLJWqRljsgds1wEVsdH_XLgZZOlQ.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/aws-java-sdk-core-1.12.106-2a5g2j72YwhD9uNhLJWqRljsgds1wEVsdH_XLgZZOlQ.jar' is inaccessible.
Dec 25, 2021 2:41:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-25T14:41:33.740Z: Staged package aws-java-sdk-dynamodb-1.12.106-s8z13aUxEWhNzx7J8mY6rVcDrK2iLAcBqT3Pb9sNgmM.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/aws-java-sdk-dynamodb-1.12.106-s8z13aUxEWhNzx7J8mY6rVcDrK2iLAcBqT3Pb9sNgmM.jar' is inaccessible.
Dec 25, 2021 2:41:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-25T14:41:33.836Z: Staged package aws-java-sdk-kinesis-1.12.106-wMPghgT8YvcVokv233DC2r8Q0eeaqh7phw3Anmz6S4o.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/aws-java-sdk-kinesis-1.12.106-wMPghgT8YvcVokv233DC2r8Q0eeaqh7phw3Anmz6S4o.jar' is inaccessible.
Dec 25, 2021 2:41:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-25T14:41:33.873Z: Staged package aws-java-sdk-kms-1.12.106-JkHOiVwKCYdQ3d-wR7YYej-O0UmkZOioL923IYI9PwM.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/aws-java-sdk-kms-1.12.106-JkHOiVwKCYdQ3d-wR7YYej-O0UmkZOioL923IYI9PwM.jar' is inaccessible.
Dec 25, 2021 2:41:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-25T14:41:33.913Z: Staged package aws-java-sdk-s3-1.12.106-5YV0cnyV551lh5cwCZGFhypcyG7NsqTcEP0IBjaAnjE.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/aws-java-sdk-s3-1.12.106-5YV0cnyV551lh5cwCZGFhypcyG7NsqTcEP0IBjaAnjE.jar' is inaccessible.
Dec 25, 2021 2:41:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-25T14:41:36.381Z: Staged package jakarta.activation-api-1.2.2-oYepORA671hJp6-EvX4nvi0SDEEK8pFDc3X_4GH08J0.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/jakarta.activation-api-1.2.2-oYepORA671hJp6-EvX4nvi0SDEEK8pFDc3X_4GH08J0.jar' is inaccessible.
Dec 25, 2021 2:41:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-25T14:41:36.416Z: Staged package jakarta.xml.bind-api-2.3.3-wEU59HLppt0MdoXqgtZ3KCJpq457rKLhRQDjgeDGzsU.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/jakarta.xml.bind-api-2.3.3-wEU59HLppt0MdoXqgtZ3KCJpq457rKLhRQDjgeDGzsU.jar' is inaccessible.
Dec 25, 2021 2:41:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-25T14:41:36.470Z: Staged package jmespath-java-1.12.106-_mWkV0KPXB56ULuR7j6dxTg6hrDhcycHHtCXYBv9V7s.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/jmespath-java-1.12.106-_mWkV0KPXB56ULuR7j6dxTg6hrDhcycHHtCXYBv9V7s.jar' is inaccessible.
Dec 25, 2021 2:41:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-12-25T14:41:37.466Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Dec 25, 2021 4:00:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-25T16:00:36.636Z: Cancel request is committed for workflow job: 2021-12-25_04_47_18-16676523383915913724.
Dec 25, 2021 4:00:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-25T16:00:36.684Z: Cleaning up.
Dec 25, 2021 4:00:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-25T16:00:36.786Z: Stopping **** pool...
Dec 25, 2021 4:00:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-25T16:00:36.841Z: Stopping **** pool...
Dec 25, 2021 4:03:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-25T16:02:57.215Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Dec 25, 2021 4:03:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-25T16:02:57.250Z: Worker pool stopped.
Dec 25, 2021 4:03:07 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-12-25_04_47_18-16676523383915913724 finished with status CANCELLED.
Load test results for test (ID): 9bad4cbc-5eda-45ba-a17a-0790d13957c7 and timestamp: 2021-12-25T12:47:12.394000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11449.001
dataflow_v2_java11_total_bytes_count              2.0612342E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211225124342
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:bebad7d86c58d54eed89480d05635972088624443b0cddccd6b50863989fb7a9
Deleted: sha256:df760ff267a791332f26d3cca3d011cba2f6cbfefd306cf517016103dd214741
Deleted: sha256:070608c4828ff34e17dd48ddf67ad230aeb5b51c2cde4e3c6c6122844fd41e65
Deleted: sha256:8d45bdd9a5a589f580cdbdde3a55663d482578d230769b2ad64b6b1d892cc2fd
Deleted: sha256:a6c5dc11c2c8d70bfc11f6b514bfc24b49dc4aba15cc716c06c1458ae1c8adb2
Deleted: sha256:830b5ace3970a11d94ee8a48004739a1942bda5aaaa0424e57598929bf9b3a4e
Deleted: sha256:1d1f4c98e49ba1f1e6aefdf8ddcfc4878c70fd6ccbe1461b945da9ff3193fef0
Deleted: sha256:0538cb8917693fc9f96e483550da7c8e2f5706a607fb6bc7f412eb5e375abe58
Deleted: sha256:66d26ec89e927225195d70eb322c605f585566bfd9e567f33cb1e5ea3744fe63
Deleted: sha256:371ae7a9cd4aae2a012ef5b22da5182d7f47e9589a663eba9f9b248bf7d74d0f
Deleted: sha256:8c3c7c3913ca7ebd7b50c513fd34a6c17dab494b1e2d3a4ec6bcd2169ea3c014
Deleted: sha256:257d8efe6f9fbbdcd8d4a5e7fa6472c2fe1c34ea65904f82cd29e7fd1a2e5ab8
Deleted: sha256:e3306b768c98a6a07e3cc238d16f252ddb8b887788456fcef54c811e0fc96eac
Deleted: sha256:0f29038e6b54989d1884dd91d755debd1252d0aa8b003f30ab688a71f95ded15
Deleted: sha256:b9318c39d70d714fa9b558511963d1d2200ab9233edf1e1d4d7aaedfc4572f05
Deleted: sha256:9e944d3449079e49245513dc9894844746295925c594d7bd12cff5f780dabacc
Deleted: sha256:88f0ea721cd3d12376cd7e39b30c4bf3749cee26e69301237399965898249eac
Deleted: sha256:efd8392f2c062eeb7dc2ce8baddf9ab0c0b977cb16b8017fcbf98b06421ebf50
Deleted: sha256:d9d8655b13ffa3b36e5e58a26091e550ce8808bd2868c651faafcadc5caf8fee
Deleted: sha256:7d7148704a86585a414759b2b46d11e2f5943d31424bf644474c56516cbc456a
Deleted: sha256:c3965f67e0a689488973e51aff064397da86ab42c7176471e7ac63281083c9e3
Deleted: sha256:e86d7e4e7c74f89e96e84006920a17d2319ca0765b23b459f849301aec421cc8
Deleted: sha256:36701e3a850164f861027992e1a9123d7d5c820e41472ddc5106c491879efce2
Deleted: sha256:def3b980082495ef69a81c1365b61bfad6bfa0e4497a4dd83750a9e4362a9fcd
Deleted: sha256:2f27fc4f1f6f33ba903d089bc6f3931432fcaa8670ec0fe7cb8744d5521467f7
Deleted: sha256:26f8987d90ad5be4b639442ebd2ddb3806f2544fca951d8202937ae2c7247e08
Deleted: sha256:ddd23614e2823a4188acb6956eac18e21e671703c840b46850f496a38d83b08d
Deleted: sha256:c112f00a596d88f0be1bcdd49b435ed995887c669934c5d18a5fadb76d867c33
Deleted: sha256:b0c075de5ecf3d7f4985951015693d71600d4f91073f63494bfe83d3e9e2e64e
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211225124342]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:bebad7d86c58d54eed89480d05635972088624443b0cddccd6b50863989fb7a9]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211225124342] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:bebad7d86c58d54eed89480d05635972088624443b0cddccd6b50863989fb7a9])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:bebad7d86c58d54eed89480d05635972088624443b0cddccd6b50863989fb7a9
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:bebad7d86c58d54eed89480d05635972088624443b0cddccd6b50863989fb7a9
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:bebad7d86c58d54eed89480d05635972088624443b0cddccd6b50863989fb7a9].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings

Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.

BUILD FAILED in 3h 19m 46s
89 actionable tasks: 52 executed, 33 from cache, 4 up-to-date

Publishing build scan...
https://gradle.com/s/oiigzphgwna52

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #190

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/190/display/redirect?page=changes>

Changes:

[alexander.zhuravlev] [BEAM-13489] Updated manifest.json and index.html to follow best

[stranniknm] [BEAM-13531] playground - fetch example info only once

[mmack] [BEAM-7559] Add integration test for DynamoDBIO (SDK v2)

[mmack] [BEAM-7559] Add integration test for DynamoDBIO (SDK v1)

[noreply] [BEAM-13399] Disable expansion service integration test on Samza and

[noreply] [BEAM-13430] Upgrade beam to gradle 7.3.2 (#16319)

[noreply] Merge pull request #16320 from [BEAM-13512] [Playground] send request to

[noreply] Merge pull request #16209 from [Playground][BEAM-13304]Deploy Python

[noreply] Merge pull request #16105 from [BEAM-13328] [Playground] Add

[noreply] Merge pull request #16103 from [BEAM-13252] [Playground] Feedback Thumbs

[noreply] Better type inference for GroupBy. (#16318)

[noreply] [BEAM-13534] Update TestAutomatedExpansionService liveness check to not

[noreply] [BEAM-13490] Bump log4j version to 2.17.0 (#16276)


------------------------------------------
[...truncated 44.47 KB...]
39d32f4aa32e: Pushed
54a608421806: Pushed
043b179ca7a1: Pushed
415c559b919d: Pushed
38f25d46b52e: Pushed
889918499d98: Pushed
3bb5258f46d2: Layer already exists
03473aea7665: Pushed
832e177bb500: Layer already exists
f9e18e59a565: Layer already exists
219e8947999a: Pushed
3a8cb18318dd: Pushed
26a504e63be4: Layer already exists
8bf42db0de72: Layer already exists
31892cc314cb: Layer already exists
11936051f93b: Layer already exists
67e69396e5de: Pushed
20211224124334: digest: sha256:819138cae5fec4c77812ba3c01567df8ebad86f3ed077f70d2f5a15586519627 size: 4311

> Task :sdks:java:testing:load-tests:run
Dec 24, 2021 12:45:31 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Dec 24, 2021 12:45:32 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 203 files. Enable logging at DEBUG level to see which files will be staged.
Dec 24, 2021 12:45:32 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Dec 24, 2021 12:45:32 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Dec 24, 2021 12:45:35 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 203 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Dec 24, 2021 12:45:36 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 203 files cached, 0 files newly uploaded in 0 seconds
Dec 24, 2021 12:45:36 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Dec 24, 2021 12:45:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <113691 bytes, hash 6ffbd572b291033ca9c7100a9da088726ae01fdb3d4a956feecfd57f453307db> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-b_vVcrKRAzypxxAKnaCIcmrgH9s9SpVv7s_Vf0UzB9s.pb
Dec 24, 2021 12:45:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Dec 24, 2021 12:45:37 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@70730db, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5793b87, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12704e15, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@512575e9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6f1a16fe, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2373ad99, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@33634f04, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4993febc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7601bc96, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@48a0c8aa, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6192a5d5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3722c145, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4cbc2e3b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2975a9e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@765ffb14, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@57562473, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7a360554, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@424de326, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4bc33720, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2dd0f797]
Dec 24, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Dec 24, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Dec 24, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Dec 24, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Dec 24, 2021 12:45:38 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6b6b3572, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@65d8dff8, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@444f44c5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@303f1234, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@24d61e4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2149594a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1f1e58ca, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@57f847af, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ed34ef5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@553bc36c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@380e1909, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2d5ef498, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@95eb320, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2f521c4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4afbb6c2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@10db6131, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6c6017b9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4730e0f0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@506a1372, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1b7332a7]
Dec 24, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Dec 24, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Dec 24, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Dec 24, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Dec 24, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Dec 24, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Dec 24, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Dec 24, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Dec 24, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Dec 24, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Dec 24, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Dec 24, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.36.0-SNAPSHOT
Dec 24, 2021 12:45:39 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-12-24_04_45_38-16419312113343538639?project=apache-beam-testing
Dec 24, 2021 12:45:39 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-12-24_04_45_38-16419312113343538639
Dec 24, 2021 12:45:39 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-12-24_04_45_38-16419312113343538639
Dec 24, 2021 12:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-12-24T12:45:45.343Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-12-jnjx. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Dec 24, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-24T12:45:49.403Z: Worker configuration: e2-standard-2 in us-central1-a.
Dec 24, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-24T12:45:49.944Z: Expanding SplittableParDo operations into optimizable parts.
Dec 24, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-24T12:45:50.000Z: Expanding CollectionToSingleton operations into optimizable parts.
Dec 24, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-24T12:45:50.058Z: Expanding CoGroupByKey operations into optimizable parts.
Dec 24, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-24T12:45:50.135Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Dec 24, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-24T12:45:50.178Z: Expanding GroupByKey operations into streaming Read/Write steps
Dec 24, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-24T12:45:50.232Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Dec 24, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-24T12:45:50.343Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Dec 24, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-24T12:45:50.378Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Dec 24, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-24T12:45:50.411Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Dec 24, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-24T12:45:50.447Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Dec 24, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-24T12:45:50.470Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Dec 24, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-24T12:45:50.494Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Dec 24, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-24T12:45:50.518Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Dec 24, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-24T12:45:50.554Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Dec 24, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-24T12:45:50.585Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Dec 24, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-24T12:45:50.616Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Dec 24, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-24T12:45:50.643Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Dec 24, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-24T12:45:50.673Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Dec 24, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-24T12:45:50.707Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Dec 24, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-24T12:45:50.743Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Dec 24, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-24T12:45:50.766Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Dec 24, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-24T12:45:50.800Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Dec 24, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-24T12:45:50.836Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Dec 24, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-24T12:45:50.873Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Dec 24, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-24T12:45:50.906Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Dec 24, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-24T12:45:50.931Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Dec 24, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-24T12:45:50.960Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Dec 24, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-24T12:45:50.993Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Dec 24, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-24T12:45:51.027Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Dec 24, 2021 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-24T12:45:51.420Z: Starting 5 ****s in us-central1-a...
Dec 24, 2021 12:46:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-24T12:46:05.233Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Dec 24, 2021 12:46:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-24T12:46:31.576Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Dec 24, 2021 12:47:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-24T12:47:29.089Z: Workers have started successfully.
Dec 24, 2021 12:47:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-24T12:47:29.120Z: Workers have started successfully.
Dec 24, 2021 4:00:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-24T16:00:38.715Z: Cancel request is committed for workflow job: 2021-12-24_04_45_38-16419312113343538639.
Dec 24, 2021 4:00:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-24T16:00:38.782Z: Cleaning up.
Dec 24, 2021 4:00:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-24T16:00:38.908Z: Stopping **** pool...
Dec 24, 2021 4:00:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-24T16:00:38.973Z: Stopping **** pool...
Dec 24, 2021 4:03:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-24T16:03:07.648Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Dec 24, 2021 4:03:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-24T16:03:07.689Z: Worker pool stopped.
Dec 24, 2021 4:03:13 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-12-24_04_45_38-16419312113343538639 finished with status CANCELLED.
Load test results for test (ID): b562686d-158d-489f-8bab-14a308559aac and timestamp: 2021-12-24T12:45:32.441000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11506.377
dataflow_v2_java11_total_bytes_count              2.2251691E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211224124334
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:819138cae5fec4c77812ba3c01567df8ebad86f3ed077f70d2f5a15586519627
Deleted: sha256:15a2bb3d9408cc97712ae8de570b63a578de82d149e3317e919e3d2e0ca12d54
Deleted: sha256:eb39734b6a7e4fce7ea1b6a9cd2f34f685c4e04a000c71a2cefc9917c2070669
Deleted: sha256:12eff510f376a49bf2efdf03127c037fbe4a8b2718b9a8510481c3214ca3b41e
Deleted: sha256:322dcc2f6b3384c4ac5670ab1b7a80c2c9c14481ea1d6ec804a64dd13a3ab014
Deleted: sha256:2671a503428579d20b59d5a644368be52bfc0455f6a0f579bb64fe81c2af531f
Deleted: sha256:d1deaecc5b580eef71bf029520c4fbcf7061e31be7b9321ef6161c75ac4df0c5
Deleted: sha256:7121bd6bc762618d48e9b29beddc0b61662547af4cbe4a808848695da9a1a27f
Deleted: sha256:1b993cddde22fd1020da220202da2b8acdd9c9a0f4d430fa79cceb037210d1cb
Deleted: sha256:b3d37efd95aa3f65e65752c0ec6b26a431b229562993622143a2929afb9868fb
Deleted: sha256:23f10af8050856eb83998d1c79ce25c00c53778a31601e1493eaf0d3304a3cbb
Deleted: sha256:27a87f950e8108111850295b14a0555df6b92daf1b0c504f7314154b8771c4e1
Deleted: sha256:94f1eb787a2c830bba6de32ad940765c9fde394604312696b90738750e4b4ddb
Deleted: sha256:c184e0d7eb6820efe3c16ac4187c976174e1b0cac171288a4a5fea09d60c6970
Deleted: sha256:36a8a2c1f198fa9d34e624cd2b6a7d2adc349b5a9b200b47e1be4933102f0be6
Deleted: sha256:dfd8015243bac8453f2c50200ca7f1c55a2a1ade48fff4b8359e06bdd719e876
Deleted: sha256:ee7b90a83ebd04b49f9e94e6dacd4efc229676297776c710a9cc7e0a982972ba
Deleted: sha256:9eb0263b3da2438eefe8e7aafc5c0ed44c8a97749c096e7bbb24d03ff615f1fe
Deleted: sha256:30d4b29a3bca342ad288ee996a1b12908d710c9bf58a7dde35d2d51abed3da8a
Deleted: sha256:0e7ea0a07f55e1c70ce9190c85c3234fb22e0784f05f00373d5d6c5c0bd29b41
Deleted: sha256:8f393789752cc312f5c12efdc8684ba91c1d1006ae4a9690259075cd1a911b29
Deleted: sha256:29841c73e094a8bc1baaf1858661e85179365182288c7bfff601ad255b797900
Deleted: sha256:557599c07cc85404825d715e2a0f8ab7d61dee5429951826cdd83993510ff371
Deleted: sha256:ac224a0445fad0908c87edf0a49d1b09aa5ef7275c869277ce45b1a6e06200b5
Deleted: sha256:944ed90d1a1bddecfae0dd79863d6388f1197aafd99c1741920f48b23d8f7f25
Deleted: sha256:5aa42f150e66458c6f44c5f551c954e4a186f70eaa6383a528f7a19207095db2
Deleted: sha256:bb03c1f7b4174733b911df01f0e7fcade678ef1159ada3a2b2663c856a5d200b
Deleted: sha256:11837a38fe967820dbbcb78be21b8469f187d8cba9a16fbe721f92ef6848983b
Deleted: sha256:e7da07073c98a02b91efe12ebd1daca1e14d2ba3eab8baa6024b212900cef2f7
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211224124334]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:819138cae5fec4c77812ba3c01567df8ebad86f3ed077f70d2f5a15586519627]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211224124334] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:819138cae5fec4c77812ba3c01567df8ebad86f3ed077f70d2f5a15586519627])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:819138cae5fec4c77812ba3c01567df8ebad86f3ed077f70d2f5a15586519627
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:819138cae5fec4c77812ba3c01567df8ebad86f3ed077f70d2f5a15586519627
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:819138cae5fec4c77812ba3c01567df8ebad86f3ed077f70d2f5a15586519627].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings

Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.

BUILD FAILED in 3h 19m 57s
89 actionable tasks: 52 executed, 33 from cache, 4 up-to-date

Publishing build scan...
https://gradle.com/s/o7delkueyonba

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #189

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/189/display/redirect?page=changes>

Changes:

[mmack] [BEAM-13494] Avoid draining Kinesis API limits & CPU when reading from

[arietis27] [BEAM-13517] Unable to write nulls to columns with logical types

[stranniknm] [BEAM-13501] playground - add log about processing and precompiled run

[noreply] Make S3 streaming more efficient (#15931)

[noreply] [Playground][BEAM-13358]Deploy Go service via Ci/CD (#16210)

[dpcollins] [BEAM-13430] Clean up tests that override the time of the JVM

[noreply] [BEAM-12830] Install go version w/host platform. (#16330)

[noreply] [BEAM-13399] Add extra time buffer to reduce likelihood of flake in

[noreply] [BEAM-12865] Fix triggering_frequency validation (#16315)

[noreply] [BEAM-13525] Sickbay new PardoTests for Dataflow V2 Java Streaming VR

[noreply] [BEAM-12830] Avoid flock and prepare task once. (OSX fix.) (#16333)

[noreply] Bump Python beam-master containers (#16332)

[arietis27] [BEAM-13517] Unable to write nulls to columns with logical types


------------------------------------------
[...truncated 43.93 KB...]
0da929f5ccc4: Pushed
39ef6f16b40e: Pushed
c3e5d2b29dd5: Pushed
3bb5258f46d2: Layer already exists
832e177bb500: Layer already exists
5867bd9c236b: Pushed
f9e18e59a565: Layer already exists
7bec1ec86199: Pushed
26a504e63be4: Layer already exists
cc49bcd0e7dd: Pushed
31892cc314cb: Layer already exists
8bf42db0de72: Layer already exists
11936051f93b: Layer already exists
dcc1c0a65110: Pushed
20211223124335: digest: sha256:c80958524a8555af812cc96f92bca50aba836ec2a12a934b4f0c5698bba2c51d size: 4311

> Task :sdks:java:testing:load-tests:run
Dec 23, 2021 12:45:22 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Dec 23, 2021 12:45:23 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 203 files. Enable logging at DEBUG level to see which files will be staged.
Dec 23, 2021 12:45:23 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Dec 23, 2021 12:45:23 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Dec 23, 2021 12:45:26 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 203 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Dec 23, 2021 12:45:26 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 203 files cached, 0 files newly uploaded in 0 seconds
Dec 23, 2021 12:45:26 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Dec 23, 2021 12:45:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <113691 bytes, hash 80b377990a10555002a96335f0a942f70a7bbedfe247a957bde66500a5481bff> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-gLN3mQoQVVACqWM18KlC9wp7vt_iR6lXveZlAKVIG_8.pb
Dec 23, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Dec 23, 2021 12:45:28 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@15405bd6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@352ed70d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@70730db, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5793b87, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12704e15, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@512575e9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6f1a16fe, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2373ad99, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@33634f04, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4993febc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7601bc96, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@48a0c8aa, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6192a5d5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3722c145, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4cbc2e3b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2975a9e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@765ffb14, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@57562473, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7a360554, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@424de326]
Dec 23, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Dec 23, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Dec 23, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Dec 23, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Dec 23, 2021 12:45:28 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@597f0937, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7ad1caa2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6b6b3572, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@65d8dff8, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@444f44c5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@303f1234, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@24d61e4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2149594a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1f1e58ca, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@57f847af, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ed34ef5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@553bc36c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@380e1909, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2d5ef498, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@95eb320, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2f521c4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4afbb6c2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@10db6131, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6c6017b9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4730e0f0]
Dec 23, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Dec 23, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Dec 23, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Dec 23, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Dec 23, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Dec 23, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Dec 23, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Dec 23, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Dec 23, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Dec 23, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Dec 23, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Dec 23, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.36.0-SNAPSHOT
Dec 23, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-12-23_04_45_28-14917756070590393734?project=apache-beam-testing
Dec 23, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-12-23_04_45_28-14917756070590393734
Dec 23, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-12-23_04_45_28-14917756070590393734
Dec 23, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-12-23T12:45:36.149Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-12-nit4. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Dec 23, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-23T12:45:40.866Z: Worker configuration: e2-standard-2 in us-central1-a.
Dec 23, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-23T12:45:41.512Z: Expanding SplittableParDo operations into optimizable parts.
Dec 23, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-23T12:45:41.552Z: Expanding CollectionToSingleton operations into optimizable parts.
Dec 23, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-23T12:45:41.619Z: Expanding CoGroupByKey operations into optimizable parts.
Dec 23, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-23T12:45:41.711Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Dec 23, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-23T12:45:41.743Z: Expanding GroupByKey operations into streaming Read/Write steps
Dec 23, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-23T12:45:41.846Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Dec 23, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-23T12:45:41.955Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Dec 23, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-23T12:45:42.000Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Dec 23, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-23T12:45:42.034Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Dec 23, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-23T12:45:42.099Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Dec 23, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-23T12:45:42.129Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Dec 23, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-23T12:45:42.160Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Dec 23, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-23T12:45:42.269Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Dec 23, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-23T12:45:42.306Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Dec 23, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-23T12:45:42.331Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Dec 23, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-23T12:45:42.355Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Dec 23, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-23T12:45:42.380Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Dec 23, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-23T12:45:42.412Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Dec 23, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-23T12:45:42.441Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Dec 23, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-23T12:45:42.497Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Dec 23, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-23T12:45:42.530Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Dec 23, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-23T12:45:42.562Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Dec 23, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-23T12:45:42.601Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Dec 23, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-23T12:45:42.636Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Dec 23, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-23T12:45:42.670Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Dec 23, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-23T12:45:42.698Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Dec 23, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-23T12:45:42.845Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Dec 23, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-23T12:45:42.900Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Dec 23, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-23T12:45:42.937Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Dec 23, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-23T12:45:43.457Z: Starting 5 ****s in us-central1-a...
Dec 23, 2021 12:46:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-23T12:46:11.521Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Dec 23, 2021 12:46:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-23T12:46:33.612Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Dec 23, 2021 12:47:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-23T12:47:29.112Z: Workers have started successfully.
Dec 23, 2021 12:47:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-23T12:47:29.150Z: Workers have started successfully.
Dec 23, 2021 2:30:57 PM org.apache.beam.sdk.metrics.MetricsEnvironment getCurrentContainer
WARNING: Reporting metrics are not supported in the current execution environment.
Dec 23, 2021 3:54:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-23T15:54:44.735Z: Staged package commons-compress-1.21-auz9VFlyillWAc-gcljRMZcv_Dm0kutIvdWWV3ovJEo.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/commons-compress-1.21-auz9VFlyillWAc-gcljRMZcv_Dm0kutIvdWWV3ovJEo.jar' is inaccessible.
Dec 23, 2021 3:54:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-23T15:54:44.929Z: Staged package conscrypt-openjdk-uber-2.5.1-AfnHQstZKhUeLmK9U5eomAYoqWcAH82s1KpHRGeGhfM.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/conscrypt-openjdk-uber-2.5.1-AfnHQstZKhUeLmK9U5eomAYoqWcAH82s1KpHRGeGhfM.jar' is inaccessible.
Dec 23, 2021 3:54:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-23T15:54:46.911Z: Staged package netty-buffer-4.1.52.Final-QcoQNa7m4PgXdZfQp2EKWqLG5vrHRKyB_eTl5Y9jlnU.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-buffer-4.1.52.Final-QcoQNa7m4PgXdZfQp2EKWqLG5vrHRKyB_eTl5Y9jlnU.jar' is inaccessible.
Dec 23, 2021 3:54:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-23T15:54:46.950Z: Staged package netty-codec-4.1.52.Final-PH543M5-NT2GrH26LCUatc3dTrhuqoic_kIpeCuCjlI.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-codec-4.1.52.Final-PH543M5-NT2GrH26LCUatc3dTrhuqoic_kIpeCuCjlI.jar' is inaccessible.
Dec 23, 2021 3:54:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-23T15:54:47.004Z: Staged package netty-codec-http-4.1.52.Final-tdoo0OqHiwfTERrPXk-vQSiO8jArJkFBkP3Rp9lzyAw.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-codec-http-4.1.52.Final-tdoo0OqHiwfTERrPXk-vQSiO8jArJkFBkP3Rp9lzyAw.jar' is inaccessible.
Dec 23, 2021 3:54:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-23T15:54:47.106Z: Staged package netty-codec-http2-4.1.52.Final-61J6QtIonRLe5tR1PKJxWXije7tBxxkVzk6VOcSfk6E.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-codec-http2-4.1.52.Final-61J6QtIonRLe5tR1PKJxWXije7tBxxkVzk6VOcSfk6E.jar' is inaccessible.
Dec 23, 2021 3:54:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-23T15:54:47.151Z: Staged package netty-codec-socks-4.1.52.Final-_sxXfgu3ch1naM_BhG7hcpEdFquA372ZldllelUdTqc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-codec-socks-4.1.52.Final-_sxXfgu3ch1naM_BhG7hcpEdFquA372ZldllelUdTqc.jar' is inaccessible.
Dec 23, 2021 3:54:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-23T15:54:47.222Z: Staged package netty-common-4.1.52.Final-DR8QHk231TDtBNfKvVfZXAdQ8Fm7az-t_tDIDUc0xxc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-common-4.1.52.Final-DR8QHk231TDtBNfKvVfZXAdQ8Fm7az-t_tDIDUc0xxc.jar' is inaccessible.
Dec 23, 2021 3:54:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-23T15:54:47.263Z: Staged package netty-handler-4.1.52.Final-LNwh-_rgSUDIWSkDl3nJP1H2GPXvshYHkxnc_PMqVcE.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-handler-4.1.52.Final-LNwh-_rgSUDIWSkDl3nJP1H2GPXvshYHkxnc_PMqVcE.jar' is inaccessible.
Dec 23, 2021 3:54:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-23T15:54:47.311Z: Staged package netty-handler-proxy-4.1.52.Final-6NB4Y860vUgG4lWh1TLIZQ8POy1NAGrW0hgS6UG0L9w.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-handler-proxy-4.1.52.Final-6NB4Y860vUgG4lWh1TLIZQ8POy1NAGrW0hgS6UG0L9w.jar' is inaccessible.
Dec 23, 2021 3:54:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-23T15:54:47.353Z: Staged package netty-resolver-4.1.52.Final-butyojOXnLRbC01tRsWkEzLFfAz4qAOx78YNu-y6CaY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-resolver-4.1.52.Final-butyojOXnLRbC01tRsWkEzLFfAz4qAOx78YNu-y6CaY.jar' is inaccessible.
Dec 23, 2021 3:54:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-23T15:54:47.419Z: Staged package netty-transport-4.1.52.Final-mj5vjA5V3jY-seoQ_ngXl-yjlOYhht8q4LTrK84LRUE.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-transport-4.1.52.Final-mj5vjA5V3jY-seoQ_ngXl-yjlOYhht8q4LTrK84LRUE.jar' is inaccessible.
Dec 23, 2021 3:54:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-23T15:54:47.480Z: Staged package opencensus-api-0.28.0-DBcj8_bTBhMjhFzouIs1_dpQCBLgp1uOtfzErYyHGpU.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/opencensus-api-0.28.0-DBcj8_bTBhMjhFzouIs1_dpQCBLgp1uOtfzErYyHGpU.jar' is inaccessible.
Dec 23, 2021 3:54:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-23T15:54:47.582Z: Staged package paranamer-2.8-aIyxGKYCHYGROOhVIIyVYDFoi-S0eiS7YVvsxjrO3wc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/paranamer-2.8-aIyxGKYCHYGROOhVIIyVYDFoi-S0eiS7YVvsxjrO3wc.jar' is inaccessible.
Dec 23, 2021 3:54:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-23T15:54:47.630Z: Staged package perfmark-api-0.23.0-xwW1wQwY_zAyuegXQrwvaw5WB_am38DIrQz_ddSRMEI.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/perfmark-api-0.23.0-xwW1wQwY_zAyuegXQrwvaw5WB_am38DIrQz_ddSRMEI.jar' is inaccessible.
Dec 23, 2021 3:54:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-23T15:54:48.075Z: Staged package snappy-java-1.1.8.4-JMTR_B6J4HgzGrj0AamcrWhZm95KLkUWBCy1SMUbHD4.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/snappy-java-1.1.8.4-JMTR_B6J4HgzGrj0AamcrWhZm95KLkUWBCy1SMUbHD4.jar' is inaccessible.
Dec 23, 2021 3:54:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-12-23T15:54:48.208Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Dec 23, 2021 3:57:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-12-23T15:57:47.907Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Dec 23, 2021 4:00:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-23T16:00:34.026Z: Cancel request is committed for workflow job: 2021-12-23_04_45_28-14917756070590393734.
Dec 23, 2021 4:00:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-23T16:00:34.068Z: Cleaning up.
Dec 23, 2021 4:00:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-23T16:00:34.167Z: Stopping **** pool...
Dec 23, 2021 4:00:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-23T16:00:34.229Z: Stopping **** pool...
Dec 23, 2021 4:02:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-23T16:02:52.348Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Dec 23, 2021 4:02:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-23T16:02:52.396Z: Worker pool stopped.
Dec 23, 2021 4:02:58 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-12-23_04_45_28-14917756070590393734 finished with status CANCELLED.
Load test results for test (ID): 6ad7f11c-0546-4785-a691-76f2243e8cf8 and timestamp: 2021-12-23T12:45:23.551000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11553.885
dataflow_v2_java11_total_bytes_count              2.6876774E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211223124335
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:c80958524a8555af812cc96f92bca50aba836ec2a12a934b4f0c5698bba2c51d
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211223124335]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:c80958524a8555af812cc96f92bca50aba836ec2a12a934b4f0c5698bba2c51d]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211223124335] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:c80958524a8555af812cc96f92bca50aba836ec2a12a934b4f0c5698bba2c51d])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:c80958524a8555af812cc96f92bca50aba836ec2a12a934b4f0c5698bba2c51d
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:c80958524a8555af812cc96f92bca50aba836ec2a12a934b4f0c5698bba2c51d
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:c80958524a8555af812cc96f92bca50aba836ec2a12a934b4f0c5698bba2c51d].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 41s
79 actionable tasks: 49 executed, 28 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/46awwn3jep3ve

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #188

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/188/display/redirect?page=changes>

Changes:

[rsl4] [BEAM-13397] Bump numpy to 1.21 for M1 chip

[rsl4] re-generate requirements.txt

[Kyle Weaver] [BEAM-13498] Sickbay portable Flink testProcessElementSkew.

[stranniknm] [BEAM-13502]: fix loading example on embedded version

[noreply] [BEAM-13430] Start the process of upgrading the Gradle 7. (#16292)

[noreply] [BEAM-13430] Introduce new testRuntimeMigration configuration that

[noreply] Update flink cluster to use a supported dataproc version (1.2 -> 1.5)

[Robert Bradshaw] Add microbenchmark for row coder.

[noreply] [BEAM-13499] Add warning about hcatalog to release notes and javadoc

[Robert Bradshaw] Compile RowCoderImpl with Cython.

[Robert Bradshaw] RowCoder optimizations.

[Robert Bradshaw] Further optimizations for common case of components in order.

[noreply] [BEAM-13430] Remove propdeps and replace with compileOnly (#16308)

[Robert Bradshaw] Optimize rows with null fields.

[noreply] Merge pull request #16304 from [BEAM-13491] [Playground] Examples'

[noreply] Merge pull request #16283 from [BEAM-13448] [Playground] track run code

[noreply] Merge pull request #16244 from [BEAM-13463] [Playground] add retries to

[noreply] Merge pull request #16241 from [BEAM-13440] [Playground] Implement

[noreply] [BEAM-13399] Add integration test for Go SDK expansion service JAR

[noreply] [BEAM-13421] Fix bug with xs called with non-tuple key (#16258)

[Robert Bradshaw] RowCoder lint.


------------------------------------------
[...truncated 41.55 KB...]
88fc27550592: Preparing
fff952342f20: Preparing
7adb2b7f3536: Preparing
cca1320cf2c3: Preparing
06f21a01682d: Preparing
6ed5b4f6ee85: Preparing
3bb5258f46d2: Preparing
832e177bb500: Preparing
f9e18e59a565: Preparing
26a504e63be4: Preparing
8bf42db0de72: Preparing
7adb2b7f3536: Waiting
31892cc314cb: Preparing
cca1320cf2c3: Waiting
11936051f93b: Preparing
8bf42db0de72: Waiting
fff952342f20: Waiting
11936051f93b: Waiting
31892cc314cb: Waiting
26a504e63be4: Waiting
fc48acd2c1e4: Waiting
88fc27550592: Waiting
3bb5258f46d2: Waiting
06f21a01682d: Waiting
6ed5b4f6ee85: Waiting
832e177bb500: Waiting
f9e18e59a565: Waiting
c78108382bb2: Pushed
45d5eb91a523: Pushed
006560ebe63d: Pushed
fc48acd2c1e4: Pushed
fff952342f20: Pushed
c11675e674af: Pushed
38ae2c2a20be: Pushed
7adb2b7f3536: Pushed
3bb5258f46d2: Layer already exists
832e177bb500: Layer already exists
f9e18e59a565: Layer already exists
26a504e63be4: Layer already exists
8bf42db0de72: Layer already exists
31892cc314cb: Layer already exists
11936051f93b: Layer already exists
06f21a01682d: Pushed
6ed5b4f6ee85: Pushed
88fc27550592: Pushed
cca1320cf2c3: Pushed
20211222124332: digest: sha256:d898f6335789a20ed87e2f7353981dd4fff41d7f0f407cf54300b81d80f0265f size: 4311

> Task :sdks:java:testing:load-tests:run
Dec 22, 2021 12:45:24 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Dec 22, 2021 12:45:25 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 203 files. Enable logging at DEBUG level to see which files will be staged.
Dec 22, 2021 12:45:25 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Dec 22, 2021 12:45:25 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Dec 22, 2021 12:45:28 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 203 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Dec 22, 2021 12:45:28 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 203 files cached, 0 files newly uploaded in 0 seconds
Dec 22, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Dec 22, 2021 12:45:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <113691 bytes, hash be6997fb8519d4f87a7ec006563219d0e65688cb86e954f47353a2a2b896c808> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-vmmX-4UZ1Ph6fsAGVjIZ0OZWiMuG6VT0c1OioriWyAg.pb
Dec 22, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Dec 22, 2021 12:45:30 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12704e15, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@512575e9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6f1a16fe, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2373ad99, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@33634f04, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4993febc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7601bc96, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@48a0c8aa, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6192a5d5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3722c145, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4cbc2e3b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2975a9e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@765ffb14, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@57562473, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7a360554, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@424de326, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4bc33720, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2dd0f797, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@67064bdc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4a7fd0c9]
Dec 22, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Dec 22, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Dec 22, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Dec 22, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Dec 22, 2021 12:45:30 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@444f44c5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@303f1234, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@24d61e4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2149594a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1f1e58ca, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@57f847af, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ed34ef5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@553bc36c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@380e1909, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2d5ef498, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@95eb320, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2f521c4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4afbb6c2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@10db6131, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6c6017b9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4730e0f0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@506a1372, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1b7332a7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77c233af, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@37b56ac7]
Dec 22, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Dec 22, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Dec 22, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Dec 22, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Dec 22, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Dec 22, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Dec 22, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Dec 22, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Dec 22, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Dec 22, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Dec 22, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Dec 22, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.36.0-SNAPSHOT
Dec 22, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-12-22_04_45_30-12883701128565853341?project=apache-beam-testing
Dec 22, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-12-22_04_45_30-12883701128565853341
Dec 22, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-12-22_04_45_30-12883701128565853341
Dec 22, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-12-22T12:45:37.892Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-12-jtjp. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Dec 22, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-22T12:45:42.037Z: Worker configuration: e2-standard-2 in us-central1-a.
Dec 22, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-22T12:45:42.841Z: Expanding SplittableParDo operations into optimizable parts.
Dec 22, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-22T12:45:42.882Z: Expanding CollectionToSingleton operations into optimizable parts.
Dec 22, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-22T12:45:42.970Z: Expanding CoGroupByKey operations into optimizable parts.
Dec 22, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-22T12:45:43.042Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Dec 22, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-22T12:45:43.073Z: Expanding GroupByKey operations into streaming Read/Write steps
Dec 22, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-22T12:45:43.138Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Dec 22, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-22T12:45:43.242Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Dec 22, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-22T12:45:43.276Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Dec 22, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-22T12:45:43.300Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Dec 22, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-22T12:45:43.322Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Dec 22, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-22T12:45:43.352Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Dec 22, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-22T12:45:43.378Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Dec 22, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-22T12:45:43.405Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Dec 22, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-22T12:45:43.438Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Dec 22, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-22T12:45:43.464Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Dec 22, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-22T12:45:43.499Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Dec 22, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-22T12:45:43.531Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Dec 22, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-22T12:45:43.569Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Dec 22, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-22T12:45:43.592Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Dec 22, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-22T12:45:43.617Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Dec 22, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-22T12:45:43.642Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Dec 22, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-22T12:45:43.679Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Dec 22, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-22T12:45:43.701Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Dec 22, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-22T12:45:43.727Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Dec 22, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-22T12:45:43.750Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Dec 22, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-22T12:45:43.777Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Dec 22, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-22T12:45:43.804Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Dec 22, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-22T12:45:43.836Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Dec 22, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-22T12:45:43.870Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Dec 22, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-22T12:45:44.248Z: Starting 5 ****s in us-central1-a...
Dec 22, 2021 12:46:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-22T12:46:04.742Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Dec 22, 2021 12:46:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-22T12:46:42.806Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Dec 22, 2021 12:47:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-22T12:47:38.165Z: Workers have started successfully.
Dec 22, 2021 12:47:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-22T12:47:38.200Z: Workers have started successfully.
Dec 22, 2021 2:03:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-22T14:03:47.356Z: Staged package kafka-clients-2.4.1-or4Vrm01S3aXE64MuzGE8iWqFvyYIFseskr9n8KjYNk.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/kafka-clients-2.4.1-or4Vrm01S3aXE64MuzGE8iWqFvyYIFseskr9n8KjYNk.jar' is inaccessible.
Dec 22, 2021 2:03:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-22T14:03:47.491Z: Staged package lz4-java-1.6.0-0ilUWqKx1SA8h2YUvbz_yswwNpf0-PJvdk4dbB7S5BY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/lz4-java-1.6.0-0ilUWqKx1SA8h2YUvbz_yswwNpf0-PJvdk4dbB7S5BY.jar' is inaccessible.
Dec 22, 2021 2:03:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-12-22T14:03:48.331Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Dec 22, 2021 4:00:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-22T16:00:32.342Z: Cancel request is committed for workflow job: 2021-12-22_04_45_30-12883701128565853341.
Dec 22, 2021 4:00:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-22T16:00:32.384Z: Cleaning up.
Dec 22, 2021 4:00:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-22T16:00:32.458Z: Stopping **** pool...
Dec 22, 2021 4:00:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-22T16:00:32.511Z: Stopping **** pool...
Dec 22, 2021 4:03:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-22T16:03:01.639Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Dec 22, 2021 4:03:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-22T16:03:01.679Z: Worker pool stopped.
Dec 22, 2021 4:03:06 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-12-22_04_45_30-12883701128565853341 finished with status CANCELLED.
Load test results for test (ID): b43f391b-5fc6-4dd1-993b-c3158d29c147 and timestamp: 2021-12-22T12:45:25.358000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11513.358
dataflow_v2_java11_total_bytes_count             2.52363928E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211222124332
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:d898f6335789a20ed87e2f7353981dd4fff41d7f0f407cf54300b81d80f0265f
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211222124332]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:d898f6335789a20ed87e2f7353981dd4fff41d7f0f407cf54300b81d80f0265f]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211222124332] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:d898f6335789a20ed87e2f7353981dd4fff41d7f0f407cf54300b81d80f0265f])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:d898f6335789a20ed87e2f7353981dd4fff41d7f0f407cf54300b81d80f0265f
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:d898f6335789a20ed87e2f7353981dd4fff41d7f0f407cf54300b81d80f0265f
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:d898f6335789a20ed87e2f7353981dd4fff41d7f0f407cf54300b81d80f0265f].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 51s
79 actionable tasks: 48 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/5uqocsbtfisw4

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #187

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/187/display/redirect?page=changes>

Changes:

[benjamin.gonzalez] [BEAM-13088] Add parameter tempWriteDataset to BigQueryIO to store temp

[stranniknm] [BEAM-13466]: sort categories and examples by name

[aydar.zaynutdinov] [BEAM-13485][Playground]

[alexander.zhuravlev] [BEAM-13476] Changed Timeout Error Text

[alexander.zhuravlev] [BEAM-13474] Changed 'Playground' logo text color

[mmack] [BEAM-13443] Avoid blocking put to Kinesis records queue to shutdown

[noreply] Merge pull request #16278 from [BEAM-13479] [Playground] Change logic

[noreply] Merge pull request #16281 from [BEAM-13475] [Playground] [Bugfix] Error

[noreply] Merge pull request #16279 from [BEAM-13473] [Playground] [Bugfix] Reset

[noreply] Merge pull request #16232 from [BEAM-13285][Playground] Add steps to

[noreply] [BEAM-13399] Add check for dev versions of JARs to download code

[noreply] Merge pull request #16122 from [BEAM-13345] [Playground] add resizable

[benjamin.gonzalez] [BEAM-13088] Make tempDataset final

[noreply] Merge pull request #16170 from [BEAM-13411][Playground] Add getting of

[noreply] Merge pull request #16172 from [BEAM-13417] [Playground] Add java

[noreply] Merge pull request #16240 from [BEAM-13465][Playground] Change error

[noreply] Merge pull request #16234 from [BEAM-13461][Playground] [Bugfix] Error

[noreply] Merge pull request #15489 from [BEAM-12865] Allow custom batch_duration

[noreply] [BEAM-13483] Increase timeout of Java Examples Dataflow suite. (#16226)

[Kyle Weaver] [BEAM-13496] Upgrade Flink runner to include log4j patches.

[Kyle Weaver] [BEAM-13497] Correct class name in Flink tests.

[noreply] Pin transitive log4j dependencies to 2.17.0 in :sdks:java:io:hcatalog

[noreply] [BEAM-13434] Bump solr to 8.11.1 (#16296)

[noreply] Use a patched shadow 6.1.0 plugin using Log4j 2.16.0 (#16269)

[noreply] [BEAM-12830] Replace GoGradle plugin with Shell Scripts. (#16291)


------------------------------------------
[...truncated 46.42 KB...]
The push refers to repository [us.gcr.io/apache-beam-testing/java-postcommit-it/java]
40f0e4a6936f: Preparing
fb75e428a79a: Preparing
96bcdbe57bfa: Preparing
a2799f6bf026: Preparing
b8a1873d363d: Preparing
bd22718b7352: Preparing
195254c02ade: Preparing
03a3183a1fe6: Preparing
876cfdbf9614: Preparing
ef26e774f41b: Preparing
0ed0430dd4ec: Preparing
6fb68bd9178f: Preparing
5c81f9330d99: Preparing
927f9fcef4cf: Preparing
a81f1846a0d2: Preparing
3b441d7cb46b: Preparing
d3710de04cb3: Preparing
91f7336bbfff: Preparing
e2e8c39e0f77: Preparing
927f9fcef4cf: Waiting
6fb68bd9178f: Waiting
0ed0430dd4ec: Waiting
ef26e774f41b: Waiting
5c81f9330d99: Waiting
876cfdbf9614: Waiting
195254c02ade: Waiting
a81f1846a0d2: Waiting
3b441d7cb46b: Waiting
bd22718b7352: Waiting
03a3183a1fe6: Waiting
e2e8c39e0f77: Waiting
d3710de04cb3: Waiting
b8a1873d363d: Pushed
96bcdbe57bfa: Pushed
fb75e428a79a: Pushed
40f0e4a6936f: Pushed
bd22718b7352: Pushed
a2799f6bf026: Pushed
03a3183a1fe6: Pushed
876cfdbf9614: Pushed
5c81f9330d99: Layer already exists
927f9fcef4cf: Layer already exists
a81f1846a0d2: Layer already exists
0ed0430dd4ec: Pushed
3b441d7cb46b: Layer already exists
d3710de04cb3: Layer already exists
91f7336bbfff: Layer already exists
e2e8c39e0f77: Layer already exists
195254c02ade: Pushed
6fb68bd9178f: Pushed
ef26e774f41b: Pushed
20211221124332: digest: sha256:672d4bf1af73bfc4fa7a079394c5343c38f3e45a87dc95cfd4ab51152a93d7a8 size: 4311

> Task :sdks:java:testing:load-tests:run
Dec 21, 2021 12:45:22 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Dec 21, 2021 12:45:23 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 203 files. Enable logging at DEBUG level to see which files will be staged.
Dec 21, 2021 12:45:23 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Dec 21, 2021 12:45:23 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Dec 21, 2021 12:45:26 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 203 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Dec 21, 2021 12:45:26 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 203 files cached, 0 files newly uploaded in 0 seconds
Dec 21, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Dec 21, 2021 12:45:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <113691 bytes, hash 94551b66a3d335eb1c3e0ca8fabb2e46bc2703bbd8d63943543bccba998f8a5e> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-lFUbZqPTNescPgyo-rsuRrwnA7vY1jlDVDvMupmPil4.pb
Dec 21, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Dec 21, 2021 12:45:28 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@429f7919, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4a2929a4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@cda6019, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@797c3c3b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4012d5bc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4375b013, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1cf0cacc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4f5b08d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@529c2a9a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c87fdf2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26bbe604, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@fe34b86, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c98781a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3f736a16, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4601203a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@53abfc07, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2c8c16c0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@80bfa9d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@47c40b56, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4b039c6d]
Dec 21, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Dec 21, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Dec 21, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Dec 21, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Dec 21, 2021 12:45:28 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@67de7a99, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@795f5d51, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@34aeacd1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@54067fdc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4098dd77, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43aeb5e0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2274160, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@65383667, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@63cd2cd2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@557a84fe, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6deee370, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@49c17ba4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43e065f2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@423c5404, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5853ca50, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1a0d96a5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5a02bfe3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7a3e5cd3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c79088e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4a37191a]
Dec 21, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Dec 21, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Dec 21, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Dec 21, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Dec 21, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Dec 21, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Dec 21, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Dec 21, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Dec 21, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Dec 21, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Dec 21, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Dec 21, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.36.0-SNAPSHOT
Dec 21, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-12-21_04_45_29-2661965857973093019?project=apache-beam-testing
Dec 21, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-12-21_04_45_29-2661965857973093019
Dec 21, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-12-21_04_45_29-2661965857973093019
Dec 21, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-12-21T12:45:38.227Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-12-lp8b. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Dec 21, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-21T12:45:44.136Z: Worker configuration: e2-standard-2 in us-central1-a.
Dec 21, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-21T12:45:44.843Z: Expanding SplittableParDo operations into optimizable parts.
Dec 21, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-21T12:45:44.872Z: Expanding CollectionToSingleton operations into optimizable parts.
Dec 21, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-21T12:45:44.956Z: Expanding CoGroupByKey operations into optimizable parts.
Dec 21, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-21T12:45:45.015Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Dec 21, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-21T12:45:45.046Z: Expanding GroupByKey operations into streaming Read/Write steps
Dec 21, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-21T12:45:45.111Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Dec 21, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-21T12:45:45.213Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Dec 21, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-21T12:45:45.238Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Dec 21, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-21T12:45:45.264Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Dec 21, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-21T12:45:45.298Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Dec 21, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-21T12:45:45.335Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Dec 21, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-21T12:45:45.393Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Dec 21, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-21T12:45:45.426Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Dec 21, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-21T12:45:45.459Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Dec 21, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-21T12:45:45.482Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Dec 21, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-21T12:45:45.515Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Dec 21, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-21T12:45:45.546Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Dec 21, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-21T12:45:45.594Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Dec 21, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-21T12:45:45.628Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Dec 21, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-21T12:45:45.664Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Dec 21, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-21T12:45:45.688Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Dec 21, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-21T12:45:45.722Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Dec 21, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-21T12:45:45.756Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Dec 21, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-21T12:45:45.788Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Dec 21, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-21T12:45:45.822Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Dec 21, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-21T12:45:45.858Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Dec 21, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-21T12:45:45.881Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Dec 21, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-21T12:45:45.916Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Dec 21, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-21T12:45:45.967Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Dec 21, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-21T12:45:46.297Z: Starting 5 ****s in us-central1-a...
Dec 21, 2021 12:46:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-21T12:46:06.171Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Dec 21, 2021 12:46:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-21T12:46:37.688Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Dec 21, 2021 12:47:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-21T12:47:39.877Z: Workers have started successfully.
Dec 21, 2021 12:47:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-21T12:47:39.907Z: Workers have started successfully.
Dec 21, 2021 4:00:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-21T16:00:34.361Z: Cancel request is committed for workflow job: 2021-12-21_04_45_29-2661965857973093019.
Dec 21, 2021 4:00:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-21T16:00:34.413Z: Cleaning up.
Dec 21, 2021 4:00:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-21T16:00:34.481Z: Stopping **** pool...
Dec 21, 2021 4:00:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-21T16:00:34.534Z: Stopping **** pool...
Dec 21, 2021 4:03:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-21T16:03:07.742Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Dec 21, 2021 4:03:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-21T16:03:07.784Z: Worker pool stopped.
Dec 21, 2021 4:03:15 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-12-21_04_45_29-2661965857973093019 finished with status CANCELLED.
Load test results for test (ID): d613daa7-5758-459e-a0f6-4fa3f16ead4a and timestamp: 2021-12-21T12:45:23.430000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11544.156
dataflow_v2_java11_total_bytes_count             2.48444433E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211221124332
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:672d4bf1af73bfc4fa7a079394c5343c38f3e45a87dc95cfd4ab51152a93d7a8
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211221124332]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:672d4bf1af73bfc4fa7a079394c5343c38f3e45a87dc95cfd4ab51152a93d7a8]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211221124332] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:672d4bf1af73bfc4fa7a079394c5343c38f3e45a87dc95cfd4ab51152a93d7a8])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:672d4bf1af73bfc4fa7a079394c5343c38f3e45a87dc95cfd4ab51152a93d7a8
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:672d4bf1af73bfc4fa7a079394c5343c38f3e45a87dc95cfd4ab51152a93d7a8
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:672d4bf1af73bfc4fa7a079394c5343c38f3e45a87dc95cfd4ab51152a93d7a8].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 20m 11s
99 actionable tasks: 68 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/4wppwxn4dvu4e

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #186

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/186/display/redirect?page=changes>

Changes:

[mmack] [BEAM-13233] Replace AWS API used to list shards from DescribeStream to


------------------------------------------
[...truncated 94.14 KB...]
  File "/usr/lib/python3.8/urllib/request.py", line 525, in open
    response = self._open(req, data)
  File "/usr/lib/python3.8/urllib/request.py", line 542, in _open
    result = self._call_chain(self.handle_open, protocol, protocol +
  File "/usr/lib/python3.8/urllib/request.py", line 502, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.8/urllib/request.py", line 1383, in http_open
    return self.do_open(http.client.HTTPConnection, req)
  File "/usr/lib/python3.8/urllib/request.py", line 1357, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error [Errno 101] Network is unreachable>
ERROR:root:Invalid url for checkstyle-8.23: http://www.gnu.org/licenses/old-licenses/lgpl-2.1.txt. Retrying...
Traceback (most recent call last):
  File "/usr/lib/python3.8/urllib/request.py", line 1354, in do_open
    h.request(req.get_method(), req.selector, req.data, headers,
  File "/usr/lib/python3.8/http/client.py", line 1252, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/lib/python3.8/http/client.py", line 1298, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.8/http/client.py", line 1247, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.8/http/client.py", line 1007, in _send_output
    self.send(msg)
  File "/usr/lib/python3.8/http/client.py", line 947, in send
    self.connect()
  File "/usr/lib/python3.8/http/client.py", line 918, in connect
    self.sock = self._create_connection(
  File "/usr/lib/python3.8/socket.py", line 808, in create_connection
    raise err
  File "/usr/lib/python3.8/socket.py", line 796, in create_connection
    sock.connect(sa)
OSError: [Errno 101] Network is unreachable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src/sdks/java/container/license_scripts/pull_licenses_java.py",> line 53, in pull_from_url
    url_read = urlopen(url)
  File "/usr/lib/python3.8/urllib/request.py", line 222, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib/python3.8/urllib/request.py", line 525, in open
    response = self._open(req, data)
  File "/usr/lib/python3.8/urllib/request.py", line 542, in _open
    result = self._call_chain(self.handle_open, protocol, protocol +
  File "/usr/lib/python3.8/urllib/request.py", line 502, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.8/urllib/request.py", line 1383, in http_open
    return self.do_open(http.client.HTTPConnection, req)
  File "/usr/lib/python3.8/urllib/request.py", line 1357, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error [Errno 101] Network is unreachable>
ERROR:root:Invalid url for jFormatString-3.0.0: http://www.gnu.org/licenses/lgpl.html. Retrying...
Traceback (most recent call last):
  File "/usr/lib/python3.8/urllib/request.py", line 1354, in do_open
    h.request(req.get_method(), req.selector, req.data, headers,
  File "/usr/lib/python3.8/http/client.py", line 1252, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/lib/python3.8/http/client.py", line 1298, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.8/http/client.py", line 1247, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.8/http/client.py", line 1007, in _send_output
    self.send(msg)
  File "/usr/lib/python3.8/http/client.py", line 947, in send
    self.connect()
  File "/usr/lib/python3.8/http/client.py", line 1414, in connect
    super().connect()
  File "/usr/lib/python3.8/http/client.py", line 918, in connect
    self.sock = self._create_connection(
  File "/usr/lib/python3.8/socket.py", line 808, in create_connection
    raise err
  File "/usr/lib/python3.8/socket.py", line 796, in create_connection
    sock.connect(sa)
OSError: [Errno 101] Network is unreachable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src/sdks/java/container/license_scripts/pull_licenses_java.py",> line 53, in pull_from_url
    url_read = urlopen(url)
  File "/usr/lib/python3.8/urllib/request.py", line 222, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib/python3.8/urllib/request.py", line 525, in open
    response = self._open(req, data)
  File "/usr/lib/python3.8/urllib/request.py", line 542, in _open
    result = self._call_chain(self.handle_open, protocol, protocol +
  File "/usr/lib/python3.8/urllib/request.py", line 502, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.8/urllib/request.py", line 1397, in https_open
    return self.do_open(http.client.HTTPSConnection, req,
  File "/usr/lib/python3.8/urllib/request.py", line 1357, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error [Errno 101] Network is unreachable>
ERROR:root:Invalid url for spotbugs-annotations-4.0.6: https://www.gnu.org/licenses/old-licenses/lgpl-2.1.en.html. Retrying...
Traceback (most recent call last):
  File "/usr/lib/python3.8/urllib/request.py", line 1354, in do_open
    h.request(req.get_method(), req.selector, req.data, headers,
  File "/usr/lib/python3.8/http/client.py", line 1252, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/lib/python3.8/http/client.py", line 1298, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.8/http/client.py", line 1247, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.8/http/client.py", line 1007, in _send_output
    self.send(msg)
  File "/usr/lib/python3.8/http/client.py", line 947, in send
    self.connect()
  File "/usr/lib/python3.8/http/client.py", line 918, in connect
    self.sock = self._create_connection(
  File "/usr/lib/python3.8/socket.py", line 808, in create_connection
    raise err
  File "/usr/lib/python3.8/socket.py", line 796, in create_connection
    sock.connect(sa)
OSError: [Errno 101] Network is unreachable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src/sdks/java/container/license_scripts/pull_licenses_java.py",> line 53, in pull_from_url
    url_read = urlopen(url)
  File "/usr/lib/python3.8/urllib/request.py", line 222, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib/python3.8/urllib/request.py", line 525, in open
    response = self._open(req, data)
  File "/usr/lib/python3.8/urllib/request.py", line 542, in _open
    result = self._call_chain(self.handle_open, protocol, protocol +
  File "/usr/lib/python3.8/urllib/request.py", line 502, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.8/urllib/request.py", line 1383, in http_open
    return self.do_open(http.client.HTTPConnection, req)
  File "/usr/lib/python3.8/urllib/request.py", line 1357, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error [Errno 101] Network is unreachable>
ERROR:root:Invalid url for checkstyle-8.23: http://www.gnu.org/licenses/old-licenses/lgpl-2.1.txt after 9 retries.
Traceback (most recent call last):
  File "/usr/lib/python3.8/urllib/request.py", line 1354, in do_open
    h.request(req.get_method(), req.selector, req.data, headers,
  File "/usr/lib/python3.8/http/client.py", line 1252, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/lib/python3.8/http/client.py", line 1298, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.8/http/client.py", line 1247, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.8/http/client.py", line 1007, in _send_output
    self.send(msg)
  File "/usr/lib/python3.8/http/client.py", line 947, in send
    self.connect()
  File "/usr/lib/python3.8/http/client.py", line 918, in connect
    self.sock = self._create_connection(
  File "/usr/lib/python3.8/socket.py", line 808, in create_connection
    raise err
  File "/usr/lib/python3.8/socket.py", line 796, in create_connection
    sock.connect(sa)
OSError: [Errno 101] Network is unreachable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src/sdks/java/container/license_scripts/pull_licenses_java.py",> line 53, in pull_from_url
    url_read = urlopen(url)
  File "/usr/lib/python3.8/urllib/request.py", line 222, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib/python3.8/urllib/request.py", line 525, in open
    response = self._open(req, data)
  File "/usr/lib/python3.8/urllib/request.py", line 542, in _open
    result = self._call_chain(self.handle_open, protocol, protocol +
  File "/usr/lib/python3.8/urllib/request.py", line 502, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.8/urllib/request.py", line 1383, in http_open
    return self.do_open(http.client.HTTPConnection, req)
  File "/usr/lib/python3.8/urllib/request.py", line 1357, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error [Errno 101] Network is unreachable>
ERROR:root:Invalid url for jFormatString-3.0.0: http://www.gnu.org/licenses/lgpl.html after 9 retries.
Traceback (most recent call last):
  File "/usr/lib/python3.8/urllib/request.py", line 1354, in do_open
    h.request(req.get_method(), req.selector, req.data, headers,
  File "/usr/lib/python3.8/http/client.py", line 1252, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/lib/python3.8/http/client.py", line 1298, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.8/http/client.py", line 1247, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.8/http/client.py", line 1007, in _send_output
    self.send(msg)
  File "/usr/lib/python3.8/http/client.py", line 947, in send
    self.connect()
  File "/usr/lib/python3.8/http/client.py", line 1414, in connect
    super().connect()
  File "/usr/lib/python3.8/http/client.py", line 918, in connect
    self.sock = self._create_connection(
  File "/usr/lib/python3.8/socket.py", line 808, in create_connection
    raise err
  File "/usr/lib/python3.8/socket.py", line 796, in create_connection
    sock.connect(sa)
OSError: [Errno 101] Network is unreachable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src/sdks/java/container/license_scripts/pull_licenses_java.py",> line 53, in pull_from_url
    url_read = urlopen(url)
  File "/usr/lib/python3.8/urllib/request.py", line 222, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib/python3.8/urllib/request.py", line 525, in open
    response = self._open(req, data)
  File "/usr/lib/python3.8/urllib/request.py", line 542, in _open
    result = self._call_chain(self.handle_open, protocol, protocol +
  File "/usr/lib/python3.8/urllib/request.py", line 502, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.8/urllib/request.py", line 1397, in https_open
    return self.do_open(http.client.HTTPSConnection, req,
  File "/usr/lib/python3.8/urllib/request.py", line 1357, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error [Errno 101] Network is unreachable>
ERROR:root:Invalid url for spotbugs-annotations-4.0.6: https://www.gnu.org/licenses/old-licenses/lgpl-2.1.en.html after 9 retries.
ERROR:root:['checkstyle-8.23', 'jFormatString-3.0.0', 'spotbugs-annotations-4.0.6']
ERROR:root:**************************************** Licenses were not able to be pulled automatically for some dependencies. Please search source code of the dependencies on the internet and add "license" and "notice" (if available) field to <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src/sdks/java/container/license_scripts/dep_urls_java.yaml> for each missing license. Dependency List: [checkstyle-8.23,jFormatString-3.0.0,spotbugs-annotations-4.0.6]
INFO:root:pull_licenses_java.py failed. It took 1217.460185 seconds with 16 threads.
Traceback (most recent call last):
  File "<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src/sdks/java/container/license_scripts/pull_licenses_java.py",> line 314, in <module>
    raise RuntimeError('{n} error(s) occurred.'.format(n=len(error_msg)),
RuntimeError: ('1 error(s) occurred.', ['**************************************** Licenses were not able to be pulled automatically for some dependencies. Please search source code of the dependencies on the internet and add "license" and "notice" (if available) field to <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src/sdks/java/container/license_scripts/dep_urls_java.yaml> for each missing license. Dependency List: [checkstyle-8.23,jFormatString-3.0.0,spotbugs-annotations-4.0.6]'])

> Task :sdks:java:container:pullLicenses FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:container:pullLicenses'.
> Process 'command './license_scripts/license_script.sh'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 20m 50s
95 actionable tasks: 64 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/2o4huixk7xtf4

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #185

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/185/display/redirect>

Changes:


------------------------------------------
[...truncated 47.79 KB...]

The push refers to repository [us.gcr.io/apache-beam-testing/java-postcommit-it/java]
94251b99fbf8: Preparing
cad9ca496af0: Preparing
1e4215f7537a: Preparing
853a66b06059: Preparing
a52a2a7ff3f4: Preparing
53a26582c8d4: Preparing
00e6dd15ed44: Preparing
c8d993d936fd: Preparing
52105e9cf5eb: Preparing
9867535ec95c: Preparing
a1d8393cccc8: Preparing
9d9cfabf0afd: Preparing
5c81f9330d99: Preparing
927f9fcef4cf: Preparing
a81f1846a0d2: Preparing
3b441d7cb46b: Preparing
d3710de04cb3: Preparing
91f7336bbfff: Preparing
e2e8c39e0f77: Preparing
52105e9cf5eb: Waiting
5c81f9330d99: Waiting
a1d8393cccc8: Waiting
9867535ec95c: Waiting
927f9fcef4cf: Waiting
9d9cfabf0afd: Waiting
e2e8c39e0f77: Waiting
a81f1846a0d2: Waiting
d3710de04cb3: Waiting
3b441d7cb46b: Waiting
00e6dd15ed44: Waiting
53a26582c8d4: Waiting
a52a2a7ff3f4: Pushed
cad9ca496af0: Pushed
1e4215f7537a: Pushed
853a66b06059: Pushed
94251b99fbf8: Pushed
c8d993d936fd: Pushed
00e6dd15ed44: Pushed
53a26582c8d4: Pushed
52105e9cf5eb: Pushed
5c81f9330d99: Layer already exists
927f9fcef4cf: Layer already exists
a81f1846a0d2: Layer already exists
3b441d7cb46b: Layer already exists
91f7336bbfff: Layer already exists
d3710de04cb3: Layer already exists
e2e8c39e0f77: Layer already exists
9d9cfabf0afd: Pushed
a1d8393cccc8: Pushed
9867535ec95c: Pushed
20211219124334: digest: sha256:3173322eaa58a48b69cacf223f1d87fbb7d8c293ef689b491d092edb1c24647d size: 4311

> Task :sdks:java:testing:load-tests:run
Dec 19, 2021 12:45:17 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Dec 19, 2021 12:45:18 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 203 files. Enable logging at DEBUG level to see which files will be staged.
Dec 19, 2021 12:45:18 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Dec 19, 2021 12:45:18 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Dec 19, 2021 12:45:21 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 203 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Dec 19, 2021 12:45:23 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 203 files cached, 0 files newly uploaded in 1 seconds
Dec 19, 2021 12:45:23 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Dec 19, 2021 12:45:23 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <113691 bytes, hash 3e48a11de31910b829385bc0ffb29fcc6abaee2b2471f2733373314c23e0bc9a> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-PkihHeMZELgpOFvA_7KfzGq67iskcfJzM3MxTCPgvJo.pb
Dec 19, 2021 12:45:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Dec 19, 2021 12:45:25 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@429f7919, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4a2929a4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@cda6019, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@797c3c3b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4012d5bc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4375b013, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1cf0cacc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4f5b08d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@529c2a9a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c87fdf2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26bbe604, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@fe34b86, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c98781a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3f736a16, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4601203a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@53abfc07, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2c8c16c0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@80bfa9d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@47c40b56, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4b039c6d]
Dec 19, 2021 12:45:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Dec 19, 2021 12:45:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Dec 19, 2021 12:45:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Dec 19, 2021 12:45:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Dec 19, 2021 12:45:25 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@67de7a99, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@795f5d51, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@34aeacd1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@54067fdc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4098dd77, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43aeb5e0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2274160, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@65383667, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@63cd2cd2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@557a84fe, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6deee370, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@49c17ba4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43e065f2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@423c5404, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5853ca50, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1a0d96a5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5a02bfe3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7a3e5cd3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c79088e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4a37191a]
Dec 19, 2021 12:45:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Dec 19, 2021 12:45:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Dec 19, 2021 12:45:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Dec 19, 2021 12:45:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Dec 19, 2021 12:45:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Dec 19, 2021 12:45:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Dec 19, 2021 12:45:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Dec 19, 2021 12:45:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Dec 19, 2021 12:45:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Dec 19, 2021 12:45:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Dec 19, 2021 12:45:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Dec 19, 2021 12:45:25 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.36.0-SNAPSHOT
Dec 19, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-12-19_04_45_25-8273944174357763602?project=apache-beam-testing
Dec 19, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-12-19_04_45_25-8273944174357763602
Dec 19, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-12-19_04_45_25-8273944174357763602
Dec 19, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-12-19T12:45:36.290Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-12-kr6o. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Dec 19, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-19T12:45:41.151Z: Worker configuration: e2-standard-2 in us-central1-a.
Dec 19, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-19T12:45:41.883Z: Expanding SplittableParDo operations into optimizable parts.
Dec 19, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-19T12:45:41.930Z: Expanding CollectionToSingleton operations into optimizable parts.
Dec 19, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-19T12:45:42.022Z: Expanding CoGroupByKey operations into optimizable parts.
Dec 19, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-19T12:45:42.093Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Dec 19, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-19T12:45:42.124Z: Expanding GroupByKey operations into streaming Read/Write steps
Dec 19, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-19T12:45:42.188Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Dec 19, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-19T12:45:42.295Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Dec 19, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-19T12:45:42.328Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Dec 19, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-19T12:45:42.379Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Dec 19, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-19T12:45:42.406Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Dec 19, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-19T12:45:42.441Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Dec 19, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-19T12:45:42.466Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Dec 19, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-19T12:45:42.507Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Dec 19, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-19T12:45:42.539Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Dec 19, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-19T12:45:42.568Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Dec 19, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-19T12:45:42.603Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Dec 19, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-19T12:45:42.636Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Dec 19, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-19T12:45:42.669Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Dec 19, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-19T12:45:42.705Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Dec 19, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-19T12:45:42.737Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Dec 19, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-19T12:45:42.763Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Dec 19, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-19T12:45:42.793Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Dec 19, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-19T12:45:42.816Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Dec 19, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-19T12:45:42.846Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Dec 19, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-19T12:45:42.884Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Dec 19, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-19T12:45:42.915Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Dec 19, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-19T12:45:42.947Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Dec 19, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-19T12:45:43.021Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Dec 19, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-19T12:45:43.055Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Dec 19, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-19T12:45:43.429Z: Starting 5 ****s in us-central1-a...
Dec 19, 2021 12:46:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-19T12:46:07.886Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Dec 19, 2021 12:46:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-19T12:46:35.714Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Dec 19, 2021 12:47:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-19T12:47:36.137Z: Workers have started successfully.
Dec 19, 2021 12:47:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-19T12:47:36.170Z: Workers have started successfully.
Dec 19, 2021 4:00:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-19T16:00:31.374Z: Cancel request is committed for workflow job: 2021-12-19_04_45_25-8273944174357763602.
Dec 19, 2021 4:00:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-19T16:00:31.447Z: Cleaning up.
Dec 19, 2021 4:00:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-19T16:00:31.505Z: Stopping **** pool...
Dec 19, 2021 4:00:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-19T16:00:31.552Z: Stopping **** pool...
Dec 19, 2021 4:03:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-19T16:03:06.724Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Dec 19, 2021 4:03:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-19T16:03:06.752Z: Worker pool stopped.
Dec 19, 2021 4:03:12 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-12-19_04_45_25-8273944174357763602 finished with status CANCELLED.
Load test results for test (ID): 758926bc-d3e1-44cd-a9ee-3af0cf403788 and timestamp: 2021-12-19T12:45:18.420000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11541.074
dataflow_v2_java11_total_bytes_count             2.67213413E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211219124334
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:3173322eaa58a48b69cacf223f1d87fbb7d8c293ef689b491d092edb1c24647d
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211219124334]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:3173322eaa58a48b69cacf223f1d87fbb7d8c293ef689b491d092edb1c24647d]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211219124334] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:3173322eaa58a48b69cacf223f1d87fbb7d8c293ef689b491d092edb1c24647d])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:3173322eaa58a48b69cacf223f1d87fbb7d8c293ef689b491d092edb1c24647d
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:3173322eaa58a48b69cacf223f1d87fbb7d8c293ef689b491d092edb1c24647d
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:3173322eaa58a48b69cacf223f1d87fbb7d8c293ef689b491d092edb1c24647d].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 55s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/lkfa7lxgqh4hg

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #184

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/184/display/redirect>

Changes:


------------------------------------------
[...truncated 48.53 KB...]
c5f64ca0958b: Preparing
5c81f9330d99: Preparing
927f9fcef4cf: Preparing
a81f1846a0d2: Preparing
3b441d7cb46b: Preparing
d3710de04cb3: Preparing
91f7336bbfff: Preparing
e2e8c39e0f77: Preparing
729213c8affb: Waiting
013a401b202f: Waiting
556d656d31ee: Waiting
42a2bbb24dda: Waiting
d60096506969: Waiting
91f7336bbfff: Waiting
a81f1846a0d2: Waiting
3b441d7cb46b: Waiting
c5f64ca0958b: Waiting
e2e8c39e0f77: Waiting
5c81f9330d99: Waiting
26add80d0857: Waiting
927f9fcef4cf: Waiting
5a9675b73172: Pushed
dafad1a5d5c8: Pushed
374dcdae9a1c: Pushed
729213c8affb: Pushed
b94371377b84: Pushed
f90fd49241ed: Pushed
013a401b202f: Pushed
42a2bbb24dda: Pushed
5c81f9330d99: Layer already exists
927f9fcef4cf: Layer already exists
a81f1846a0d2: Layer already exists
3b441d7cb46b: Layer already exists
d3710de04cb3: Layer already exists
91f7336bbfff: Layer already exists
e2e8c39e0f77: Layer already exists
556d656d31ee: Pushed
c5f64ca0958b: Pushed
26add80d0857: Pushed
d60096506969: Pushed
20211218124335: digest: sha256:d6a535b4748b1064dfaed5c40963664267214ce85025f9395ceed633d522792a size: 4311

> Task :sdks:java:testing:load-tests:run
Dec 18, 2021 12:45:59 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Dec 18, 2021 12:46:00 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 203 files. Enable logging at DEBUG level to see which files will be staged.
Dec 18, 2021 12:46:01 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Dec 18, 2021 12:46:01 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Dec 18, 2021 12:46:04 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 203 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Dec 18, 2021 12:46:04 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 203 files cached, 0 files newly uploaded in 0 seconds
Dec 18, 2021 12:46:04 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Dec 18, 2021 12:46:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <113691 bytes, hash 53b7448f8272c57275841216216b348c41d3ad246704ed53fca49c6fbeedc693> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-U7dEj4JyxXJ1hBIWIWs0jEHTrSRnBO1T_KScb77txpM.pb
Dec 18, 2021 12:46:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Dec 18, 2021 12:46:06 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@cda6019, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@797c3c3b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4012d5bc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4375b013, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1cf0cacc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4f5b08d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@529c2a9a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c87fdf2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26bbe604, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@fe34b86, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c98781a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3f736a16, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4601203a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@53abfc07, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2c8c16c0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@80bfa9d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@47c40b56, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4b039c6d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7f5b9db, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@507d64aa]
Dec 18, 2021 12:46:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Dec 18, 2021 12:46:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Dec 18, 2021 12:46:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Dec 18, 2021 12:46:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Dec 18, 2021 12:46:06 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@34aeacd1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@54067fdc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4098dd77, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43aeb5e0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2274160, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@65383667, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@63cd2cd2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@557a84fe, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6deee370, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@49c17ba4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43e065f2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@423c5404, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5853ca50, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1a0d96a5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5a02bfe3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7a3e5cd3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c79088e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4a37191a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5854a18, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@d5556bf]
Dec 18, 2021 12:46:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Dec 18, 2021 12:46:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Dec 18, 2021 12:46:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Dec 18, 2021 12:46:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Dec 18, 2021 12:46:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Dec 18, 2021 12:46:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Dec 18, 2021 12:46:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Dec 18, 2021 12:46:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Dec 18, 2021 12:46:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Dec 18, 2021 12:46:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Dec 18, 2021 12:46:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Dec 18, 2021 12:46:06 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.36.0-SNAPSHOT
Dec 18, 2021 12:46:07 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-12-18_04_46_06-16435623552373968227?project=apache-beam-testing
Dec 18, 2021 12:46:07 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-12-18_04_46_06-16435623552373968227
Dec 18, 2021 12:46:07 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-12-18_04_46_06-16435623552373968227
Dec 18, 2021 12:46:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-12-18T12:46:13.230Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-12-8oa2. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Dec 18, 2021 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-18T12:46:17.355Z: Worker configuration: e2-standard-2 in us-central1-a.
Dec 18, 2021 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-18T12:46:18.056Z: Expanding SplittableParDo operations into optimizable parts.
Dec 18, 2021 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-18T12:46:18.086Z: Expanding CollectionToSingleton operations into optimizable parts.
Dec 18, 2021 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-18T12:46:18.124Z: Expanding CoGroupByKey operations into optimizable parts.
Dec 18, 2021 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-18T12:46:18.174Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Dec 18, 2021 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-18T12:46:18.194Z: Expanding GroupByKey operations into streaming Read/Write steps
Dec 18, 2021 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-18T12:46:18.242Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Dec 18, 2021 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-18T12:46:18.314Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Dec 18, 2021 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-18T12:46:18.339Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Dec 18, 2021 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-18T12:46:18.360Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Dec 18, 2021 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-18T12:46:18.380Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Dec 18, 2021 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-18T12:46:18.403Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Dec 18, 2021 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-18T12:46:18.446Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Dec 18, 2021 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-18T12:46:18.474Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Dec 18, 2021 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-18T12:46:18.497Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Dec 18, 2021 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-18T12:46:18.518Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Dec 18, 2021 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-18T12:46:18.541Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Dec 18, 2021 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-18T12:46:18.582Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Dec 18, 2021 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-18T12:46:18.600Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Dec 18, 2021 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-18T12:46:18.622Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Dec 18, 2021 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-18T12:46:18.643Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Dec 18, 2021 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-18T12:46:18.667Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Dec 18, 2021 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-18T12:46:18.692Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Dec 18, 2021 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-18T12:46:18.738Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Dec 18, 2021 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-18T12:46:18.765Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Dec 18, 2021 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-18T12:46:18.784Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Dec 18, 2021 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-18T12:46:18.805Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Dec 18, 2021 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-18T12:46:18.830Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Dec 18, 2021 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-18T12:46:18.850Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Dec 18, 2021 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-18T12:46:18.894Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Dec 18, 2021 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-18T12:46:19.166Z: Starting 5 ****s in us-central1-a...
Dec 18, 2021 12:46:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-18T12:46:43.157Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Dec 18, 2021 12:47:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-18T12:47:04.744Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Dec 18, 2021 12:48:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-18T12:48:00.068Z: Workers have started successfully.
Dec 18, 2021 12:48:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-18T12:48:00.116Z: Workers have started successfully.
Dec 18, 2021 1:52:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-18T13:52:20.370Z: Staged package error_prone_annotations-2.3.4-uvfW6pfOYGxT4RtoVLpfLOfvXCTd3wr6GNEmC9JbACw.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/error_prone_annotations-2.3.4-uvfW6pfOYGxT4RtoVLpfLOfvXCTd3wr6GNEmC9JbACw.jar' is inaccessible.
Dec 18, 2021 1:52:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-12-18T13:52:22.825Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Dec 18, 2021 1:55:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-12-18T13:55:22.343Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Dec 18, 2021 1:58:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-18T13:58:20.435Z: Staged package error_prone_annotations-2.3.4-uvfW6pfOYGxT4RtoVLpfLOfvXCTd3wr6GNEmC9JbACw.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/error_prone_annotations-2.3.4-uvfW6pfOYGxT4RtoVLpfLOfvXCTd3wr6GNEmC9JbACw.jar' is inaccessible.
Dec 18, 2021 1:58:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-12-18T13:58:22.797Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Dec 18, 2021 2:01:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-12-18T14:01:24.722Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Dec 18, 2021 4:00:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-18T16:00:30.419Z: Cancel request is committed for workflow job: 2021-12-18_04_46_06-16435623552373968227.
Dec 18, 2021 4:00:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-18T16:00:30.477Z: Cleaning up.
Dec 18, 2021 4:00:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-18T16:00:30.569Z: Stopping **** pool...
Dec 18, 2021 4:00:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-18T16:00:30.615Z: Stopping **** pool...
Dec 18, 2021 4:02:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-18T16:02:57.261Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Dec 18, 2021 4:02:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-18T16:02:57.339Z: Worker pool stopped.
Dec 18, 2021 4:03:03 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-12-18_04_46_06-16435623552373968227 finished with status CANCELLED.
Load test results for test (ID): 031b0ca1-1ebc-4de9-a2d4-d7d3c4f5931b and timestamp: 2021-12-18T12:46:00.666000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11518.684
dataflow_v2_java11_total_bytes_count             3.08272235E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211218124335
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:d6a535b4748b1064dfaed5c40963664267214ce85025f9395ceed633d522792a
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211218124335]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:d6a535b4748b1064dfaed5c40963664267214ce85025f9395ceed633d522792a]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211218124335] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:d6a535b4748b1064dfaed5c40963664267214ce85025f9395ceed633d522792a])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:d6a535b4748b1064dfaed5c40963664267214ce85025f9395ceed633d522792a
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:d6a535b4748b1064dfaed5c40963664267214ce85025f9395ceed633d522792a
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:d6a535b4748b1064dfaed5c40963664267214ce85025f9395ceed633d522792a].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 45s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/g2qfngguyxniu

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #183

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/183/display/redirect?page=changes>

Changes:

[noreply] Merge pull request #16061 from [BEAM-13428] [Playground] Integrate

[noreply] Clarify CoGroupByKey creates Iterable, not list. (#16099)

[noreply] [BEAM-12931] Allow for DoFn#getAllowedTimestampSkew() when checking the

[noreply] [BEAM-13467] Properly handle null argument types for logical types.

[noreply] [BEAM-10277] Initial implementation for encoding position in Python

[noreply] [BEAM-11545] State & timer for batched RPC calls pattern (#13643)

[noreply] Automatically prune local images before building an RC. (#16238)

[noreply] Add verbose error messages to container-related scripts. (#16056)

[noreply] [BEAM-13456] Rollback #15890 to fix timeout in Java PostCommit (#16257)

[noreply] [BEAM-13015] Add a state backed iterable that can be mutated under

[noreply] [BEAM-13388] Update Cloud DLP after breaking changes. (#16236)

[noreply] [BEAM-13434] Bump google pubsublite on master. (#16265)

[mmack] [adhoc] Fix guava imports in tests


------------------------------------------
[...truncated 48.92 KB...]
d3710de04cb3: Preparing
91f7336bbfff: Preparing
e2e8c39e0f77: Preparing
a4ed61e3b241: Waiting
26aca54b886b: Waiting
6fd958ea90e6: Waiting
2eca0713fd62: Waiting
96d460f945b4: Waiting
5c81f9330d99: Waiting
de5518a41be7: Waiting
d3710de04cb3: Waiting
a4b5b297c9a6: Waiting
927f9fcef4cf: Waiting
91f7336bbfff: Waiting
3b441d7cb46b: Waiting
5f39dae7ca77: Pushed
f3c91d198746: Pushed
2b3e06b4bb3a: Pushed
5f2c7671d676: Pushed
d1b9b5402f54: Pushed
a4b5b297c9a6: Pushed
de5518a41be7: Pushed
2eca0713fd62: Pushed
a4ed61e3b241: Pushed
5c81f9330d99: Layer already exists
927f9fcef4cf: Layer already exists
a81f1846a0d2: Layer already exists
96d460f945b4: Pushed
d3710de04cb3: Layer already exists
3b441d7cb46b: Layer already exists
e2e8c39e0f77: Layer already exists
91f7336bbfff: Layer already exists
26aca54b886b: Pushed
6fd958ea90e6: Pushed
20211217124333: digest: sha256:3da14525b976f8e2f56ed6793ab3bd58b4a238bb2d965ae7e5f87ede13227d22 size: 4311

> Task :sdks:java:testing:load-tests:run
Dec 17, 2021 12:45:23 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Dec 17, 2021 12:45:23 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 203 files. Enable logging at DEBUG level to see which files will be staged.
Dec 17, 2021 12:45:24 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Dec 17, 2021 12:45:24 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Dec 17, 2021 12:45:26 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 203 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Dec 17, 2021 12:45:27 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 203 files cached, 0 files newly uploaded in 0 seconds
Dec 17, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Dec 17, 2021 12:45:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <113691 bytes, hash e72b00cfebc0709df0fda30f6949fce89db16b6fec385d9f81e73f700d4ad0e9> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-5ysAz-vAcJ3w_aMPaUn86J2xa2_sOF2fgec_cA1K0Ok.pb
Dec 17, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Dec 17, 2021 12:45:29 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@429f7919, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4a2929a4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@cda6019, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@797c3c3b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4012d5bc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4375b013, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1cf0cacc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4f5b08d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@529c2a9a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c87fdf2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26bbe604, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@fe34b86, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c98781a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3f736a16, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4601203a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@53abfc07, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2c8c16c0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@80bfa9d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@47c40b56, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4b039c6d]
Dec 17, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Dec 17, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Dec 17, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Dec 17, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Dec 17, 2021 12:45:29 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@67de7a99, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@795f5d51, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@34aeacd1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@54067fdc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4098dd77, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43aeb5e0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2274160, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@65383667, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@63cd2cd2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@557a84fe, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6deee370, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@49c17ba4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43e065f2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@423c5404, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5853ca50, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1a0d96a5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5a02bfe3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7a3e5cd3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c79088e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4a37191a]
Dec 17, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Dec 17, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Dec 17, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Dec 17, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Dec 17, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Dec 17, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Dec 17, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Dec 17, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Dec 17, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Dec 17, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Dec 17, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Dec 17, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.36.0-SNAPSHOT
Dec 17, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-12-17_04_45_29-12572704560890537700?project=apache-beam-testing
Dec 17, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-12-17_04_45_29-12572704560890537700
Dec 17, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-12-17_04_45_29-12572704560890537700
Dec 17, 2021 12:45:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-12-17T12:45:36.615Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-12-lbzj. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Dec 17, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-17T12:45:41.924Z: Worker configuration: e2-standard-2 in us-central1-a.
Dec 17, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-17T12:45:42.657Z: Expanding SplittableParDo operations into optimizable parts.
Dec 17, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-17T12:45:42.687Z: Expanding CollectionToSingleton operations into optimizable parts.
Dec 17, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-17T12:45:42.764Z: Expanding CoGroupByKey operations into optimizable parts.
Dec 17, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-17T12:45:42.980Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Dec 17, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-17T12:45:43.016Z: Expanding GroupByKey operations into streaming Read/Write steps
Dec 17, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-17T12:45:43.067Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Dec 17, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-17T12:45:43.166Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Dec 17, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-17T12:45:43.211Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Dec 17, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-17T12:45:43.266Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Dec 17, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-17T12:45:43.299Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Dec 17, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-17T12:45:43.335Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Dec 17, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-17T12:45:43.366Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Dec 17, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-17T12:45:43.401Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Dec 17, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-17T12:45:43.427Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Dec 17, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-17T12:45:43.465Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Dec 17, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-17T12:45:43.490Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Dec 17, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-17T12:45:43.519Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Dec 17, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-17T12:45:43.544Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Dec 17, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-17T12:45:43.579Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Dec 17, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-17T12:45:43.610Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Dec 17, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-17T12:45:43.643Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Dec 17, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-17T12:45:43.676Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Dec 17, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-17T12:45:43.710Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Dec 17, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-17T12:45:43.742Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Dec 17, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-17T12:45:43.776Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Dec 17, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-17T12:45:43.850Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Dec 17, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-17T12:45:43.885Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Dec 17, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-17T12:45:43.906Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Dec 17, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-17T12:45:43.950Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Dec 17, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-17T12:45:44.333Z: Starting 5 ****s in us-central1-a...
Dec 17, 2021 12:45:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-17T12:45:55.150Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Dec 17, 2021 12:46:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-17T12:46:29.848Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Dec 17, 2021 12:47:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-17T12:47:23.994Z: Workers have started successfully.
Dec 17, 2021 12:47:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-17T12:47:24.019Z: Workers have started successfully.
Dec 17, 2021 2:06:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-17T14:06:48.546Z: Staged package slf4j-api-1.7.30-zboHlk0btAoHYUhcax6ML4_Z6x0ZxTkorA1_lRAQXFc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/slf4j-api-1.7.30-zboHlk0btAoHYUhcax6ML4_Z6x0ZxTkorA1_lRAQXFc.jar' is inaccessible.
Dec 17, 2021 2:06:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-17T14:06:48.796Z: Staged package slf4j-jdk14-1.7.30-4PnbBJN49kZ5QXcUVJlSMyhft633LEZ-ZdryXmc6y6g.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/slf4j-jdk14-1.7.30-4PnbBJN49kZ5QXcUVJlSMyhft633LEZ-ZdryXmc6y6g.jar' is inaccessible.
Dec 17, 2021 2:06:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-17T14:06:48.981Z: Staged package xz-1.5-hvMPqHdfo6Ys2znR7XimAZFkwQWIZASNQsvuJE4m6EA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/xz-1.5-hvMPqHdfo6Ys2znR7XimAZFkwQWIZASNQsvuJE4m6EA.jar' is inaccessible.
Dec 17, 2021 2:06:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-12-17T14:06:49.043Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Dec 17, 2021 2:09:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-12-17T14:09:48.484Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Dec 17, 2021 2:12:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-17T14:12:48.335Z: Staged package slf4j-api-1.7.30-zboHlk0btAoHYUhcax6ML4_Z6x0ZxTkorA1_lRAQXFc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/slf4j-api-1.7.30-zboHlk0btAoHYUhcax6ML4_Z6x0ZxTkorA1_lRAQXFc.jar' is inaccessible.
Dec 17, 2021 2:12:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-17T14:12:48.378Z: Staged package slf4j-jdk14-1.7.30-4PnbBJN49kZ5QXcUVJlSMyhft633LEZ-ZdryXmc6y6g.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/slf4j-jdk14-1.7.30-4PnbBJN49kZ5QXcUVJlSMyhft633LEZ-ZdryXmc6y6g.jar' is inaccessible.
Dec 17, 2021 2:12:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-17T14:12:48.515Z: Staged package xz-1.5-hvMPqHdfo6Ys2znR7XimAZFkwQWIZASNQsvuJE4m6EA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/xz-1.5-hvMPqHdfo6Ys2znR7XimAZFkwQWIZASNQsvuJE4m6EA.jar' is inaccessible.
Dec 17, 2021 2:12:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-12-17T14:12:48.574Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Dec 17, 2021 4:00:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-17T16:00:26.163Z: Cancel request is committed for workflow job: 2021-12-17_04_45_29-12572704560890537700.
Dec 17, 2021 4:00:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-17T16:00:26.223Z: Cleaning up.
Dec 17, 2021 4:00:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-17T16:00:26.304Z: Stopping **** pool...
Dec 17, 2021 4:00:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-17T16:00:26.387Z: Stopping **** pool...
Dec 17, 2021 4:03:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-17T16:03:00.156Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Dec 17, 2021 4:03:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-17T16:03:00.224Z: Worker pool stopped.
Dec 17, 2021 4:03:11 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-12-17_04_45_29-12572704560890537700 finished with status CANCELLED.
Load test results for test (ID): 846c7e79-cc90-4b72-93cc-d692b6de4572 and timestamp: 2021-12-17T12:45:24.277000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11550.176
dataflow_v2_java11_total_bytes_count             2.08122015E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211217124333
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:3da14525b976f8e2f56ed6793ab3bd58b4a238bb2d965ae7e5f87ede13227d22
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211217124333]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:3da14525b976f8e2f56ed6793ab3bd58b4a238bb2d965ae7e5f87ede13227d22]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211217124333] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:3da14525b976f8e2f56ed6793ab3bd58b4a238bb2d965ae7e5f87ede13227d22])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:3da14525b976f8e2f56ed6793ab3bd58b4a238bb2d965ae7e5f87ede13227d22
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:3da14525b976f8e2f56ed6793ab3bd58b4a238bb2d965ae7e5f87ede13227d22
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:3da14525b976f8e2f56ed6793ab3bd58b4a238bb2d965ae7e5f87ede13227d22].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 55s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/txh6tnl4boxpk

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #182

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/182/display/redirect?page=changes>

Changes:

[thiagotnunes] [BEAM-12164] Add Spanner Change Stream DAOs

[noreply] [BEAM-13341] Serialize reflection parameter in AvroCoder cloud object

[noreply] Update grafana from 8.1.6 to 8.1.8

[noreply] [BEAM-13015] Update FakeBeamFnStateClient to generate elements that stop

[noreply] [BEAM-13218] Sickbay

[noreply] [BEAM-13399] Add infrastructure to start JARs from Go functions (#16214)


------------------------------------------
[...truncated 49.25 KB...]
--verbosity=error -- pull gcr.io/project-id/my-image`.

See: https://cloud.google.com/container-registry/docs/support/deprecation-notices#gcloud-docker

The push refers to repository [us.gcr.io/apache-beam-testing/java-postcommit-it/java]
979821cbbfd3: Preparing
b145012a2908: Preparing
7ed69227c2c7: Preparing
7187d543cc5b: Preparing
d35cae3a437e: Preparing
9000f5ee7237: Preparing
b85415acf7ec: Preparing
a52437ae58f3: Preparing
6409b6c711e3: Preparing
b76032430ac0: Preparing
4bad0a5bdce6: Preparing
b85415acf7ec: Waiting
d51dd5d7ea8e: Preparing
5c81f9330d99: Preparing
a52437ae58f3: Waiting
6409b6c711e3: Waiting
927f9fcef4cf: Preparing
d51dd5d7ea8e: Waiting
5c81f9330d99: Waiting
a81f1846a0d2: Preparing
927f9fcef4cf: Waiting
3b441d7cb46b: Preparing
d3710de04cb3: Preparing
91f7336bbfff: Preparing
e2e8c39e0f77: Preparing
d3710de04cb3: Waiting
91f7336bbfff: Waiting
e2e8c39e0f77: Waiting
d35cae3a437e: Pushed
7ed69227c2c7: Pushed
b145012a2908: Pushed
7187d543cc5b: Pushed
979821cbbfd3: Pushed
9000f5ee7237: Pushed
a52437ae58f3: Pushed
6409b6c711e3: Pushed
5c81f9330d99: Layer already exists
927f9fcef4cf: Layer already exists
4bad0a5bdce6: Pushed
a81f1846a0d2: Layer already exists
3b441d7cb46b: Layer already exists
b85415acf7ec: Pushed
d3710de04cb3: Layer already exists
91f7336bbfff: Layer already exists
e2e8c39e0f77: Layer already exists
d51dd5d7ea8e: Pushed
b76032430ac0: Pushed
20211216124331: digest: sha256:928bf6cb464d1d69a15b79707de65bca46c61955e9a231cc433259d92fd92918 size: 4311

> Task :sdks:java:testing:load-tests:run
Dec 16, 2021 12:45:29 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Dec 16, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 203 files. Enable logging at DEBUG level to see which files will be staged.
Dec 16, 2021 12:45:30 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Dec 16, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Dec 16, 2021 12:45:32 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 203 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Dec 16, 2021 12:45:33 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 203 files cached, 0 files newly uploaded in 0 seconds
Dec 16, 2021 12:45:33 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Dec 16, 2021 12:45:33 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <113691 bytes, hash 19cd5482a8334519433fcb84863911c94794d7405b09df933ec542a57a44bfc0> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-Gc1UgqgzRRlDP8uEhjkRyUeU10BbCd-TPsVCpXpEv8A.pb
Dec 16, 2021 12:45:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Dec 16, 2021 12:45:35 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@429f7919, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4a2929a4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@cda6019, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@797c3c3b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4012d5bc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4375b013, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1cf0cacc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4f5b08d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@529c2a9a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c87fdf2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26bbe604, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@fe34b86, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c98781a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3f736a16, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4601203a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@53abfc07, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2c8c16c0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@80bfa9d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@47c40b56, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4b039c6d]
Dec 16, 2021 12:45:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Dec 16, 2021 12:45:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Dec 16, 2021 12:45:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Dec 16, 2021 12:45:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Dec 16, 2021 12:45:35 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@67de7a99, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@795f5d51, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@34aeacd1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@54067fdc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4098dd77, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43aeb5e0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2274160, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@65383667, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@63cd2cd2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@557a84fe, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6deee370, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@49c17ba4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43e065f2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@423c5404, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5853ca50, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1a0d96a5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5a02bfe3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7a3e5cd3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c79088e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4a37191a]
Dec 16, 2021 12:45:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Dec 16, 2021 12:45:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Dec 16, 2021 12:45:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Dec 16, 2021 12:45:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Dec 16, 2021 12:45:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Dec 16, 2021 12:45:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Dec 16, 2021 12:45:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Dec 16, 2021 12:45:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Dec 16, 2021 12:45:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Dec 16, 2021 12:45:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Dec 16, 2021 12:45:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Dec 16, 2021 12:45:35 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.36.0-SNAPSHOT
Dec 16, 2021 12:45:36 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-12-16_04_45_35-10483991347121633974?project=apache-beam-testing
Dec 16, 2021 12:45:36 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-12-16_04_45_35-10483991347121633974
Dec 16, 2021 12:45:36 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-12-16_04_45_35-10483991347121633974
Dec 16, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-12-16T12:45:42.646Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-12-7zc. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Dec 16, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-16T12:45:47.213Z: Worker configuration: e2-standard-2 in us-central1-a.
Dec 16, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-16T12:45:47.946Z: Expanding SplittableParDo operations into optimizable parts.
Dec 16, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-16T12:45:47.979Z: Expanding CollectionToSingleton operations into optimizable parts.
Dec 16, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-16T12:45:48.035Z: Expanding CoGroupByKey operations into optimizable parts.
Dec 16, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-16T12:45:48.101Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Dec 16, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-16T12:45:48.133Z: Expanding GroupByKey operations into streaming Read/Write steps
Dec 16, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-16T12:45:48.201Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Dec 16, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-16T12:45:48.318Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Dec 16, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-16T12:45:48.371Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Dec 16, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-16T12:45:48.436Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Dec 16, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-16T12:45:48.469Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Dec 16, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-16T12:45:48.503Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Dec 16, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-16T12:45:48.538Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Dec 16, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-16T12:45:48.564Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Dec 16, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-16T12:45:48.587Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Dec 16, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-16T12:45:48.623Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Dec 16, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-16T12:45:48.647Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Dec 16, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-16T12:45:48.678Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Dec 16, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-16T12:45:48.711Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Dec 16, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-16T12:45:48.744Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Dec 16, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-16T12:45:48.769Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Dec 16, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-16T12:45:48.810Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Dec 16, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-16T12:45:48.835Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Dec 16, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-16T12:45:48.861Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Dec 16, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-16T12:45:48.888Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Dec 16, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-16T12:45:48.915Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Dec 16, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-16T12:45:48.948Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Dec 16, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-16T12:45:48.983Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Dec 16, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-16T12:45:49.016Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Dec 16, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-16T12:45:49.056Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Dec 16, 2021 12:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-16T12:45:49.517Z: Starting 5 ****s in us-central1-a...
Dec 16, 2021 12:46:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-16T12:46:02.319Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Dec 16, 2021 12:46:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-16T12:46:38.599Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Dec 16, 2021 12:47:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-16T12:47:35.246Z: Workers have started successfully.
Dec 16, 2021 12:47:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-16T12:47:35.284Z: Workers have started successfully.
Dec 16, 2021 4:00:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-16T16:00:24.576Z: Cancel request is committed for workflow job: 2021-12-16_04_45_35-10483991347121633974.
Dec 16, 2021 4:00:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-16T16:00:24.633Z: Cleaning up.
Dec 16, 2021 4:00:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-16T16:00:24.703Z: Stopping **** pool...
Dec 16, 2021 4:00:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-16T16:00:24.757Z: Stopping **** pool...
Dec 16, 2021 4:02:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-16T16:02:50.835Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Dec 16, 2021 4:02:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-16T16:02:50.869Z: Worker pool stopped.
Dec 16, 2021 4:02:57 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-12-16_04_45_35-10483991347121633974 finished with status CANCELLED.
Load test results for test (ID): ba554a53-b20c-47fc-8030-b29d681a884c and timestamp: 2021-12-16T12:45:29.794000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11544.556
dataflow_v2_java11_total_bytes_count             2.40614275E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211216124331
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:928bf6cb464d1d69a15b79707de65bca46c61955e9a231cc433259d92fd92918
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211216124331]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:928bf6cb464d1d69a15b79707de65bca46c61955e9a231cc433259d92fd92918]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211216124331] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:928bf6cb464d1d69a15b79707de65bca46c61955e9a231cc433259d92fd92918])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:928bf6cb464d1d69a15b79707de65bca46c61955e9a231cc433259d92fd92918
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:928bf6cb464d1d69a15b79707de65bca46c61955e9a231cc433259d92fd92918
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:928bf6cb464d1d69a15b79707de65bca46c61955e9a231cc433259d92fd92918].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 42s
101 actionable tasks: 72 executed, 27 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/bsx7pty42qhqo

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #181

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/181/display/redirect?page=changes>

Changes:

[aydar.zaynutdinov] [BEAM-13438][Playground]

[stranniknm] [BEAM-13446]: fix incorrect tab symbol

[noreply] Removes the comment that seems no longer relevant.

[Valentyn Tymofieiev] Update a few dependencies that may depend on log4j transitively.

[noreply] [BEAM-13159] Add Redis Stream (XADD) Write Support (#15858)

[noreply] Merge pull request #15994: [BEAM-13263] Support OnWindowExpiration in

[noreply] Bump dataflow container version to beam-master-20211213 (#16213)

[noreply] Merge pull request #16198 from [BEAM-13437][Playground] Add

[noreply] Merge pull request #16195 from a[BEAM-13436][Playground] Add

[noreply] [BEAM-13355] add Big Query parameter to enable users to specify load_…

[noreply] Updated the base images to use debian:bullseye (#16221)

[noreply] [BEAM-13434] Bump log4j to 2.16.0. (#16237)

[mmack] [BEAM-13209] Fix DynamoDBIO.write to properly handle partial success

[mmack] [BEAM-13209] Fix DynamoDBIO.write to properly handle partial success


------------------------------------------
[...truncated 48.51 KB...]
a6f21ca4c13a: Preparing
355bcc10b031: Preparing
6715dd53b547: Preparing
37871a55cfa4: Preparing
2c67ebf7a7fe: Preparing
cb734847c41f: Preparing
34516ab260ab: Preparing
240094277970: Preparing
066dc1a81373: Preparing
1aac0408ad5e: Preparing
1a4e4daffa72: Preparing
a10e6b6dd906: Preparing
5c81f9330d99: Preparing
927f9fcef4cf: Preparing
a81f1846a0d2: Preparing
3b441d7cb46b: Preparing
d3710de04cb3: Preparing
91f7336bbfff: Preparing
e2e8c39e0f77: Preparing
240094277970: Waiting
1a4e4daffa72: Waiting
066dc1a81373: Waiting
a10e6b6dd906: Waiting
1aac0408ad5e: Waiting
5c81f9330d99: Waiting
927f9fcef4cf: Waiting
cb734847c41f: Waiting
a81f1846a0d2: Waiting
3b441d7cb46b: Waiting
d3710de04cb3: Waiting
34516ab260ab: Waiting
91f7336bbfff: Waiting
e2e8c39e0f77: Waiting
2c67ebf7a7fe: Pushed
355bcc10b031: Pushed
6715dd53b547: Pushed
cb734847c41f: Pushed
a6f21ca4c13a: Pushed
37871a55cfa4: Pushed
240094277970: Pushed
066dc1a81373: Pushed
5c81f9330d99: Layer already exists
927f9fcef4cf: Layer already exists
1a4e4daffa72: Pushed
34516ab260ab: Pushed
a81f1846a0d2: Layer already exists
d3710de04cb3: Layer already exists
3b441d7cb46b: Layer already exists
91f7336bbfff: Layer already exists
e2e8c39e0f77: Layer already exists
a10e6b6dd906: Pushed
1aac0408ad5e: Pushed
20211215124334: digest: sha256:805d323837d7649ae3ceecb2a5c90d3f21fa2c7bd3ede8482f2fea45c32e032e size: 4311

> Task :sdks:java:testing:load-tests:run
Dec 15, 2021 12:45:38 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Dec 15, 2021 12:45:39 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 203 files. Enable logging at DEBUG level to see which files will be staged.
Dec 15, 2021 12:45:39 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Dec 15, 2021 12:45:39 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Dec 15, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 203 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Dec 15, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 203 files cached, 0 files newly uploaded in 0 seconds
Dec 15, 2021 12:45:42 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Dec 15, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <113691 bytes, hash d68f5f8d5e255a25c6e559ed626763190554af23f0e4c8a868c946a0d7e536a5> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-1o9fjV4lWiXG5VntYmdjGQVUryPw5MioaMlGoNflNqU.pb
Dec 15, 2021 12:45:44 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Dec 15, 2021 12:45:44 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@429f7919, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4a2929a4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@cda6019, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@797c3c3b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4012d5bc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4375b013, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1cf0cacc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4f5b08d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@529c2a9a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c87fdf2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26bbe604, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@fe34b86, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c98781a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3f736a16, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4601203a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@53abfc07, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2c8c16c0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@80bfa9d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@47c40b56, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4b039c6d]
Dec 15, 2021 12:45:44 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Dec 15, 2021 12:45:44 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Dec 15, 2021 12:45:44 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Dec 15, 2021 12:45:44 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Dec 15, 2021 12:45:44 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@67de7a99, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@795f5d51, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@34aeacd1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@54067fdc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4098dd77, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43aeb5e0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2274160, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@65383667, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@63cd2cd2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@557a84fe, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6deee370, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@49c17ba4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43e065f2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@423c5404, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5853ca50, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1a0d96a5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5a02bfe3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7a3e5cd3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c79088e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4a37191a]
Dec 15, 2021 12:45:44 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Dec 15, 2021 12:45:44 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Dec 15, 2021 12:45:44 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Dec 15, 2021 12:45:44 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Dec 15, 2021 12:45:44 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Dec 15, 2021 12:45:44 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Dec 15, 2021 12:45:44 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Dec 15, 2021 12:45:44 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Dec 15, 2021 12:45:44 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Dec 15, 2021 12:45:44 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Dec 15, 2021 12:45:44 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Dec 15, 2021 12:45:45 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.36.0-SNAPSHOT
Dec 15, 2021 12:45:45 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-12-15_04_45_45-17558208930821720948?project=apache-beam-testing
Dec 15, 2021 12:45:45 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-12-15_04_45_45-17558208930821720948
Dec 15, 2021 12:45:45 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-12-15_04_45_45-17558208930821720948
Dec 15, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-12-15T12:45:52.557Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-12-qwxw. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Dec 15, 2021 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-15T12:45:57.454Z: Worker configuration: e2-standard-2 in us-central1-a.
Dec 15, 2021 12:46:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-15T12:45:58.081Z: Expanding SplittableParDo operations into optimizable parts.
Dec 15, 2021 12:46:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-15T12:45:58.113Z: Expanding CollectionToSingleton operations into optimizable parts.
Dec 15, 2021 12:46:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-15T12:45:58.170Z: Expanding CoGroupByKey operations into optimizable parts.
Dec 15, 2021 12:46:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-15T12:45:58.241Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Dec 15, 2021 12:46:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-15T12:45:58.288Z: Expanding GroupByKey operations into streaming Read/Write steps
Dec 15, 2021 12:46:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-15T12:45:58.352Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Dec 15, 2021 12:46:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-15T12:45:58.485Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Dec 15, 2021 12:46:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-15T12:45:58.524Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Dec 15, 2021 12:46:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-15T12:45:58.557Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Dec 15, 2021 12:46:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-15T12:45:58.591Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Dec 15, 2021 12:46:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-15T12:45:58.624Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Dec 15, 2021 12:46:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-15T12:45:58.651Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Dec 15, 2021 12:46:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-15T12:45:58.675Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Dec 15, 2021 12:46:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-15T12:45:58.734Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Dec 15, 2021 12:46:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-15T12:45:58.769Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Dec 15, 2021 12:46:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-15T12:45:58.796Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Dec 15, 2021 12:46:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-15T12:45:58.827Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Dec 15, 2021 12:46:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-15T12:45:58.861Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Dec 15, 2021 12:46:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-15T12:45:58.892Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Dec 15, 2021 12:46:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-15T12:45:58.925Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Dec 15, 2021 12:46:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-15T12:45:58.959Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Dec 15, 2021 12:46:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-15T12:45:58.994Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Dec 15, 2021 12:46:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-15T12:45:59.021Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Dec 15, 2021 12:46:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-15T12:45:59.044Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Dec 15, 2021 12:46:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-15T12:45:59.077Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Dec 15, 2021 12:46:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-15T12:45:59.120Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Dec 15, 2021 12:46:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-15T12:45:59.140Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Dec 15, 2021 12:46:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-15T12:45:59.169Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Dec 15, 2021 12:46:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-15T12:45:59.197Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Dec 15, 2021 12:46:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-15T12:45:59.644Z: Starting 5 ****s in us-central1-a...
Dec 15, 2021 12:46:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-15T12:46:06.577Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Dec 15, 2021 12:46:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-15T12:46:48.870Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Dec 15, 2021 12:47:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-15T12:47:46.059Z: Workers have started successfully.
Dec 15, 2021 12:47:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-15T12:47:46.094Z: Workers have started successfully.
Dec 15, 2021 4:00:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-15T16:00:33.178Z: Cancel request is committed for workflow job: 2021-12-15_04_45_45-17558208930821720948.
Dec 15, 2021 4:00:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-15T16:00:33.290Z: Cleaning up.
Dec 15, 2021 4:00:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-15T16:00:33.398Z: Stopping **** pool...
Dec 15, 2021 4:00:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-15T16:00:33.472Z: Stopping **** pool...
Dec 15, 2021 4:03:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-15T16:02:59.817Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Dec 15, 2021 4:03:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-15T16:02:59.856Z: Worker pool stopped.
Dec 15, 2021 4:03:06 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-12-15_04_45_45-17558208930821720948 finished with status CANCELLED.
Load test results for test (ID): 3718cf26-2e64-4308-a399-4e20a0c9fa25 and timestamp: 2021-12-15T12:45:39.535000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11428.455
dataflow_v2_java11_total_bytes_count             1.90166544E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211215124334
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:805d323837d7649ae3ceecb2a5c90d3f21fa2c7bd3ede8482f2fea45c32e032e
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211215124334]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:805d323837d7649ae3ceecb2a5c90d3f21fa2c7bd3ede8482f2fea45c32e032e]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211215124334] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:805d323837d7649ae3ceecb2a5c90d3f21fa2c7bd3ede8482f2fea45c32e032e])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:805d323837d7649ae3ceecb2a5c90d3f21fa2c7bd3ede8482f2fea45c32e032e
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:805d323837d7649ae3ceecb2a5c90d3f21fa2c7bd3ede8482f2fea45c32e032e
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:805d323837d7649ae3ceecb2a5c90d3f21fa2c7bd3ede8482f2fea45c32e032e].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 51s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/cjb5s4ekmw7vs

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #180

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/180/display/redirect?page=changes>

Changes:

[Daniel Oliveira] [BEAM-13321] Pass TempLocation as pipeline option to Dataflow Go for

[Robert Bradshaw] Better type hints for Count combiners.

[Kyle Weaver] Include name of missing tag in error message.

[stranniknm] [BEAM-13423]: fix frontend failure if no examples

[daria.malkova] change return type of 2 methods

[mmack] [BEAM-13441] Use quiet delete for S3 batch deletes. In quiet mode only

[noreply] Updating Grafana from v8.1.2 to v8.1.6

[daria.malkova] Docs for validators tests

[daria.malkova] change context type

[noreply] Merge pull request #16140 from [BEAM-13377][Playground] Update CI/CD

[noreply] Merge pull request #16120 from [BEAM-13333][Playground] Save Python logs

[noreply] Merge pull request #16185 from [BEAM-13425][Playground][Bugfix] Support

[mmack] [BEAM-13445] Correctly set data limit when flushing S3 upload buffer and

[noreply] Merge pull request #16121 from [BEAM-13334][Playground] Save Go logs to

[noreply] Merge pull request #16179 from [BEAM-13344][Playground] support python

[noreply] Merge pull request #16208 from [BEAM-13442][Playground] Filepath to log

[noreply] [BEAM-13276] bump jackson-core to 2.13.0 for .test-infra (#16062)

[noreply] Change Pub/Sub Lite PollResult to set explicit watermark (#16216)

[noreply] [BEAM-13454] Fix and test dataframe read_fwf. (#16064)

[noreply] [BEAM-12976] Pipeline visitor to discover pushdown opportunities.

[noreply] [BEAM-13015] Allow decoding a set of elements until we hit the block


------------------------------------------
[...truncated 48.58 KB...]
91f7336bbfff: Waiting
78c3a7b74ad8: Waiting
1a7bf77856fc: Waiting
5626069a74e0: Waiting
e2e8c39e0f77: Waiting
db60aa4405a5: Pushed
3ac9c2fecc70: Pushed
cb89b8924d43: Pushed
752b3aca70b1: Pushed
afa057d8b3b8: Pushed
19bab224b506: Pushed
28c351f2219b: Pushed
78c3a7b74ad8: Pushed
5c81f9330d99: Layer already exists
927f9fcef4cf: Layer already exists
a81f1846a0d2: Layer already exists
3b441d7cb46b: Layer already exists
d3710de04cb3: Layer already exists
aeef33a16417: Pushed
0028bf7c6381: Pushed
91f7336bbfff: Layer already exists
e2e8c39e0f77: Layer already exists
1a7bf77856fc: Pushed
5626069a74e0: Pushed
20211214124339: digest: sha256:45bb2db85b92cc45e097bd29bc34b54a987268c83de0688d5487a1df2fb46615 size: 4311

> Task :sdks:java:testing:load-tests:run
Dec 14, 2021 12:45:58 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Dec 14, 2021 12:45:59 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 203 files. Enable logging at DEBUG level to see which files will be staged.
Dec 14, 2021 12:45:59 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Dec 14, 2021 12:45:59 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Dec 14, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 203 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Dec 14, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 203 files cached, 0 files newly uploaded in 0 seconds
Dec 14, 2021 12:46:02 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Dec 14, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <113691 bytes, hash cfa809eda01b4a61e8be5729d895f703dd9635230eb18f9a8af2c0a8d40a307b> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-z6gJ7aAbSmHovlcp2JX3A92WNSMOsY-aivLAqNQKMHs.pb
Dec 14, 2021 12:46:04 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Dec 14, 2021 12:46:04 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@429f7919, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4a2929a4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@cda6019, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@797c3c3b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4012d5bc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4375b013, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1cf0cacc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4f5b08d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@529c2a9a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c87fdf2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26bbe604, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@fe34b86, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c98781a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3f736a16, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4601203a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@53abfc07, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2c8c16c0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@80bfa9d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@47c40b56, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4b039c6d]
Dec 14, 2021 12:46:04 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Dec 14, 2021 12:46:04 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Dec 14, 2021 12:46:04 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Dec 14, 2021 12:46:04 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Dec 14, 2021 12:46:04 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@65e0b505, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@67de7a99, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@795f5d51, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@34aeacd1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@54067fdc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4098dd77, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43aeb5e0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2274160, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@65383667, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@63cd2cd2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@557a84fe, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6deee370, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@49c17ba4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43e065f2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@423c5404, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5853ca50, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1a0d96a5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5a02bfe3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7a3e5cd3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c79088e]
Dec 14, 2021 12:46:04 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Dec 14, 2021 12:46:04 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Dec 14, 2021 12:46:04 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Dec 14, 2021 12:46:04 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Dec 14, 2021 12:46:04 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Dec 14, 2021 12:46:04 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Dec 14, 2021 12:46:04 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Dec 14, 2021 12:46:04 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Dec 14, 2021 12:46:04 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Dec 14, 2021 12:46:04 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Dec 14, 2021 12:46:04 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Dec 14, 2021 12:46:04 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.36.0-SNAPSHOT
Dec 14, 2021 12:46:05 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-12-14_04_46_04-10847864043267183166?project=apache-beam-testing
Dec 14, 2021 12:46:05 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-12-14_04_46_04-10847864043267183166
Dec 14, 2021 12:46:05 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-12-14_04_46_04-10847864043267183166
Dec 14, 2021 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-12-14T12:46:13.628Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-12-rstx. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Dec 14, 2021 12:46:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-14T12:46:23.055Z: Worker configuration: e2-standard-2 in us-central1-a.
Dec 14, 2021 12:46:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-14T12:46:24.009Z: Expanding SplittableParDo operations into optimizable parts.
Dec 14, 2021 12:46:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-14T12:46:24.028Z: Expanding CollectionToSingleton operations into optimizable parts.
Dec 14, 2021 12:46:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-14T12:46:24.101Z: Expanding CoGroupByKey operations into optimizable parts.
Dec 14, 2021 12:46:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-14T12:46:24.178Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Dec 14, 2021 12:46:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-14T12:46:24.210Z: Expanding GroupByKey operations into streaming Read/Write steps
Dec 14, 2021 12:46:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-14T12:46:24.285Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Dec 14, 2021 12:46:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-14T12:46:24.396Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Dec 14, 2021 12:46:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-14T12:46:24.459Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Dec 14, 2021 12:46:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-14T12:46:24.498Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Dec 14, 2021 12:46:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-14T12:46:24.537Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Dec 14, 2021 12:46:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-14T12:46:24.598Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Dec 14, 2021 12:46:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-14T12:46:24.622Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Dec 14, 2021 12:46:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-14T12:46:24.670Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Dec 14, 2021 12:46:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-14T12:46:24.716Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Dec 14, 2021 12:46:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-14T12:46:24.737Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Dec 14, 2021 12:46:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-14T12:46:24.765Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Dec 14, 2021 12:46:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-14T12:46:24.815Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Dec 14, 2021 12:46:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-14T12:46:24.845Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Dec 14, 2021 12:46:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-14T12:46:24.886Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Dec 14, 2021 12:46:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-14T12:46:24.916Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Dec 14, 2021 12:46:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-14T12:46:24.976Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Dec 14, 2021 12:46:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-14T12:46:25.014Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Dec 14, 2021 12:46:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-14T12:46:25.063Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Dec 14, 2021 12:46:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-14T12:46:25.095Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Dec 14, 2021 12:46:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-14T12:46:25.127Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Dec 14, 2021 12:46:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-14T12:46:25.158Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Dec 14, 2021 12:46:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-14T12:46:25.190Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Dec 14, 2021 12:46:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-14T12:46:25.221Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Dec 14, 2021 12:46:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-14T12:46:25.266Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Dec 14, 2021 12:46:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-14T12:46:25.716Z: Starting 5 ****s in us-central1-a...
Dec 14, 2021 12:46:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-14T12:46:55.690Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Dec 14, 2021 12:47:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-14T12:47:17.195Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Dec 14, 2021 12:48:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-14T12:48:12.483Z: Workers have started successfully.
Dec 14, 2021 12:48:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-14T12:48:12.530Z: Workers have started successfully.
Dec 14, 2021 4:00:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-14T16:00:32.573Z: Cancel request is committed for workflow job: 2021-12-14_04_46_04-10847864043267183166.
Dec 14, 2021 4:00:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-14T16:00:32.675Z: Cleaning up.
Dec 14, 2021 4:00:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-14T16:00:32.805Z: Stopping **** pool...
Dec 14, 2021 4:00:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-14T16:00:32.878Z: Stopping **** pool...
Dec 14, 2021 4:02:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-14T16:02:58.021Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Dec 14, 2021 4:02:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-14T16:02:58.066Z: Worker pool stopped.
Dec 14, 2021 4:03:04 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-12-14_04_46_04-10847864043267183166 finished with status CANCELLED.
Load test results for test (ID): 13c2cfc1-d692-4f89-91f5-e3d08ce48dfa and timestamp: 2021-12-14T12:45:59.353000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11458.167
dataflow_v2_java11_total_bytes_count               9.8452935E9
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211214124339
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:45bb2db85b92cc45e097bd29bc34b54a987268c83de0688d5487a1df2fb46615
Deleted: sha256:31984e3cd5f1effd1bd83c233ba0c78e20abcf7cd4e861d837ab4d3d7a601cca
Deleted: sha256:7fd662f7010957ea9bf06bd7b335722f5425f5945f1b6cc6feab97ac52e8e7fd
Deleted: sha256:4d4d41f751686de528528116d227644035570c1cd97b37e2a8fc057f3255a238
Deleted: sha256:b1cd886d55c654df34127fe34baea43da56864e78bd8334e683ae77875d579ee
Deleted: sha256:6cac39d37c9cd6d832a471a67b21f82cea908b29ed529d18a28e98fb71187c38
Deleted: sha256:abc187c15a666a531d7b847e877975e16c07725ef8767bfe06da04752e67a56e
Deleted: sha256:32ff2e0b2822bc7a8a2071756d89f88ade4ce577aed6bf7a7ce31d90ba210748
Deleted: sha256:27bdec2438eaa53e37f2b4a553f9707b0d3bdecfae90f21a68cac6c523940144
Deleted: sha256:7e8e36f085f7cf3e8956b3bfc793a2b02a56c2ac02dc59f085176da8671725a3
Deleted: sha256:93a07ba63b9743f9a6cada9153e29b0e758df3bcff1fb2a90841c118da9edd44
Deleted: sha256:f50a1ee842578cc7b5f3d913bee9a4fbb4d86328d385d08089531af1d3d4f5b8
Deleted: sha256:e199d0e1d081f216364c30f580245c8cdb9db6e689151bc793cb2cda03e9f397
Deleted: sha256:42892fa12be941b704b86cb93cc9e2139322b7c45f4089b886ca89ea055ee6a6
Deleted: sha256:ff7bf5914e1b701f46657e3bcf9b6f6a6fdaff2c75b540bc12ff3f054321747a
Deleted: sha256:46d1b1c2bff7d6ade58020d09acacc3bc18d3b02622d0ea58722c8c996ecf650
Deleted: sha256:f814d152d91795c598d29fb102db03bbf5151b48a024179e2aae3f59650c44a8
Deleted: sha256:6661f2432f0a2d022d406c9a506f7cbc19fa30a84d0a692fffe816142af49cb8
Deleted: sha256:e261346411209ddcad0986f86b18732b00d752be3e07f8712c5cc9a83e9d68c0
Deleted: sha256:c3b179df6819bb1c89c64d2323066dda947d731827c0fc910b1771fd3babf718
Deleted: sha256:97bce8c173e989faa9c6b4a92db55d37e3c7300324a2c3787328833058e8310e
Deleted: sha256:50f45e4b8146d82054d62f41c00de62a55d34918ecbc435fe6771daacb327c8d
Deleted: sha256:b72af5263a1234b9da845a5ed2eec5385fb5af6bafda0ef97ed62716b6a1daf8
Deleted: sha256:6fff26c6d7afe5201f873571c0985c804e842f6d38c75668705dc18afba2060a
Deleted: sha256:30bfe25a08623fd67fa95e4486c93c14b2f2e42a6c52f2dfde8f711953017b17
Deleted: sha256:e8f9be52dfdcf23f1ad37027bf2ecbcbf928be8e2ac3a11b34e1f37e26911cdb
Deleted: sha256:8712d68b22b4e09124a8acea3fe418cd751713e1c17343ea7d26fa783c834a28
Deleted: sha256:5cb8d21e7d1bb69f836eed28a20d53e4aa42076bdd1484ee9fc65e4e1ff5cc22
Deleted: sha256:033cdcc5ad9788237d1333d911b1bbda210747136cbf0e29a4500410f955db70
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211214124339]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:45bb2db85b92cc45e097bd29bc34b54a987268c83de0688d5487a1df2fb46615]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211214124339] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:45bb2db85b92cc45e097bd29bc34b54a987268c83de0688d5487a1df2fb46615])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:45bb2db85b92cc45e097bd29bc34b54a987268c83de0688d5487a1df2fb46615
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:45bb2db85b92cc45e097bd29bc34b54a987268c83de0688d5487a1df2fb46615
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:45bb2db85b92cc45e097bd29bc34b54a987268c83de0688d5487a1df2fb46615].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 45s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/dj6jyepho2omm

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #179

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/179/display/redirect>

Changes:


------------------------------------------
[...truncated 47.91 KB...]
c3b63d9fc1de: Preparing
ae06d66027ce: Preparing
a18cd92eaadd: Preparing
b7e6ea413031: Preparing
12dc5cb5c701: Preparing
c5af9847714e: Preparing
11ddc425808c: Preparing
2dc9c8e01d86: Preparing
7c5469ff15b9: Preparing
cfea4461c462: Preparing
fc3cb2ae0622: Preparing
2c94c0d5ec2a: Preparing
5c81f9330d99: Preparing
927f9fcef4cf: Preparing
a81f1846a0d2: Preparing
3b441d7cb46b: Preparing
d3710de04cb3: Preparing
91f7336bbfff: Preparing
e2e8c39e0f77: Preparing
5c81f9330d99: Waiting
c5af9847714e: Waiting
11ddc425808c: Waiting
2dc9c8e01d86: Waiting
7c5469ff15b9: Waiting
927f9fcef4cf: Waiting
a81f1846a0d2: Waiting
3b441d7cb46b: Waiting
d3710de04cb3: Waiting
cfea4461c462: Waiting
91f7336bbfff: Waiting
e2e8c39e0f77: Waiting
fc3cb2ae0622: Waiting
2c94c0d5ec2a: Waiting
a18cd92eaadd: Pushed
ae06d66027ce: Pushed
12dc5cb5c701: Pushed
c3b63d9fc1de: Pushed
b7e6ea413031: Pushed
c5af9847714e: Pushed
2dc9c8e01d86: Pushed
7c5469ff15b9: Pushed
5c81f9330d99: Layer already exists
927f9fcef4cf: Layer already exists
a81f1846a0d2: Layer already exists
fc3cb2ae0622: Pushed
3b441d7cb46b: Layer already exists
d3710de04cb3: Layer already exists
91f7336bbfff: Layer already exists
e2e8c39e0f77: Layer already exists
11ddc425808c: Pushed
2c94c0d5ec2a: Pushed
cfea4461c462: Pushed
20211213124334: digest: sha256:a96d644e82f013762c58cbeed13ed6efaf9972ee34c6e1cd7455717bf792f725 size: 4311

> Task :sdks:java:testing:load-tests:run
Dec 13, 2021 12:45:31 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Dec 13, 2021 12:45:32 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 203 files. Enable logging at DEBUG level to see which files will be staged.
Dec 13, 2021 12:45:32 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Dec 13, 2021 12:45:32 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Dec 13, 2021 12:45:35 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 203 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Dec 13, 2021 12:45:36 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 203 files cached, 0 files newly uploaded in 0 seconds
Dec 13, 2021 12:45:36 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Dec 13, 2021 12:45:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <113691 bytes, hash 427c36924631917117938c224b9e85327c30559c04a58a5931bf5fcf6e0bfd8e> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-Qnw2kkYxkXEXk4wiS56FMnwwVZwEpYpZMb9fz24L_Y4.pb
Dec 13, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Dec 13, 2021 12:45:38 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@429f7919, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4a2929a4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@cda6019, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@797c3c3b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4012d5bc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4375b013, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1cf0cacc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4f5b08d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@529c2a9a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c87fdf2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26bbe604, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@fe34b86, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c98781a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3f736a16, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4601203a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@53abfc07, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2c8c16c0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@80bfa9d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@47c40b56, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4b039c6d]
Dec 13, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Dec 13, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Dec 13, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Dec 13, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Dec 13, 2021 12:45:38 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@67de7a99, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@795f5d51, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@34aeacd1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@54067fdc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4098dd77, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43aeb5e0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2274160, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@65383667, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@63cd2cd2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@557a84fe, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6deee370, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@49c17ba4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43e065f2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@423c5404, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5853ca50, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1a0d96a5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5a02bfe3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7a3e5cd3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c79088e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4a37191a]
Dec 13, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Dec 13, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Dec 13, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Dec 13, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Dec 13, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Dec 13, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Dec 13, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Dec 13, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Dec 13, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Dec 13, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Dec 13, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Dec 13, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.36.0-SNAPSHOT
Dec 13, 2021 12:45:39 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-12-13_04_45_38-8427878871215758655?project=apache-beam-testing
Dec 13, 2021 12:45:39 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-12-13_04_45_38-8427878871215758655
Dec 13, 2021 12:45:39 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-12-13_04_45_38-8427878871215758655
Dec 13, 2021 12:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-12-13T12:45:44.978Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-12-51om. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Dec 13, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-13T12:45:50.828Z: Worker configuration: e2-standard-2 in us-central1-a.
Dec 13, 2021 12:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-13T12:45:51.392Z: Expanding SplittableParDo operations into optimizable parts.
Dec 13, 2021 12:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-13T12:45:51.416Z: Expanding CollectionToSingleton operations into optimizable parts.
Dec 13, 2021 12:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-13T12:45:51.481Z: Expanding CoGroupByKey operations into optimizable parts.
Dec 13, 2021 12:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-13T12:45:51.561Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Dec 13, 2021 12:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-13T12:45:51.590Z: Expanding GroupByKey operations into streaming Read/Write steps
Dec 13, 2021 12:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-13T12:45:51.646Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Dec 13, 2021 12:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-13T12:45:51.726Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Dec 13, 2021 12:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-13T12:45:51.751Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Dec 13, 2021 12:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-13T12:45:51.774Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Dec 13, 2021 12:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-13T12:45:51.815Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Dec 13, 2021 12:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-13T12:45:51.874Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Dec 13, 2021 12:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-13T12:45:51.904Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Dec 13, 2021 12:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-13T12:45:51.926Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Dec 13, 2021 12:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-13T12:45:51.947Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Dec 13, 2021 12:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-13T12:45:51.979Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Dec 13, 2021 12:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-13T12:45:52.002Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Dec 13, 2021 12:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-13T12:45:52.025Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Dec 13, 2021 12:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-13T12:45:52.066Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Dec 13, 2021 12:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-13T12:45:52.097Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Dec 13, 2021 12:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-13T12:45:52.128Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Dec 13, 2021 12:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-13T12:45:52.159Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Dec 13, 2021 12:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-13T12:45:52.188Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Dec 13, 2021 12:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-13T12:45:52.210Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Dec 13, 2021 12:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-13T12:45:52.235Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Dec 13, 2021 12:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-13T12:45:52.258Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Dec 13, 2021 12:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-13T12:45:52.299Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Dec 13, 2021 12:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-13T12:45:52.333Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Dec 13, 2021 12:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-13T12:45:52.367Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Dec 13, 2021 12:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-13T12:45:52.390Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Dec 13, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-13T12:45:52.836Z: Starting 5 ****s in us-central1-a...
Dec 13, 2021 12:46:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-13T12:46:23.770Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Dec 13, 2021 12:46:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-13T12:46:45.413Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Dec 13, 2021 12:47:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-13T12:47:42.506Z: Workers have started successfully.
Dec 13, 2021 12:47:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-13T12:47:42.544Z: Workers have started successfully.
Dec 13, 2021 4:00:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-13T16:00:41.242Z: Cancel request is committed for workflow job: 2021-12-13_04_45_38-8427878871215758655.
Dec 13, 2021 4:00:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-13T16:00:41.343Z: Cleaning up.
Dec 13, 2021 4:00:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-13T16:00:41.422Z: Stopping **** pool...
Dec 13, 2021 4:00:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-13T16:00:41.477Z: Stopping **** pool...
Dec 13, 2021 4:03:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-13T16:03:02.467Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Dec 13, 2021 4:03:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-13T16:03:02.512Z: Worker pool stopped.
Dec 13, 2021 4:03:09 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-12-13_04_45_38-8427878871215758655 finished with status CANCELLED.
Load test results for test (ID): e91ae5fc-48e2-4750-8649-239f3cf28650 and timestamp: 2021-12-13T12:45:32.427000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11553.599
dataflow_v2_java11_total_bytes_count             1.37818616E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211213124334
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:a96d644e82f013762c58cbeed13ed6efaf9972ee34c6e1cd7455717bf792f725
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211213124334]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:a96d644e82f013762c58cbeed13ed6efaf9972ee34c6e1cd7455717bf792f725]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211213124334] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:a96d644e82f013762c58cbeed13ed6efaf9972ee34c6e1cd7455717bf792f725])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:a96d644e82f013762c58cbeed13ed6efaf9972ee34c6e1cd7455717bf792f725
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:a96d644e82f013762c58cbeed13ed6efaf9972ee34c6e1cd7455717bf792f725
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:a96d644e82f013762c58cbeed13ed6efaf9972ee34c6e1cd7455717bf792f725].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 53s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/wny2bv5m3y2js

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #178

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/178/display/redirect?page=changes>

Changes:

[noreply] [BEAM-13015] Plumb the cache through contexts and transform executors.


------------------------------------------
[...truncated 48.47 KB...]
The push refers to repository [us.gcr.io/apache-beam-testing/java-postcommit-it/java]
bdb7d24d918e: Preparing
11b84a6aa952: Preparing
d4c1a9bb5a7d: Preparing
a655e3d39ede: Preparing
b2e27d664be5: Preparing
facb02d1b60d: Preparing
78adbc62f53b: Preparing
7ed4397a3936: Preparing
bb796e33bed4: Preparing
704cd2e7c191: Preparing
1585f76ccba2: Preparing
c6b20a252f15: Preparing
5c81f9330d99: Preparing
927f9fcef4cf: Preparing
a81f1846a0d2: Preparing
3b441d7cb46b: Preparing
d3710de04cb3: Preparing
91f7336bbfff: Preparing
e2e8c39e0f77: Preparing
bb796e33bed4: Waiting
704cd2e7c191: Waiting
5c81f9330d99: Waiting
d3710de04cb3: Waiting
1585f76ccba2: Waiting
c6b20a252f15: Waiting
91f7336bbfff: Waiting
e2e8c39e0f77: Waiting
927f9fcef4cf: Waiting
a81f1846a0d2: Waiting
3b441d7cb46b: Waiting
7ed4397a3936: Waiting
78adbc62f53b: Waiting
11b84a6aa952: Pushed
d4c1a9bb5a7d: Pushed
b2e27d664be5: Pushed
bdb7d24d918e: Pushed
facb02d1b60d: Pushed
a655e3d39ede: Pushed
7ed4397a3936: Pushed
bb796e33bed4: Pushed
1585f76ccba2: Pushed
5c81f9330d99: Layer already exists
927f9fcef4cf: Layer already exists
a81f1846a0d2: Layer already exists
3b441d7cb46b: Layer already exists
d3710de04cb3: Layer already exists
e2e8c39e0f77: Layer already exists
91f7336bbfff: Layer already exists
78adbc62f53b: Pushed
c6b20a252f15: Pushed
704cd2e7c191: Pushed
20211212124331: digest: sha256:998f5f145ac83bda9ffc267aaf52ecee8dcf982df2e5f1512f38911771983144 size: 4311

> Task :sdks:java:testing:load-tests:run
Dec 12, 2021 12:45:24 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Dec 12, 2021 12:45:25 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 203 files. Enable logging at DEBUG level to see which files will be staged.
Dec 12, 2021 12:45:25 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Dec 12, 2021 12:45:25 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Dec 12, 2021 12:45:28 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 203 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Dec 12, 2021 12:45:28 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 203 files cached, 0 files newly uploaded in 0 seconds
Dec 12, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Dec 12, 2021 12:45:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <113691 bytes, hash 5d9254e89b25ac2c37390ef094ab6858354b518890d68901dc0ac6d816123d44> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-XZJU6JslrCw3OQ7wlKtoWDVLUYiQ1okB3ArG2BYSPUQ.pb
Dec 12, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Dec 12, 2021 12:45:30 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@429f7919, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4a2929a4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@cda6019, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@797c3c3b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4012d5bc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4375b013, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1cf0cacc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4f5b08d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@529c2a9a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c87fdf2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26bbe604, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@fe34b86, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c98781a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3f736a16, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4601203a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@53abfc07, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2c8c16c0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@80bfa9d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@47c40b56, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4b039c6d]
Dec 12, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Dec 12, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Dec 12, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Dec 12, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Dec 12, 2021 12:45:30 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@67de7a99, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@795f5d51, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@34aeacd1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@54067fdc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4098dd77, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43aeb5e0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2274160, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@65383667, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@63cd2cd2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@557a84fe, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6deee370, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@49c17ba4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43e065f2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@423c5404, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5853ca50, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1a0d96a5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5a02bfe3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7a3e5cd3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c79088e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4a37191a]
Dec 12, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Dec 12, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Dec 12, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Dec 12, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Dec 12, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Dec 12, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Dec 12, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Dec 12, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Dec 12, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Dec 12, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Dec 12, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Dec 12, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.36.0-SNAPSHOT
Dec 12, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-12-12_04_45_30-13716156934319324213?project=apache-beam-testing
Dec 12, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-12-12_04_45_30-13716156934319324213
Dec 12, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-12-12_04_45_30-13716156934319324213
Dec 12, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-12-12T12:45:39.271Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-12-maqb. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Dec 12, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-12T12:45:42.606Z: Worker configuration: e2-standard-2 in us-central1-a.
Dec 12, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-12T12:45:43.302Z: Expanding SplittableParDo operations into optimizable parts.
Dec 12, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-12T12:45:43.335Z: Expanding CollectionToSingleton operations into optimizable parts.
Dec 12, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-12T12:45:43.406Z: Expanding CoGroupByKey operations into optimizable parts.
Dec 12, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-12T12:45:43.518Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Dec 12, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-12T12:45:43.561Z: Expanding GroupByKey operations into streaming Read/Write steps
Dec 12, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-12T12:45:43.622Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Dec 12, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-12T12:45:43.749Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Dec 12, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-12T12:45:43.778Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Dec 12, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-12T12:45:43.808Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Dec 12, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-12T12:45:43.834Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Dec 12, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-12T12:45:43.870Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Dec 12, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-12T12:45:43.911Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Dec 12, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-12T12:45:43.957Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Dec 12, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-12T12:45:43.994Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Dec 12, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-12T12:45:44.024Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Dec 12, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-12T12:45:44.092Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Dec 12, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-12T12:45:44.118Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Dec 12, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-12T12:45:44.151Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Dec 12, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-12T12:45:44.183Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Dec 12, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-12T12:45:44.203Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Dec 12, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-12T12:45:44.234Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Dec 12, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-12T12:45:44.265Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Dec 12, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-12T12:45:44.288Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Dec 12, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-12T12:45:44.309Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Dec 12, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-12T12:45:44.336Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Dec 12, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-12T12:45:44.395Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Dec 12, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-12T12:45:44.446Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Dec 12, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-12T12:45:44.470Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Dec 12, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-12T12:45:44.522Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Dec 12, 2021 12:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-12T12:45:44.945Z: Starting 5 ****s in us-central1-a...
Dec 12, 2021 12:46:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-12T12:46:01.111Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Dec 12, 2021 12:46:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-12T12:46:32.025Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Dec 12, 2021 12:47:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-12T12:47:33.770Z: Workers have started successfully.
Dec 12, 2021 12:47:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-12T12:47:33.807Z: Workers have started successfully.
Dec 12, 2021 4:00:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-12T16:00:33.061Z: Cancel request is committed for workflow job: 2021-12-12_04_45_30-13716156934319324213.
Dec 12, 2021 4:00:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-12T16:00:33.145Z: Cleaning up.
Dec 12, 2021 4:00:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-12T16:00:33.231Z: Stopping **** pool...
Dec 12, 2021 4:00:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-12T16:00:33.293Z: Stopping **** pool...
Dec 12, 2021 4:03:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-12T16:03:00.132Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Dec 12, 2021 4:03:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-12T16:03:00.188Z: Worker pool stopped.
Dec 12, 2021 4:03:06 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-12-12_04_45_30-13716156934319324213 finished with status CANCELLED.
Load test results for test (ID): c46ba322-b8ed-40fd-8c55-b02c55d5d85e and timestamp: 2021-12-12T12:45:25.604000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11552.879
dataflow_v2_java11_total_bytes_count             1.24155733E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211212124331
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:998f5f145ac83bda9ffc267aaf52ecee8dcf982df2e5f1512f38911771983144
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211212124331]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:998f5f145ac83bda9ffc267aaf52ecee8dcf982df2e5f1512f38911771983144]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211212124331] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:998f5f145ac83bda9ffc267aaf52ecee8dcf982df2e5f1512f38911771983144])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:998f5f145ac83bda9ffc267aaf52ecee8dcf982df2e5f1512f38911771983144
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:998f5f145ac83bda9ffc267aaf52ecee8dcf982df2e5f1512f38911771983144
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:998f5f145ac83bda9ffc267aaf52ecee8dcf982df2e5f1512f38911771983144].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 51s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/qu2jacavsjiu2

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #177

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/177/display/redirect?page=changes>

Changes:

[aydar.zaynutdinov] [BEAM-13424][Playground]

[aydar.zaynutdinov] [BEAM-13424][Playground]

[stranniknm] [BEAM-13426] clear log and output panels on example change

[aydar.zaynutdinov] [BEAM-13428][Playground]

[aydar.zaynutdinov] [BEAM-13424][Playground]

[daria.malkova] add type of precompiled object

[Alexey Romanenko] [BEAM-13171] Fix Duration ambiguity

[mmack] [BEAM-13410][BEAM-11788] Integration test for S3Filesystem using

[Alexey Romanenko] [BEAM-13434] Bump up Apache log4j2 vulnerability to 2.15.0

[noreply] Merge pull request #16138 from [BEAM-13343][Playground] Support go unit

[noreply] [BEAM-12560] Dataframe idxmin and idxmax implementation (#15827)

[noreply] [BEAM-12565] Dataframe compare implementation (#16027)

[noreply] [BEAM-12976] Use a map to pass all pushdown requests at once. (#16189)

[noreply] [BEAM-12683]  Fix failing integration tests for Python Recommendation AI


------------------------------------------
[...truncated 49.27 KB...]
153afbc36442: Waiting
b5b597fd8241: Waiting
46804051de77: Waiting
0f0e7b544a14: Waiting
5c81f9330d99: Waiting
a81f1846a0d2: Waiting
d3710de04cb3: Waiting
5eb36e995c47: Waiting
3b441d7cb46b: Waiting
91f7336bbfff: Waiting
927f9fcef4cf: Waiting
e9a1ac956616: Pushed
189ba2214737: Pushed
dcb926f09b43: Pushed
4df5ad5e1754: Pushed
0f0e7b544a14: Pushed
6f85da26d475: Pushed
db1e8994175d: Pushed
79642cf2a3d7: Pushed
5c81f9330d99: Layer already exists
927f9fcef4cf: Layer already exists
a81f1846a0d2: Layer already exists
153afbc36442: Pushed
3b441d7cb46b: Layer already exists
5eb36e995c47: Pushed
46804051de77: Pushed
d3710de04cb3: Layer already exists
91f7336bbfff: Layer already exists
e2e8c39e0f77: Layer already exists
b5b597fd8241: Pushed
20211211124333: digest: sha256:9ac788139e4d04ce1bc9fb2b5f402a69fd9f83c427e8008bc77be5e9c5d5cfba size: 4311

> Task :sdks:java:testing:load-tests:run
Dec 11, 2021 12:45:27 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Dec 11, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 203 files. Enable logging at DEBUG level to see which files will be staged.
Dec 11, 2021 12:45:28 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Dec 11, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Dec 11, 2021 12:45:31 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 203 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Dec 11, 2021 12:45:31 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 203 files cached, 0 files newly uploaded in 0 seconds
Dec 11, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Dec 11, 2021 12:45:31 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <113691 bytes, hash a24d499f43aa5afb43675ad7cfbe5e8c97a1d32636640a9e741dc88edd8e4ebc> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-ok1Jn0OqWvtDZ1rXz75ejJeh0yY2ZAqedB3Ijt2OTrw.pb
Dec 11, 2021 12:45:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Dec 11, 2021 12:45:33 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@429f7919, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4a2929a4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@cda6019, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@797c3c3b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4012d5bc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4375b013, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1cf0cacc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4f5b08d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@529c2a9a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c87fdf2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26bbe604, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@fe34b86, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c98781a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3f736a16, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4601203a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@53abfc07, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2c8c16c0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@80bfa9d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@47c40b56, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4b039c6d]
Dec 11, 2021 12:45:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Dec 11, 2021 12:45:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Dec 11, 2021 12:45:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Dec 11, 2021 12:45:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Dec 11, 2021 12:45:33 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@67de7a99, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@795f5d51, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@34aeacd1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@54067fdc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4098dd77, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43aeb5e0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2274160, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@65383667, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@63cd2cd2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@557a84fe, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6deee370, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@49c17ba4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43e065f2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@423c5404, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5853ca50, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1a0d96a5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5a02bfe3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7a3e5cd3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c79088e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4a37191a]
Dec 11, 2021 12:45:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Dec 11, 2021 12:45:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Dec 11, 2021 12:45:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Dec 11, 2021 12:45:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Dec 11, 2021 12:45:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Dec 11, 2021 12:45:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Dec 11, 2021 12:45:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Dec 11, 2021 12:45:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Dec 11, 2021 12:45:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Dec 11, 2021 12:45:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Dec 11, 2021 12:45:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Dec 11, 2021 12:45:33 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.36.0-SNAPSHOT
Dec 11, 2021 12:45:34 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-12-11_04_45_33-8265912439203147639?project=apache-beam-testing
Dec 11, 2021 12:45:34 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-12-11_04_45_33-8265912439203147639
Dec 11, 2021 12:45:34 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-12-11_04_45_33-8265912439203147639
Dec 11, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-12-11T12:45:40.772Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-12-nrkf. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Dec 11, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-11T12:45:45.560Z: Worker configuration: e2-standard-2 in us-central1-a.
Dec 11, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-11T12:45:46.279Z: Expanding SplittableParDo operations into optimizable parts.
Dec 11, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-11T12:45:46.321Z: Expanding CollectionToSingleton operations into optimizable parts.
Dec 11, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-11T12:45:46.414Z: Expanding CoGroupByKey operations into optimizable parts.
Dec 11, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-11T12:45:46.619Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Dec 11, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-11T12:45:46.712Z: Expanding GroupByKey operations into streaming Read/Write steps
Dec 11, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-11T12:45:46.794Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Dec 11, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-11T12:45:46.938Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Dec 11, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-11T12:45:46.991Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Dec 11, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-11T12:45:47.054Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Dec 11, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-11T12:45:47.104Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Dec 11, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-11T12:45:47.144Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Dec 11, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-11T12:45:47.198Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Dec 11, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-11T12:45:47.241Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Dec 11, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-11T12:45:47.282Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Dec 11, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-11T12:45:47.327Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Dec 11, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-11T12:45:47.368Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Dec 11, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-11T12:45:47.401Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Dec 11, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-11T12:45:47.437Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Dec 11, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-11T12:45:47.471Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Dec 11, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-11T12:45:47.504Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Dec 11, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-11T12:45:47.536Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Dec 11, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-11T12:45:47.570Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Dec 11, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-11T12:45:47.611Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Dec 11, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-11T12:45:47.646Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Dec 11, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-11T12:45:47.677Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Dec 11, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-11T12:45:47.714Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Dec 11, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-11T12:45:47.737Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Dec 11, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-11T12:45:47.780Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Dec 11, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-11T12:45:47.831Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Dec 11, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-11T12:45:48.285Z: Starting 5 ****s in us-central1-a...
Dec 11, 2021 12:45:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-11T12:45:59.401Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Dec 11, 2021 12:46:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-11T12:46:35.464Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Dec 11, 2021 12:47:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-11T12:47:32.918Z: Workers have started successfully.
Dec 11, 2021 12:47:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-11T12:47:32.952Z: Workers have started successfully.
Dec 11, 2021 2:24:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-11T14:24:48.458Z: Staged package amazon-kinesis-client-1.14.2-qtBh3OuKuLE7S26ITTJzYsoYfG-mWjqbGmv3VJPhec8.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/amazon-kinesis-client-1.14.2-qtBh3OuKuLE7S26ITTJzYsoYfG-mWjqbGmv3VJPhec8.jar' is inaccessible.
Dec 11, 2021 2:24:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-11T14:24:48.823Z: Staged package aws-java-sdk-cloudwatch-1.12.106-kUP1xZSNspLnoAnC0aAzibk9w0IXNbBNeMBESBYiZtg.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/aws-java-sdk-cloudwatch-1.12.106-kUP1xZSNspLnoAnC0aAzibk9w0IXNbBNeMBESBYiZtg.jar' is inaccessible.
Dec 11, 2021 2:24:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-11T14:24:48.894Z: Staged package aws-java-sdk-core-1.12.106-2a5g2j72YwhD9uNhLJWqRljsgds1wEVsdH_XLgZZOlQ.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/aws-java-sdk-core-1.12.106-2a5g2j72YwhD9uNhLJWqRljsgds1wEVsdH_XLgZZOlQ.jar' is inaccessible.
Dec 11, 2021 2:24:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-11T14:24:48.998Z: Staged package aws-java-sdk-dynamodb-1.12.106-s8z13aUxEWhNzx7J8mY6rVcDrK2iLAcBqT3Pb9sNgmM.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/aws-java-sdk-dynamodb-1.12.106-s8z13aUxEWhNzx7J8mY6rVcDrK2iLAcBqT3Pb9sNgmM.jar' is inaccessible.
Dec 11, 2021 2:24:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-11T14:24:49.096Z: Staged package aws-java-sdk-kinesis-1.12.106-wMPghgT8YvcVokv233DC2r8Q0eeaqh7phw3Anmz6S4o.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/aws-java-sdk-kinesis-1.12.106-wMPghgT8YvcVokv233DC2r8Q0eeaqh7phw3Anmz6S4o.jar' is inaccessible.
Dec 11, 2021 2:24:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-11T14:24:49.186Z: Staged package aws-java-sdk-kms-1.12.106-JkHOiVwKCYdQ3d-wR7YYej-O0UmkZOioL923IYI9PwM.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/aws-java-sdk-kms-1.12.106-JkHOiVwKCYdQ3d-wR7YYej-O0UmkZOioL923IYI9PwM.jar' is inaccessible.
Dec 11, 2021 2:24:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-11T14:24:49.274Z: Staged package aws-java-sdk-s3-1.12.106-5YV0cnyV551lh5cwCZGFhypcyG7NsqTcEP0IBjaAnjE.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/aws-java-sdk-s3-1.12.106-5YV0cnyV551lh5cwCZGFhypcyG7NsqTcEP0IBjaAnjE.jar' is inaccessible.
Dec 11, 2021 2:24:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-11T14:24:52.095Z: Staged package jakarta.activation-api-1.2.2-oYepORA671hJp6-EvX4nvi0SDEEK8pFDc3X_4GH08J0.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/jakarta.activation-api-1.2.2-oYepORA671hJp6-EvX4nvi0SDEEK8pFDc3X_4GH08J0.jar' is inaccessible.
Dec 11, 2021 2:24:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-11T14:24:52.184Z: Staged package jakarta.xml.bind-api-2.3.3-wEU59HLppt0MdoXqgtZ3KCJpq457rKLhRQDjgeDGzsU.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/jakarta.xml.bind-api-2.3.3-wEU59HLppt0MdoXqgtZ3KCJpq457rKLhRQDjgeDGzsU.jar' is inaccessible.
Dec 11, 2021 2:24:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-11T14:24:52.291Z: Staged package jmespath-java-1.12.106-_mWkV0KPXB56ULuR7j6dxTg6hrDhcycHHtCXYBv9V7s.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/jmespath-java-1.12.106-_mWkV0KPXB56ULuR7j6dxTg6hrDhcycHHtCXYBv9V7s.jar' is inaccessible.
Dec 11, 2021 2:24:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-12-11T14:24:53.513Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Dec 11, 2021 4:00:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-11T16:00:36.512Z: Cancel request is committed for workflow job: 2021-12-11_04_45_33-8265912439203147639.
Dec 11, 2021 4:00:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-11T16:00:36.560Z: Cleaning up.
Dec 11, 2021 4:00:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-11T16:00:36.692Z: Stopping **** pool...
Dec 11, 2021 4:00:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-11T16:00:36.737Z: Stopping **** pool...
Dec 11, 2021 4:03:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-11T16:03:08.331Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Dec 11, 2021 4:03:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-11T16:03:08.390Z: Worker pool stopped.
Dec 11, 2021 4:03:13 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-12-11_04_45_33-8265912439203147639 finished with status CANCELLED.
Load test results for test (ID): a7ec4f31-1219-4c92-83ee-701fcf7fb5ca and timestamp: 2021-12-11T12:45:28.549000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11539.658
dataflow_v2_java11_total_bytes_count             2.05228429E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211211124333
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:9ac788139e4d04ce1bc9fb2b5f402a69fd9f83c427e8008bc77be5e9c5d5cfba
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211211124333]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:9ac788139e4d04ce1bc9fb2b5f402a69fd9f83c427e8008bc77be5e9c5d5cfba]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211211124333] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:9ac788139e4d04ce1bc9fb2b5f402a69fd9f83c427e8008bc77be5e9c5d5cfba])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:9ac788139e4d04ce1bc9fb2b5f402a69fd9f83c427e8008bc77be5e9c5d5cfba
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:9ac788139e4d04ce1bc9fb2b5f402a69fd9f83c427e8008bc77be5e9c5d5cfba
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:9ac788139e4d04ce1bc9fb2b5f402a69fd9f83c427e8008bc77be5e9c5d5cfba].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 59s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/kbe3wuhsiqndw

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #176

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/176/display/redirect?page=changes>

Changes:

[slebedev] [BEAM-13384] Re-exported the metrics subpackage in

[zyichi] [BEAM-13401] Wait for worker start to reduce flakiness in

[zyichi] Fix pytest unknown markers warning.

[noreply] [BEAM-11936] Remove suppression in ReadFromKafkaDoFn (#16174)

[noreply] [BEAM-12561] method truncate on series and dataframe (#15833)

[noreply] [BEAM-13294] Widen key schema for all keys before use (#16158)

[noreply] [BEAM-13171] Support for stopReadTime on KafkaIO SDF (#15951)

[noreply] [BEAM-13399] Add functionality to download Beam JARs from Maven 

[noreply] [BEAM-13402] Add another workaround for


------------------------------------------
[...truncated 49.96 KB...]
bb4cbb83d9b0: Pushed
bf3006ee3329: Pushed
a610e68468f8: Pushed
5c81f9330d99: Layer already exists
927f9fcef4cf: Layer already exists
a81f1846a0d2: Layer already exists
a5e4846680ab: Pushed
3b441d7cb46b: Layer already exists
d3710de04cb3: Layer already exists
91f7336bbfff: Layer already exists
e2e8c39e0f77: Layer already exists
b110851a7e00: Pushed
0a3213d92595: Pushed
9ba9cab944bf: Pushed
20211210124332: digest: sha256:ab60f3f67a394688a065bc59fab36c1a9e596035691253dfe305d091bb97fcc9 size: 4311

> Task :sdks:java:testing:load-tests:run
Dec 10, 2021 12:45:27 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Dec 10, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 203 files. Enable logging at DEBUG level to see which files will be staged.
Dec 10, 2021 12:45:28 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Dec 10, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Dec 10, 2021 12:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 203 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Dec 10, 2021 12:45:31 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 203 files cached, 0 files newly uploaded in 0 seconds
Dec 10, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Dec 10, 2021 12:45:31 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <113691 bytes, hash 9833b2477d8cb7464bc8248aa2e368e67d7aabdd2aa80e1ca78491e3babb1760> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-mDOyR32Mt0ZLyCSKouNo5n16q90qqA4cp4SR47q7F2A.pb
Dec 10, 2021 12:45:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Dec 10, 2021 12:45:33 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@429f7919, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4a2929a4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@cda6019, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@797c3c3b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4012d5bc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4375b013, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1cf0cacc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4f5b08d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@529c2a9a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c87fdf2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26bbe604, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@fe34b86, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c98781a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3f736a16, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4601203a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@53abfc07, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2c8c16c0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@80bfa9d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@47c40b56, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4b039c6d]
Dec 10, 2021 12:45:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Dec 10, 2021 12:45:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Dec 10, 2021 12:45:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Dec 10, 2021 12:45:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Dec 10, 2021 12:45:33 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@67de7a99, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@795f5d51, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@34aeacd1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@54067fdc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4098dd77, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43aeb5e0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2274160, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@65383667, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@63cd2cd2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@557a84fe, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6deee370, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@49c17ba4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43e065f2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@423c5404, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5853ca50, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1a0d96a5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5a02bfe3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7a3e5cd3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c79088e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4a37191a]
Dec 10, 2021 12:45:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Dec 10, 2021 12:45:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Dec 10, 2021 12:45:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Dec 10, 2021 12:45:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Dec 10, 2021 12:45:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Dec 10, 2021 12:45:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Dec 10, 2021 12:45:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Dec 10, 2021 12:45:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Dec 10, 2021 12:45:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Dec 10, 2021 12:45:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Dec 10, 2021 12:45:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Dec 10, 2021 12:45:33 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.36.0-SNAPSHOT
Dec 10, 2021 12:45:34 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-12-10_04_45_33-8039644437749157402?project=apache-beam-testing
Dec 10, 2021 12:45:34 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-12-10_04_45_33-8039644437749157402
Dec 10, 2021 12:45:34 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-12-10_04_45_33-8039644437749157402
Dec 10, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-12-10T12:45:44.540Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-12-dp3d. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Dec 10, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-10T12:45:50.739Z: Worker configuration: e2-standard-2 in us-central1-a.
Dec 10, 2021 12:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-10T12:45:51.747Z: Expanding SplittableParDo operations into optimizable parts.
Dec 10, 2021 12:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-10T12:45:51.873Z: Expanding CollectionToSingleton operations into optimizable parts.
Dec 10, 2021 12:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-10T12:45:52.358Z: Expanding CoGroupByKey operations into optimizable parts.
Dec 10, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-10T12:45:52.757Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Dec 10, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-10T12:45:52.869Z: Expanding GroupByKey operations into streaming Read/Write steps
Dec 10, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-10T12:45:53.246Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Dec 10, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-10T12:45:53.879Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Dec 10, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-10T12:45:54.076Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Dec 10, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-10T12:45:54.243Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Dec 10, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-10T12:45:54.341Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Dec 10, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-10T12:45:54.476Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Dec 10, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-10T12:45:54.667Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Dec 10, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-10T12:45:54.881Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Dec 10, 2021 12:45:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-10T12:45:55.217Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Dec 10, 2021 12:45:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-10T12:45:55.362Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Dec 10, 2021 12:45:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-10T12:45:55.951Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Dec 10, 2021 12:45:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-10T12:45:56.160Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Dec 10, 2021 12:45:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-10T12:45:56.267Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Dec 10, 2021 12:45:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-10T12:45:56.553Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Dec 10, 2021 12:45:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-10T12:45:56.654Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Dec 10, 2021 12:45:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-10T12:45:56.777Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Dec 10, 2021 12:45:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-10T12:45:57.006Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Dec 10, 2021 12:45:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-10T12:45:57.239Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Dec 10, 2021 12:45:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-10T12:45:57.344Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Dec 10, 2021 12:45:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-10T12:45:57.449Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Dec 10, 2021 12:45:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-10T12:45:57.558Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Dec 10, 2021 12:45:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-10T12:45:57.674Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Dec 10, 2021 12:45:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-10T12:45:57.863Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Dec 10, 2021 12:45:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-10T12:45:57.965Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Dec 10, 2021 12:45:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-10T12:45:59.103Z: Starting 5 ****s in us-central1-a...
Dec 10, 2021 12:46:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-10T12:46:08.924Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Dec 10, 2021 12:46:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-10T12:46:39.650Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Dec 10, 2021 12:47:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-10T12:47:40.140Z: Workers have started successfully.
Dec 10, 2021 12:47:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-10T12:47:40.187Z: Workers have started successfully.
Dec 10, 2021 1:16:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-10T13:16:00.107Z: Staged package annotations-4.1.1.4-unNOHoTAnWFa9qCdMwNLTwRC-Hct7BIO-zdthqVlrhU.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/annotations-4.1.1.4-unNOHoTAnWFa9qCdMwNLTwRC-Hct7BIO-zdthqVlrhU.jar' is inaccessible.
Dec 10, 2021 1:16:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-10T13:16:00.437Z: Staged package avro-1.8.2-91SggwzmelqfpnpU7BXRA-8V4chQ17Jvr3tkfu3cgtM.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/avro-1.8.2-91SggwzmelqfpnpU7BXRA-8V4chQ17Jvr3tkfu3cgtM.jar' is inaccessible.
Dec 10, 2021 1:16:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-10T13:16:01.090Z: Staged package beam-vendor-bytebuddy-1_11_0-0.1-SneHjzQgCpxxLkvyN8dugqO_sTzwXJ2995kqFu4l2i4.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-bytebuddy-1_11_0-0.1-SneHjzQgCpxxLkvyN8dugqO_sTzwXJ2995kqFu4l2i4.jar' is inaccessible.
Dec 10, 2021 1:16:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-10T13:16:01.239Z: Staged package beam-vendor-guava-26_0-jre-0.1-8quizfXTrmcvJyOXi3zWDFtP5EofgqjksY5sIbsBqws.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-guava-26_0-jre-0.1-8quizfXTrmcvJyOXi3zWDFtP5EofgqjksY5sIbsBqws.jar' is inaccessible.
Dec 10, 2021 1:16:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-10T13:16:01.386Z: Staged package checker-compat-qual-2.5.5-EdE0skXpysxHRRTS1mtbhhj4A5oUZc3FW7wLNOAAi3o.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/checker-compat-qual-2.5.5-EdE0skXpysxHRRTS1mtbhhj4A5oUZc3FW7wLNOAAi3o.jar' is inaccessible.
Dec 10, 2021 1:16:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-10T13:16:01.702Z: Staged package commons-lang3-3.9-3i4dzc8--ReozoWGYaBnJqmpRPKOM61_ngi-pE3DwjA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/commons-lang3-3.9-3i4dzc8--ReozoWGYaBnJqmpRPKOM61_ngi-pE3DwjA.jar' is inaccessible.
Dec 10, 2021 1:16:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-10T13:16:01.846Z: Staged package commons-logging-1.2-2t3qHqC-D1aXirMAa4rJKDSv7vvZt-TmMW_KV98PpjY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/commons-logging-1.2-2t3qHqC-D1aXirMAa4rJKDSv7vvZt-TmMW_KV98PpjY.jar' is inaccessible.
Dec 10, 2021 1:16:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-10T13:16:01.936Z: Staged package commons-math3-3.6.1-HlbXsFjSi2Wr0la4RY44hbZ0wdWI-kPNfRy7nH7yswg.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/commons-math3-3.6.1-HlbXsFjSi2Wr0la4RY44hbZ0wdWI-kPNfRy7nH7yswg.jar' is inaccessible.
Dec 10, 2021 1:16:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-10T13:16:03.464Z: Staged package hamcrest-2.1-upOy46ViMiukMvChtTrdzFXLGIJTMZoCDtd_gk5pIFA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/hamcrest-2.1-upOy46ViMiukMvChtTrdzFXLGIJTMZoCDtd_gk5pIFA.jar' is inaccessible.
Dec 10, 2021 1:16:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-10T13:16:03.583Z: Staged package hamcrest-core-2.1-4JEJ5UoonYhQa5v-yYfd0Zn0IXyUZBMmaDUbmk8Avuk.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/hamcrest-core-2.1-4JEJ5UoonYhQa5v-yYfd0Zn0IXyUZBMmaDUbmk8Avuk.jar' is inaccessible.
Dec 10, 2021 1:16:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-10T13:16:03.746Z: Staged package j2objc-annotations-1.3-Ia8wySJnvWEiwOC00gzMtmQaN-r5VsZUDsRx1YTmSns.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/j2objc-annotations-1.3-Ia8wySJnvWEiwOC00gzMtmQaN-r5VsZUDsRx1YTmSns.jar' is inaccessible.
Dec 10, 2021 1:16:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-10T13:16:03.886Z: Staged package jackson-core-asl-1.9.13-RAqctcqVshX5U9OiCmsaENofCbUpqd3qX4pJBd2rT1o.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/jackson-core-asl-1.9.13-RAqctcqVshX5U9OiCmsaENofCbUpqd3qX4pJBd2rT1o.jar' is inaccessible.
Dec 10, 2021 1:16:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-10T13:16:03.995Z: Staged package jackson-mapper-asl-1.9.13-dOegenby7breKTEqWi68z6AZEovAIezjhW12GX6b4MI.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/jackson-mapper-asl-1.9.13-dOegenby7breKTEqWi68z6AZEovAIezjhW12GX6b4MI.jar' is inaccessible.
Dec 10, 2021 1:16:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-10T13:16:04.087Z: Staged package javax.annotation-api-1.3.2-4EulGVvNVV3JVlD3zGFNFR5LzVLSmhC4qiGX86uJq5s.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/javax.annotation-api-1.3.2-4EulGVvNVV3JVlD3zGFNFR5LzVLSmhC4qiGX86uJq5s.jar' is inaccessible.
Dec 10, 2021 1:16:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-10T13:16:04.187Z: Staged package json-20200518-D_MObagvFa-vDna9bplqerVWaFEWaoP5c8ZKmg-UbS0.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/json-20200518-D_MObagvFa-vDna9bplqerVWaFEWaoP5c8ZKmg-UbS0.jar' is inaccessible.
Dec 10, 2021 1:16:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-10T13:16:04.260Z: Staged package jsr305-3.0.2-dmrSoHg_JoeWLIrXTO7MOKKLn3Ki0IXuQ4t4E-ko0Mc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/jsr305-3.0.2-dmrSoHg_JoeWLIrXTO7MOKKLn3Ki0IXuQ4t4E-ko0Mc.jar' is inaccessible.
Dec 10, 2021 1:16:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-12-10T13:16:05.967Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Dec 10, 2021 4:00:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-10T16:00:32.457Z: Cancel request is committed for workflow job: 2021-12-10_04_45_33-8039644437749157402.
Dec 10, 2021 4:00:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-10T16:00:32.533Z: Cleaning up.
Dec 10, 2021 4:00:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-10T16:00:32.709Z: Stopping **** pool...
Dec 10, 2021 4:00:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-10T16:00:32.781Z: Stopping **** pool...
Dec 10, 2021 4:03:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-10T16:03:01.029Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Dec 10, 2021 4:03:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-10T16:03:01.216Z: Worker pool stopped.
Dec 10, 2021 4:03:09 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-12-10_04_45_33-8039644437749157402 finished with status CANCELLED.
Load test results for test (ID): ce69be53-432f-4e46-a534-1b78b7868de6 and timestamp: 2021-12-10T12:45:28.320000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11534.626
dataflow_v2_java11_total_bytes_count             2.12050114E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211210124332
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ab60f3f67a394688a065bc59fab36c1a9e596035691253dfe305d091bb97fcc9
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211210124332]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ab60f3f67a394688a065bc59fab36c1a9e596035691253dfe305d091bb97fcc9]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211210124332] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ab60f3f67a394688a065bc59fab36c1a9e596035691253dfe305d091bb97fcc9])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ab60f3f67a394688a065bc59fab36c1a9e596035691253dfe305d091bb97fcc9
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ab60f3f67a394688a065bc59fab36c1a9e596035691253dfe305d091bb97fcc9
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ab60f3f67a394688a065bc59fab36c1a9e596035691253dfe305d091bb97fcc9].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:c1d0672425677151b308da8514055cc06bccff2f62e23ecec8908f3ef953eac4
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:c1d0672425677151b308da8514055cc06bccff2f62e23ecec8908f3ef953eac4
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:c1d0672425677151b308da8514055cc06bccff2f62e23ecec8908f3ef953eac4].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 54s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/52do64cfepmyo

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #175

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/175/display/redirect?page=changes>

Changes:

[stranniknm] fix playground frontend licences

[noreply] Merge pull request #16167 from [BEAM-13409][Playground] [Bugfix] Change

[noreply] Merge pull request #16136 from [BEAM-13365] [Playground] Add Pipelines

[noreply] [BEAM-13244] Support STS Assume role credentials provider for AWS SDK v2

[noreply] [BEAM-11936] Fix errorprone UnusedVariable in core,examples,harness..

[noreply] [BEAM-11936] Remove suppressUnusedVariable flag (#16171)

[noreply] [BEAM-13090] Adding SDK harness container overrides option to Java SDK

[noreply] [BEAM-11936] Fix errorprone warnings (#15890)

[noreply] [BEAM-13015] Start integrating a process wide cache. (#16130)


------------------------------------------
[...truncated 61.38 KB...]
Dec 09, 2021 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-09T12:45:52.921Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Dec 09, 2021 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-09T12:45:52.946Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Dec 09, 2021 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-09T12:45:52.977Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Dec 09, 2021 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-09T12:45:53.001Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Dec 09, 2021 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-09T12:45:53.047Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Dec 09, 2021 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-09T12:45:53.076Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Dec 09, 2021 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-09T12:45:53.102Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Dec 09, 2021 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-09T12:45:53.121Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Dec 09, 2021 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-09T12:45:53.150Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Dec 09, 2021 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-09T12:45:53.178Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Dec 09, 2021 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-09T12:45:53.199Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Dec 09, 2021 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-09T12:45:53.225Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Dec 09, 2021 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-09T12:45:53.247Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Dec 09, 2021 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-09T12:45:53.273Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Dec 09, 2021 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-09T12:45:53.301Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Dec 09, 2021 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-09T12:45:53.325Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Dec 09, 2021 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-09T12:45:53.348Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Dec 09, 2021 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-09T12:45:53.375Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Dec 09, 2021 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-09T12:45:53.815Z: Starting 5 ****s in us-central1-a...
Dec 09, 2021 12:46:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-09T12:46:11.514Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Dec 09, 2021 12:46:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-09T12:46:45.472Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Dec 09, 2021 12:47:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-09T12:47:41.493Z: Workers have started successfully.
Dec 09, 2021 12:47:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-09T12:47:41.522Z: Workers have started successfully.
Dec 09, 2021 1:45:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-09T13:45:55.069Z: Staged package commons-compress-1.21-auz9VFlyillWAc-gcljRMZcv_Dm0kutIvdWWV3ovJEo.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/commons-compress-1.21-auz9VFlyillWAc-gcljRMZcv_Dm0kutIvdWWV3ovJEo.jar' is inaccessible.
Dec 09, 2021 1:45:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-09T13:45:55.277Z: Staged package conscrypt-openjdk-uber-2.5.1-AfnHQstZKhUeLmK9U5eomAYoqWcAH82s1KpHRGeGhfM.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/conscrypt-openjdk-uber-2.5.1-AfnHQstZKhUeLmK9U5eomAYoqWcAH82s1KpHRGeGhfM.jar' is inaccessible.
Dec 09, 2021 1:45:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-09T13:45:57.602Z: Staged package netty-buffer-4.1.52.Final-QcoQNa7m4PgXdZfQp2EKWqLG5vrHRKyB_eTl5Y9jlnU.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-buffer-4.1.52.Final-QcoQNa7m4PgXdZfQp2EKWqLG5vrHRKyB_eTl5Y9jlnU.jar' is inaccessible.
Dec 09, 2021 1:45:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-09T13:45:57.658Z: Staged package netty-codec-4.1.52.Final-PH543M5-NT2GrH26LCUatc3dTrhuqoic_kIpeCuCjlI.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-codec-4.1.52.Final-PH543M5-NT2GrH26LCUatc3dTrhuqoic_kIpeCuCjlI.jar' is inaccessible.
Dec 09, 2021 1:45:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-09T13:45:57.715Z: Staged package netty-codec-http-4.1.52.Final-tdoo0OqHiwfTERrPXk-vQSiO8jArJkFBkP3Rp9lzyAw.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-codec-http-4.1.52.Final-tdoo0OqHiwfTERrPXk-vQSiO8jArJkFBkP3Rp9lzyAw.jar' is inaccessible.
Dec 09, 2021 1:45:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-09T13:45:57.767Z: Staged package netty-codec-http2-4.1.52.Final-61J6QtIonRLe5tR1PKJxWXije7tBxxkVzk6VOcSfk6E.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-codec-http2-4.1.52.Final-61J6QtIonRLe5tR1PKJxWXije7tBxxkVzk6VOcSfk6E.jar' is inaccessible.
Dec 09, 2021 1:45:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-09T13:45:57.811Z: Staged package netty-codec-socks-4.1.52.Final-_sxXfgu3ch1naM_BhG7hcpEdFquA372ZldllelUdTqc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-codec-socks-4.1.52.Final-_sxXfgu3ch1naM_BhG7hcpEdFquA372ZldllelUdTqc.jar' is inaccessible.
Dec 09, 2021 1:45:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-09T13:45:57.856Z: Staged package netty-common-4.1.52.Final-DR8QHk231TDtBNfKvVfZXAdQ8Fm7az-t_tDIDUc0xxc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-common-4.1.52.Final-DR8QHk231TDtBNfKvVfZXAdQ8Fm7az-t_tDIDUc0xxc.jar' is inaccessible.
Dec 09, 2021 1:45:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-09T13:45:57.897Z: Staged package netty-handler-4.1.52.Final-LNwh-_rgSUDIWSkDl3nJP1H2GPXvshYHkxnc_PMqVcE.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-handler-4.1.52.Final-LNwh-_rgSUDIWSkDl3nJP1H2GPXvshYHkxnc_PMqVcE.jar' is inaccessible.
Dec 09, 2021 1:45:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-09T13:45:57.947Z: Staged package netty-handler-proxy-4.1.52.Final-6NB4Y860vUgG4lWh1TLIZQ8POy1NAGrW0hgS6UG0L9w.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-handler-proxy-4.1.52.Final-6NB4Y860vUgG4lWh1TLIZQ8POy1NAGrW0hgS6UG0L9w.jar' is inaccessible.
Dec 09, 2021 1:45:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-09T13:45:57.998Z: Staged package netty-resolver-4.1.52.Final-butyojOXnLRbC01tRsWkEzLFfAz4qAOx78YNu-y6CaY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-resolver-4.1.52.Final-butyojOXnLRbC01tRsWkEzLFfAz4qAOx78YNu-y6CaY.jar' is inaccessible.
Dec 09, 2021 1:45:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-09T13:45:58.063Z: Staged package netty-transport-4.1.52.Final-mj5vjA5V3jY-seoQ_ngXl-yjlOYhht8q4LTrK84LRUE.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-transport-4.1.52.Final-mj5vjA5V3jY-seoQ_ngXl-yjlOYhht8q4LTrK84LRUE.jar' is inaccessible.
Dec 09, 2021 1:45:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-09T13:45:58.110Z: Staged package opencensus-api-0.28.0-DBcj8_bTBhMjhFzouIs1_dpQCBLgp1uOtfzErYyHGpU.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/opencensus-api-0.28.0-DBcj8_bTBhMjhFzouIs1_dpQCBLgp1uOtfzErYyHGpU.jar' is inaccessible.
Dec 09, 2021 1:45:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-09T13:45:58.233Z: Staged package paranamer-2.8-aIyxGKYCHYGROOhVIIyVYDFoi-S0eiS7YVvsxjrO3wc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/paranamer-2.8-aIyxGKYCHYGROOhVIIyVYDFoi-S0eiS7YVvsxjrO3wc.jar' is inaccessible.
Dec 09, 2021 1:45:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-09T13:45:58.365Z: Staged package perfmark-api-0.23.0-xwW1wQwY_zAyuegXQrwvaw5WB_am38DIrQz_ddSRMEI.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/perfmark-api-0.23.0-xwW1wQwY_zAyuegXQrwvaw5WB_am38DIrQz_ddSRMEI.jar' is inaccessible.
Dec 09, 2021 1:45:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-09T13:45:58.832Z: Staged package snappy-java-1.1.8.4-JMTR_B6J4HgzGrj0AamcrWhZm95KLkUWBCy1SMUbHD4.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/snappy-java-1.1.8.4-JMTR_B6J4HgzGrj0AamcrWhZm95KLkUWBCy1SMUbHD4.jar' is inaccessible.
Dec 09, 2021 1:45:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-12-09T13:45:58.957Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Dec 09, 2021 1:48:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-12-09T13:48:58.087Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Dec 09, 2021 1:51:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-09T13:51:55.096Z: Staged package commons-compress-1.21-auz9VFlyillWAc-gcljRMZcv_Dm0kutIvdWWV3ovJEo.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/commons-compress-1.21-auz9VFlyillWAc-gcljRMZcv_Dm0kutIvdWWV3ovJEo.jar' is inaccessible.
Dec 09, 2021 1:51:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-09T13:51:55.238Z: Staged package conscrypt-openjdk-uber-2.5.1-AfnHQstZKhUeLmK9U5eomAYoqWcAH82s1KpHRGeGhfM.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/conscrypt-openjdk-uber-2.5.1-AfnHQstZKhUeLmK9U5eomAYoqWcAH82s1KpHRGeGhfM.jar' is inaccessible.
Dec 09, 2021 1:51:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-09T13:51:57.242Z: Staged package netty-buffer-4.1.52.Final-QcoQNa7m4PgXdZfQp2EKWqLG5vrHRKyB_eTl5Y9jlnU.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-buffer-4.1.52.Final-QcoQNa7m4PgXdZfQp2EKWqLG5vrHRKyB_eTl5Y9jlnU.jar' is inaccessible.
Dec 09, 2021 1:51:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-09T13:51:57.304Z: Staged package netty-codec-4.1.52.Final-PH543M5-NT2GrH26LCUatc3dTrhuqoic_kIpeCuCjlI.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-codec-4.1.52.Final-PH543M5-NT2GrH26LCUatc3dTrhuqoic_kIpeCuCjlI.jar' is inaccessible.
Dec 09, 2021 1:51:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-09T13:51:57.347Z: Staged package netty-codec-http-4.1.52.Final-tdoo0OqHiwfTERrPXk-vQSiO8jArJkFBkP3Rp9lzyAw.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-codec-http-4.1.52.Final-tdoo0OqHiwfTERrPXk-vQSiO8jArJkFBkP3Rp9lzyAw.jar' is inaccessible.
Dec 09, 2021 1:51:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-09T13:51:57.386Z: Staged package netty-codec-http2-4.1.52.Final-61J6QtIonRLe5tR1PKJxWXije7tBxxkVzk6VOcSfk6E.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-codec-http2-4.1.52.Final-61J6QtIonRLe5tR1PKJxWXije7tBxxkVzk6VOcSfk6E.jar' is inaccessible.
Dec 09, 2021 1:51:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-09T13:51:57.426Z: Staged package netty-codec-socks-4.1.52.Final-_sxXfgu3ch1naM_BhG7hcpEdFquA372ZldllelUdTqc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-codec-socks-4.1.52.Final-_sxXfgu3ch1naM_BhG7hcpEdFquA372ZldllelUdTqc.jar' is inaccessible.
Dec 09, 2021 1:51:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-09T13:51:57.466Z: Staged package netty-common-4.1.52.Final-DR8QHk231TDtBNfKvVfZXAdQ8Fm7az-t_tDIDUc0xxc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-common-4.1.52.Final-DR8QHk231TDtBNfKvVfZXAdQ8Fm7az-t_tDIDUc0xxc.jar' is inaccessible.
Dec 09, 2021 1:51:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-09T13:51:57.501Z: Staged package netty-handler-4.1.52.Final-LNwh-_rgSUDIWSkDl3nJP1H2GPXvshYHkxnc_PMqVcE.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-handler-4.1.52.Final-LNwh-_rgSUDIWSkDl3nJP1H2GPXvshYHkxnc_PMqVcE.jar' is inaccessible.
Dec 09, 2021 1:51:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-09T13:51:57.535Z: Staged package netty-handler-proxy-4.1.52.Final-6NB4Y860vUgG4lWh1TLIZQ8POy1NAGrW0hgS6UG0L9w.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-handler-proxy-4.1.52.Final-6NB4Y860vUgG4lWh1TLIZQ8POy1NAGrW0hgS6UG0L9w.jar' is inaccessible.
Dec 09, 2021 1:51:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-09T13:51:57.572Z: Staged package netty-resolver-4.1.52.Final-butyojOXnLRbC01tRsWkEzLFfAz4qAOx78YNu-y6CaY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-resolver-4.1.52.Final-butyojOXnLRbC01tRsWkEzLFfAz4qAOx78YNu-y6CaY.jar' is inaccessible.
Dec 09, 2021 1:51:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-09T13:51:57.639Z: Staged package netty-transport-4.1.52.Final-mj5vjA5V3jY-seoQ_ngXl-yjlOYhht8q4LTrK84LRUE.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-transport-4.1.52.Final-mj5vjA5V3jY-seoQ_ngXl-yjlOYhht8q4LTrK84LRUE.jar' is inaccessible.
Dec 09, 2021 1:51:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-09T13:51:57.675Z: Staged package opencensus-api-0.28.0-DBcj8_bTBhMjhFzouIs1_dpQCBLgp1uOtfzErYyHGpU.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/opencensus-api-0.28.0-DBcj8_bTBhMjhFzouIs1_dpQCBLgp1uOtfzErYyHGpU.jar' is inaccessible.
Dec 09, 2021 1:51:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-09T13:51:57.777Z: Staged package paranamer-2.8-aIyxGKYCHYGROOhVIIyVYDFoi-S0eiS7YVvsxjrO3wc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/paranamer-2.8-aIyxGKYCHYGROOhVIIyVYDFoi-S0eiS7YVvsxjrO3wc.jar' is inaccessible.
Dec 09, 2021 1:51:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-09T13:51:57.814Z: Staged package perfmark-api-0.23.0-xwW1wQwY_zAyuegXQrwvaw5WB_am38DIrQz_ddSRMEI.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/perfmark-api-0.23.0-xwW1wQwY_zAyuegXQrwvaw5WB_am38DIrQz_ddSRMEI.jar' is inaccessible.
Dec 09, 2021 1:51:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-09T13:51:58.290Z: Staged package snappy-java-1.1.8.4-JMTR_B6J4HgzGrj0AamcrWhZm95KLkUWBCy1SMUbHD4.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/snappy-java-1.1.8.4-JMTR_B6J4HgzGrj0AamcrWhZm95KLkUWBCy1SMUbHD4.jar' is inaccessible.
Dec 09, 2021 1:51:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-12-09T13:51:58.494Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Dec 09, 2021 1:54:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-12-09T13:54:58.170Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Dec 09, 2021 1:57:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-09T13:57:55.270Z: Staged package commons-compress-1.21-auz9VFlyillWAc-gcljRMZcv_Dm0kutIvdWWV3ovJEo.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/commons-compress-1.21-auz9VFlyillWAc-gcljRMZcv_Dm0kutIvdWWV3ovJEo.jar' is inaccessible.
Dec 09, 2021 1:57:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-09T13:57:55.431Z: Staged package conscrypt-openjdk-uber-2.5.1-AfnHQstZKhUeLmK9U5eomAYoqWcAH82s1KpHRGeGhfM.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/conscrypt-openjdk-uber-2.5.1-AfnHQstZKhUeLmK9U5eomAYoqWcAH82s1KpHRGeGhfM.jar' is inaccessible.
Dec 09, 2021 1:57:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-09T13:57:57.432Z: Staged package netty-buffer-4.1.52.Final-QcoQNa7m4PgXdZfQp2EKWqLG5vrHRKyB_eTl5Y9jlnU.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-buffer-4.1.52.Final-QcoQNa7m4PgXdZfQp2EKWqLG5vrHRKyB_eTl5Y9jlnU.jar' is inaccessible.
Dec 09, 2021 1:57:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-09T13:57:57.481Z: Staged package netty-codec-4.1.52.Final-PH543M5-NT2GrH26LCUatc3dTrhuqoic_kIpeCuCjlI.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-codec-4.1.52.Final-PH543M5-NT2GrH26LCUatc3dTrhuqoic_kIpeCuCjlI.jar' is inaccessible.
Dec 09, 2021 1:57:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-09T13:57:57.519Z: Staged package netty-codec-http-4.1.52.Final-tdoo0OqHiwfTERrPXk-vQSiO8jArJkFBkP3Rp9lzyAw.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-codec-http-4.1.52.Final-tdoo0OqHiwfTERrPXk-vQSiO8jArJkFBkP3Rp9lzyAw.jar' is inaccessible.
Dec 09, 2021 1:57:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-09T13:57:57.560Z: Staged package netty-codec-http2-4.1.52.Final-61J6QtIonRLe5tR1PKJxWXije7tBxxkVzk6VOcSfk6E.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-codec-http2-4.1.52.Final-61J6QtIonRLe5tR1PKJxWXije7tBxxkVzk6VOcSfk6E.jar' is inaccessible.
Dec 09, 2021 1:57:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-09T13:57:57.608Z: Staged package netty-codec-socks-4.1.52.Final-_sxXfgu3ch1naM_BhG7hcpEdFquA372ZldllelUdTqc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-codec-socks-4.1.52.Final-_sxXfgu3ch1naM_BhG7hcpEdFquA372ZldllelUdTqc.jar' is inaccessible.
Dec 09, 2021 1:57:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-09T13:57:57.656Z: Staged package netty-common-4.1.52.Final-DR8QHk231TDtBNfKvVfZXAdQ8Fm7az-t_tDIDUc0xxc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-common-4.1.52.Final-DR8QHk231TDtBNfKvVfZXAdQ8Fm7az-t_tDIDUc0xxc.jar' is inaccessible.
Dec 09, 2021 1:57:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-09T13:57:57.703Z: Staged package netty-handler-4.1.52.Final-LNwh-_rgSUDIWSkDl3nJP1H2GPXvshYHkxnc_PMqVcE.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-handler-4.1.52.Final-LNwh-_rgSUDIWSkDl3nJP1H2GPXvshYHkxnc_PMqVcE.jar' is inaccessible.
Dec 09, 2021 1:57:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-09T13:57:57.754Z: Staged package netty-handler-proxy-4.1.52.Final-6NB4Y860vUgG4lWh1TLIZQ8POy1NAGrW0hgS6UG0L9w.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-handler-proxy-4.1.52.Final-6NB4Y860vUgG4lWh1TLIZQ8POy1NAGrW0hgS6UG0L9w.jar' is inaccessible.
Dec 09, 2021 1:57:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-09T13:57:57.796Z: Staged package netty-resolver-4.1.52.Final-butyojOXnLRbC01tRsWkEzLFfAz4qAOx78YNu-y6CaY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-resolver-4.1.52.Final-butyojOXnLRbC01tRsWkEzLFfAz4qAOx78YNu-y6CaY.jar' is inaccessible.
Dec 09, 2021 1:58:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-09T13:57:57.862Z: Staged package netty-transport-4.1.52.Final-mj5vjA5V3jY-seoQ_ngXl-yjlOYhht8q4LTrK84LRUE.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-transport-4.1.52.Final-mj5vjA5V3jY-seoQ_ngXl-yjlOYhht8q4LTrK84LRUE.jar' is inaccessible.
Dec 09, 2021 1:58:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-09T13:57:57.929Z: Staged package opencensus-api-0.28.0-DBcj8_bTBhMjhFzouIs1_dpQCBLgp1uOtfzErYyHGpU.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/opencensus-api-0.28.0-DBcj8_bTBhMjhFzouIs1_dpQCBLgp1uOtfzErYyHGpU.jar' is inaccessible.
Dec 09, 2021 1:58:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-09T13:57:58.020Z: Staged package paranamer-2.8-aIyxGKYCHYGROOhVIIyVYDFoi-S0eiS7YVvsxjrO3wc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/paranamer-2.8-aIyxGKYCHYGROOhVIIyVYDFoi-S0eiS7YVvsxjrO3wc.jar' is inaccessible.
Dec 09, 2021 1:58:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-09T13:57:58.072Z: Staged package perfmark-api-0.23.0-xwW1wQwY_zAyuegXQrwvaw5WB_am38DIrQz_ddSRMEI.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/perfmark-api-0.23.0-xwW1wQwY_zAyuegXQrwvaw5WB_am38DIrQz_ddSRMEI.jar' is inaccessible.
Dec 09, 2021 1:58:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-09T13:57:58.620Z: Staged package snappy-java-1.1.8.4-JMTR_B6J4HgzGrj0AamcrWhZm95KLkUWBCy1SMUbHD4.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/snappy-java-1.1.8.4-JMTR_B6J4HgzGrj0AamcrWhZm95KLkUWBCy1SMUbHD4.jar' is inaccessible.
Dec 09, 2021 1:58:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-12-09T13:57:58.777Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Dec 09, 2021 2:00:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-12-09T14:00:58.659Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Dec 09, 2021 2:03:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-09T14:03:55.100Z: Staged package commons-compress-1.21-auz9VFlyillWAc-gcljRMZcv_Dm0kutIvdWWV3ovJEo.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/commons-compress-1.21-auz9VFlyillWAc-gcljRMZcv_Dm0kutIvdWWV3ovJEo.jar' is inaccessible.
Dec 09, 2021 2:03:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-09T14:03:55.235Z: Staged package conscrypt-openjdk-uber-2.5.1-AfnHQstZKhUeLmK9U5eomAYoqWcAH82s1KpHRGeGhfM.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/conscrypt-openjdk-uber-2.5.1-AfnHQstZKhUeLmK9U5eomAYoqWcAH82s1KpHRGeGhfM.jar' is inaccessible.
Dec 09, 2021 2:03:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-09T14:03:57.402Z: Staged package netty-buffer-4.1.52.Final-QcoQNa7m4PgXdZfQp2EKWqLG5vrHRKyB_eTl5Y9jlnU.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-buffer-4.1.52.Final-QcoQNa7m4PgXdZfQp2EKWqLG5vrHRKyB_eTl5Y9jlnU.jar' is inaccessible.
Dec 09, 2021 2:03:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-09T14:03:57.458Z: Staged package netty-codec-4.1.52.Final-PH543M5-NT2GrH26LCUatc3dTrhuqoic_kIpeCuCjlI.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-codec-4.1.52.Final-PH543M5-NT2GrH26LCUatc3dTrhuqoic_kIpeCuCjlI.jar' is inaccessible.
Dec 09, 2021 2:03:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-09T14:03:57.506Z: Staged package netty-codec-http-4.1.52.Final-tdoo0OqHiwfTERrPXk-vQSiO8jArJkFBkP3Rp9lzyAw.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-codec-http-4.1.52.Final-tdoo0OqHiwfTERrPXk-vQSiO8jArJkFBkP3Rp9lzyAw.jar' is inaccessible.
Dec 09, 2021 2:03:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-09T14:03:57.551Z: Staged package netty-codec-http2-4.1.52.Final-61J6QtIonRLe5tR1PKJxWXije7tBxxkVzk6VOcSfk6E.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-codec-http2-4.1.52.Final-61J6QtIonRLe5tR1PKJxWXije7tBxxkVzk6VOcSfk6E.jar' is inaccessible.
Dec 09, 2021 2:03:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-09T14:03:57.605Z: Staged package netty-codec-socks-4.1.52.Final-_sxXfgu3ch1naM_BhG7hcpEdFquA372ZldllelUdTqc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-codec-socks-4.1.52.Final-_sxXfgu3ch1naM_BhG7hcpEdFquA372ZldllelUdTqc.jar' is inaccessible.
Dec 09, 2021 2:03:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-09T14:03:57.643Z: Staged package netty-common-4.1.52.Final-DR8QHk231TDtBNfKvVfZXAdQ8Fm7az-t_tDIDUc0xxc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-common-4.1.52.Final-DR8QHk231TDtBNfKvVfZXAdQ8Fm7az-t_tDIDUc0xxc.jar' is inaccessible.
Dec 09, 2021 2:03:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-09T14:03:57.687Z: Staged package netty-handler-4.1.52.Final-LNwh-_rgSUDIWSkDl3nJP1H2GPXvshYHkxnc_PMqVcE.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-handler-4.1.52.Final-LNwh-_rgSUDIWSkDl3nJP1H2GPXvshYHkxnc_PMqVcE.jar' is inaccessible.
Dec 09, 2021 2:03:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-09T14:03:57.729Z: Staged package netty-handler-proxy-4.1.52.Final-6NB4Y860vUgG4lWh1TLIZQ8POy1NAGrW0hgS6UG0L9w.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-handler-proxy-4.1.52.Final-6NB4Y860vUgG4lWh1TLIZQ8POy1NAGrW0hgS6UG0L9w.jar' is inaccessible.
Dec 09, 2021 2:03:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-09T14:03:57.780Z: Staged package netty-resolver-4.1.52.Final-butyojOXnLRbC01tRsWkEzLFfAz4qAOx78YNu-y6CaY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-resolver-4.1.52.Final-butyojOXnLRbC01tRsWkEzLFfAz4qAOx78YNu-y6CaY.jar' is inaccessible.
Dec 09, 2021 2:03:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-09T14:03:57.836Z: Staged package netty-transport-4.1.52.Final-mj5vjA5V3jY-seoQ_ngXl-yjlOYhht8q4LTrK84LRUE.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-transport-4.1.52.Final-mj5vjA5V3jY-seoQ_ngXl-yjlOYhht8q4LTrK84LRUE.jar' is inaccessible.
Dec 09, 2021 2:03:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-09T14:03:57.873Z: Staged package opencensus-api-0.28.0-DBcj8_bTBhMjhFzouIs1_dpQCBLgp1uOtfzErYyHGpU.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/opencensus-api-0.28.0-DBcj8_bTBhMjhFzouIs1_dpQCBLgp1uOtfzErYyHGpU.jar' is inaccessible.
Dec 09, 2021 2:03:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-09T14:03:57.987Z: Staged package paranamer-2.8-aIyxGKYCHYGROOhVIIyVYDFoi-S0eiS7YVvsxjrO3wc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/paranamer-2.8-aIyxGKYCHYGROOhVIIyVYDFoi-S0eiS7YVvsxjrO3wc.jar' is inaccessible.
Dec 09, 2021 2:03:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-09T14:03:58.021Z: Staged package perfmark-api-0.23.0-xwW1wQwY_zAyuegXQrwvaw5WB_am38DIrQz_ddSRMEI.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/perfmark-api-0.23.0-xwW1wQwY_zAyuegXQrwvaw5WB_am38DIrQz_ddSRMEI.jar' is inaccessible.
Dec 09, 2021 2:03:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-09T14:03:58.461Z: Staged package snappy-java-1.1.8.4-JMTR_B6J4HgzGrj0AamcrWhZm95KLkUWBCy1SMUbHD4.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/snappy-java-1.1.8.4-JMTR_B6J4HgzGrj0AamcrWhZm95KLkUWBCy1SMUbHD4.jar' is inaccessible.
Dec 09, 2021 2:03:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-12-09T14:03:58.657Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Dec 09, 2021 4:00:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-09T16:00:35.716Z: Cancel request is committed for workflow job: 2021-12-09_04_45_32-5426460733272879647.
Dec 09, 2021 4:00:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-09T16:00:35.775Z: Cleaning up.
Dec 09, 2021 4:00:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-09T16:00:35.840Z: Stopping **** pool...
Dec 09, 2021 4:00:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-09T16:00:35.883Z: Stopping **** pool...
Dec 09, 2021 4:02:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-09T16:02:57.594Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Dec 09, 2021 4:02:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-09T16:02:57.645Z: Worker pool stopped.
Dec 09, 2021 4:03:04 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-12-09_04_45_32-5426460733272879647 finished with status CANCELLED.
Load test results for test (ID): 8e65f303-f75b-47e3-98fb-ae3c0a81809a and timestamp: 2021-12-09T12:45:27.262000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11485.904
dataflow_v2_java11_total_bytes_count             2.31259995E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211209124333
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:6fc362308a5b8eeb128bfb04e8dd64602273890055124df7791dc1c938246498
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211209124333]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:6fc362308a5b8eeb128bfb04e8dd64602273890055124df7791dc1c938246498]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211209124333] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:6fc362308a5b8eeb128bfb04e8dd64602273890055124df7791dc1c938246498])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:6fc362308a5b8eeb128bfb04e8dd64602273890055124df7791dc1c938246498
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:6fc362308a5b8eeb128bfb04e8dd64602273890055124df7791dc1c938246498
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:6fc362308a5b8eeb128bfb04e8dd64602273890055124df7791dc1c938246498].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 48s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/woqhzjvimojh4

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure


---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #174

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/174/display/redirect?page=changes>

Changes:

[heejong] [BEAM-13092] Adding dummy external transform translators for Dataflow

[stranniknm] [BEAM-13112]: playground embedded version

[zyichi] [BEAM-13388] Fix broken google cloud dlp test.

[relax] don't store unserializable DatasetService as a member variable

[noreply] [BEAM-13236] Properly close kinesis producer on teardown (#15955)

[noreply] Merge pull request #16150 from [BEAM-13396][Playground][Bugfix] Issues

[noreply] Merge pull request #16148 from [BEAM-13394][Playground] [Bugfix] Fix

[noreply] Avoid overriding explicit portable job submission disabling. (#16143)

[noreply] Merge pull request #16151 from [BEAM-13350][Playground] Support running

[zyichi] [BEAM-13373] Increase python post commit timeout to reduce chance of

[melissapa] [BEAM-11758] Final cleanup for Beam Basics doc content

[msbukal] Exclude FhirIOPatientEverything from v2 dataflow runner intg test.

[noreply] [BEAM-13371] Fix bug where DataFrame overview snippets don't show up

[dpcollins] Add a workaround for https://github.com/googleapis/gax-java/issues/1577

[noreply] [BEAM-12976] Implement pipeline visitor to get global field access in…

[noreply] [BEAM-13388] Use 3.0.0 as lower bound for google-cloud-dlp (#16164)

[noreply] Merge pull request #16127 from [BEAM-13366] [Playground] Add support


------------------------------------------
[...truncated 48.88 KB...]
a81f1846a0d2: Preparing
aa7ec7dfc8db: Waiting
3b441d7cb46b: Preparing
d3710de04cb3: Preparing
927f9fcef4cf: Waiting
91f7336bbfff: Preparing
e2e8c39e0f77: Preparing
3b441d7cb46b: Waiting
d3710de04cb3: Waiting
91f7336bbfff: Waiting
e2e8c39e0f77: Waiting
5c81f9330d99: Waiting
27fdb79dd13a: Waiting
f1bf480ea614: Waiting
9d28abd78e00: Waiting
40d28832fb02: Waiting
a81f1846a0d2: Waiting
14c6fe2aabd9: Waiting
9c4912baf16d: Pushed
40287395a625: Pushed
5cb939ca3ff9: Pushed
e211da1ccfa6: Pushed
27fdb79dd13a: Pushed
35e0c0e168c1: Pushed
9d28abd78e00: Pushed
a6d14faf6ddb: Pushed
aa7ec7dfc8db: Pushed
f1bf480ea614: Pushed
5c81f9330d99: Layer already exists
40d28832fb02: Pushed
927f9fcef4cf: Layer already exists
3b441d7cb46b: Layer already exists
a81f1846a0d2: Layer already exists
91f7336bbfff: Layer already exists
d3710de04cb3: Layer already exists
e2e8c39e0f77: Layer already exists
14c6fe2aabd9: Pushed
20211208124335: digest: sha256:6fbf7af1c11155cb57821773962348b6b456c2c572fe026b2d5791065f5c7205 size: 4311

> Task :sdks:java:testing:load-tests:run
Dec 08, 2021 12:45:30 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Dec 08, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 203 files. Enable logging at DEBUG level to see which files will be staged.
Dec 08, 2021 12:45:31 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Dec 08, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Dec 08, 2021 12:45:34 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 203 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Dec 08, 2021 12:45:34 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 203 files cached, 0 files newly uploaded in 0 seconds
Dec 08, 2021 12:45:34 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Dec 08, 2021 12:45:34 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <113691 bytes, hash 49687711aa323dd7890ed414b0ec0b8666e3f6958cb1fe9bf1a0f53d929cd00c> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-SWh3EaoyPdeJDtQUsOwLhmbj9pWMsf6b8aD1PZKc0Aw.pb
Dec 08, 2021 12:45:36 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Dec 08, 2021 12:45:36 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6fa13e6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3af7d855, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77049094, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@f88bfbe, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@59bbe88a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5d8ab698, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@ed91d8d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@446626a7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@429f7919, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4a2929a4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@cda6019, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@797c3c3b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4012d5bc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4375b013, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1cf0cacc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4f5b08d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@529c2a9a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c87fdf2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26bbe604, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@fe34b86]
Dec 08, 2021 12:45:36 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Dec 08, 2021 12:45:36 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Dec 08, 2021 12:45:36 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Dec 08, 2021 12:45:36 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Dec 08, 2021 12:45:36 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@f324455, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3a894088, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@370c1968, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2f0bfe17, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3206174f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@42fb8c87, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@15eb0ae9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@65e0b505, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@67de7a99, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@795f5d51, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@34aeacd1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@54067fdc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4098dd77, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43aeb5e0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2274160, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@65383667, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@63cd2cd2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@557a84fe, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6deee370, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@49c17ba4]
Dec 08, 2021 12:45:36 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Dec 08, 2021 12:45:36 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Dec 08, 2021 12:45:36 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Dec 08, 2021 12:45:36 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Dec 08, 2021 12:45:36 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Dec 08, 2021 12:45:36 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Dec 08, 2021 12:45:36 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Dec 08, 2021 12:45:36 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Dec 08, 2021 12:45:36 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Dec 08, 2021 12:45:36 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Dec 08, 2021 12:45:36 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Dec 08, 2021 12:45:36 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.36.0-SNAPSHOT
Dec 08, 2021 12:45:37 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-12-08_04_45_36-18125747131358354902?project=apache-beam-testing
Dec 08, 2021 12:45:37 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-12-08_04_45_36-18125747131358354902
Dec 08, 2021 12:45:37 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-12-08_04_45_36-18125747131358354902
Dec 08, 2021 12:45:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-12-08T12:45:54.921Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-12-287c. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Dec 08, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-08T12:46:00.429Z: Worker configuration: e2-standard-2 in us-central1-a.
Dec 08, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-08T12:46:01.193Z: Expanding SplittableParDo operations into optimizable parts.
Dec 08, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-08T12:46:01.227Z: Expanding CollectionToSingleton operations into optimizable parts.
Dec 08, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-08T12:46:01.323Z: Expanding CoGroupByKey operations into optimizable parts.
Dec 08, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-08T12:46:01.411Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Dec 08, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-08T12:46:01.432Z: Expanding GroupByKey operations into streaming Read/Write steps
Dec 08, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-08T12:46:01.492Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Dec 08, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-08T12:46:01.625Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Dec 08, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-08T12:46:01.661Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Dec 08, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-08T12:46:01.692Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Dec 08, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-08T12:46:01.751Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Dec 08, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-08T12:46:01.783Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Dec 08, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-08T12:46:01.814Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Dec 08, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-08T12:46:01.846Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Dec 08, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-08T12:46:01.880Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Dec 08, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-08T12:46:01.912Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Dec 08, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-08T12:46:01.944Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Dec 08, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-08T12:46:01.993Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Dec 08, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-08T12:46:02.036Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Dec 08, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-08T12:46:02.075Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Dec 08, 2021 12:46:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-08T12:46:02.111Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Dec 08, 2021 12:46:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-08T12:46:02.160Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Dec 08, 2021 12:46:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-08T12:46:02.196Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Dec 08, 2021 12:46:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-08T12:46:02.233Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Dec 08, 2021 12:46:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-08T12:46:02.267Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Dec 08, 2021 12:46:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-08T12:46:02.299Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Dec 08, 2021 12:46:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-08T12:46:02.332Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Dec 08, 2021 12:46:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-08T12:46:02.359Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Dec 08, 2021 12:46:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-08T12:46:02.412Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Dec 08, 2021 12:46:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-08T12:46:02.446Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Dec 08, 2021 12:46:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-08T12:46:02.873Z: Starting 5 ****s in us-central1-a...
Dec 08, 2021 12:46:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-08T12:46:25.381Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Dec 08, 2021 12:46:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-08T12:46:51.422Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Dec 08, 2021 12:47:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-08T12:47:47.797Z: Workers have started successfully.
Dec 08, 2021 12:47:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-08T12:47:47.829Z: Workers have started successfully.
Dec 08, 2021 4:00:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-08T16:00:36.671Z: Cancel request is committed for workflow job: 2021-12-08_04_45_36-18125747131358354902.
Dec 08, 2021 4:00:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-08T16:00:36.788Z: Cleaning up.
Dec 08, 2021 4:00:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-08T16:00:36.862Z: Stopping **** pool...
Dec 08, 2021 4:00:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-08T16:00:36.939Z: Stopping **** pool...
Dec 08, 2021 4:03:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-08T16:03:01.170Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Dec 08, 2021 4:03:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-08T16:03:01.210Z: Worker pool stopped.
Dec 08, 2021 4:03:07 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-12-08_04_45_36-18125747131358354902 finished with status CANCELLED.
Load test results for test (ID): 33695c4a-5fe0-4038-aae3-07348008d0fa and timestamp: 2021-12-08T12:45:30.982000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11525.754
dataflow_v2_java11_total_bytes_count             2.08982509E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211208124335
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:6fbf7af1c11155cb57821773962348b6b456c2c572fe026b2d5791065f5c7205
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211208124335]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:6fbf7af1c11155cb57821773962348b6b456c2c572fe026b2d5791065f5c7205]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211208124335] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:6fbf7af1c11155cb57821773962348b6b456c2c572fe026b2d5791065f5c7205])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:6fbf7af1c11155cb57821773962348b6b456c2c572fe026b2d5791065f5c7205
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:6fbf7af1c11155cb57821773962348b6b456c2c572fe026b2d5791065f5c7205
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:6fbf7af1c11155cb57821773962348b6b456c2c572fe026b2d5791065f5c7205].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 49s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date

Publishing build scan...

Publishing failed.

The build scan server appears to be unavailable.
Please check https://status.gradle.com for the latest service status.

If the service is reported as available, please report this problem via https://gradle.com/help/plugin and include the following via copy/paste:

----------
Gradle version: 6.9.1
Plugin version: 3.4.1
Request URL: https://status.gradle.com
Request ID: 7032d7e9-65d9-4731-bfdd-41ffe1371627
Response status code: 405
Response server type: Varnish
----------

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #173

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/173/display/redirect?page=changes>

Changes:

[Daniel Oliveira] Minor Gradle Fixups: Removing unneeded dependency in expansion service.

[aydar.zaynutdinov] [BEAM-13363][Playground]

[aydar.zaynutdinov] [BEAM-13357][Playground]

[aydar.zaynutdinov] [BEAM-13357][Playground]

[Luke Cwik] [BEAM-13015] Simplify fake state API testing client and make it

[zyichi] [BEAM-13380] Fix broken seedjob.

[aydar.zaynutdinov] [BEAM-13357][Playground]

[noreply] Merge pull request #16137 from [BEAM-13381][Playground][Bugfix] Update

[noreply] Merge pull request #16107 from [BEAM-13332] [Playground] Add logs output

[noreply] Merge pull request #16109 from [BEAM-13330][Playground] Save Java logs

[noreply] Merge pull request #16125 from [BEAM-13322][Playground] Support Java

[melissapa] [BEAM-11758] Update basics page: Window, Watermark


------------------------------------------
[...truncated 48.44 KB...]
The push refers to repository [us.gcr.io/apache-beam-testing/java-postcommit-it/java]
271c32230488: Preparing
726a61695ab5: Preparing
9a1d1db61f48: Preparing
ececa2c8bc7a: Preparing
e29ba269002e: Preparing
c1a2f2951374: Preparing
3b2ceb1f7cca: Preparing
c2c0a62080b9: Preparing
c1268161e168: Preparing
b2e8fca4b0ea: Preparing
c1b9b416beeb: Preparing
8b5c1dd46102: Preparing
5c81f9330d99: Preparing
927f9fcef4cf: Preparing
a81f1846a0d2: Preparing
3b441d7cb46b: Preparing
d3710de04cb3: Preparing
91f7336bbfff: Preparing
e2e8c39e0f77: Preparing
c1a2f2951374: Waiting
3b2ceb1f7cca: Waiting
3b441d7cb46b: Waiting
b2e8fca4b0ea: Waiting
927f9fcef4cf: Waiting
a81f1846a0d2: Waiting
8b5c1dd46102: Waiting
c1b9b416beeb: Waiting
d3710de04cb3: Waiting
c1268161e168: Waiting
e2e8c39e0f77: Waiting
5c81f9330d99: Waiting
c2c0a62080b9: Waiting
e29ba269002e: Pushed
9a1d1db61f48: Pushed
726a61695ab5: Pushed
ececa2c8bc7a: Pushed
c2c0a62080b9: Pushed
c1a2f2951374: Pushed
271c32230488: Pushed
3b2ceb1f7cca: Pushed
8b5c1dd46102: Pushed
5c81f9330d99: Layer already exists
c1b9b416beeb: Pushed
c1268161e168: Pushed
3b441d7cb46b: Layer already exists
927f9fcef4cf: Layer already exists
d3710de04cb3: Layer already exists
e2e8c39e0f77: Layer already exists
91f7336bbfff: Layer already exists
a81f1846a0d2: Layer already exists
b2e8fca4b0ea: Pushed
20211207124330: digest: sha256:c57efefbfb8148143d6649d313cfe5976a74525602f128add8238605ea64fcdf size: 4311

> Task :sdks:java:testing:load-tests:run
Dec 07, 2021 12:45:23 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Dec 07, 2021 12:45:24 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 197 files. Enable logging at DEBUG level to see which files will be staged.
Dec 07, 2021 12:45:24 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Dec 07, 2021 12:45:24 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Dec 07, 2021 12:45:27 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 197 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Dec 07, 2021 12:45:27 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 197 files cached, 0 files newly uploaded in 0 seconds
Dec 07, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Dec 07, 2021 12:45:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <111769 bytes, hash a233856ecb74030dd04be2b5678462ed1d5deb20cecf53897c3ba34cd21372e5> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-ojOFbst0Aw3QS-K1Z4Ri7R1d6yDOz1OJfDujTNITcuU.pb
Dec 07, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Dec 07, 2021 12:45:29 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@480ad82c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4d18b73a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@177515d1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@52ff99cd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4c2af006, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@44032fde, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b676112, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5578be42, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4e49ce2b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@136965e3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@53c6f96d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@435cc7f9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4364712f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1b7a52dd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7f93dd4e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5ad5be4a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ad85136, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@737d100a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12e5da86, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6535117e]
Dec 07, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Dec 07, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Dec 07, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Dec 07, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Dec 07, 2021 12:45:29 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58437801, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6af5bbd0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@76464795, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@b34832b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@48f4713c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@f1868c9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4ba6ec50, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@642413d4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@fb2e3fd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43a09ce2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3f183caa, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b66322e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@63538bb4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@24534cb0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5a50d9fc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@106d77da, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6f9c5048, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5114b7c7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@dd71b20, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@767f6ee7]
Dec 07, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Dec 07, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Dec 07, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Dec 07, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Dec 07, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Dec 07, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Dec 07, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Dec 07, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Dec 07, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Dec 07, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Dec 07, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Dec 07, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.36.0-SNAPSHOT
Dec 07, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-12-07_04_45_29-6626945001967699222?project=apache-beam-testing
Dec 07, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-12-07_04_45_29-6626945001967699222
Dec 07, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-12-07_04_45_29-6626945001967699222
Dec 07, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-12-07T12:45:38.017Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-12-sx2g. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Dec 07, 2021 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-07T12:45:45.494Z: Worker configuration: e2-standard-2 in us-central1-a.
Dec 07, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-07T12:45:46.202Z: Expanding SplittableParDo operations into optimizable parts.
Dec 07, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-07T12:45:46.240Z: Expanding CollectionToSingleton operations into optimizable parts.
Dec 07, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-07T12:45:46.312Z: Expanding CoGroupByKey operations into optimizable parts.
Dec 07, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-07T12:45:46.374Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Dec 07, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-07T12:45:46.407Z: Expanding GroupByKey operations into streaming Read/Write steps
Dec 07, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-07T12:45:46.470Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Dec 07, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-07T12:45:46.553Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Dec 07, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-07T12:45:46.576Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Dec 07, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-07T12:45:46.602Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Dec 07, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-07T12:45:46.647Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Dec 07, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-07T12:45:46.675Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Dec 07, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-07T12:45:46.701Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Dec 07, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-07T12:45:46.724Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Dec 07, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-07T12:45:46.748Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Dec 07, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-07T12:45:46.778Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Dec 07, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-07T12:45:46.811Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Dec 07, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-07T12:45:46.836Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Dec 07, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-07T12:45:46.867Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Dec 07, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-07T12:45:46.911Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Dec 07, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-07T12:45:46.943Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Dec 07, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-07T12:45:46.984Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Dec 07, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-07T12:45:47.007Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Dec 07, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-07T12:45:47.044Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Dec 07, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-07T12:45:47.091Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Dec 07, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-07T12:45:47.119Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Dec 07, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-07T12:45:47.145Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Dec 07, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-07T12:45:47.167Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Dec 07, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-07T12:45:47.240Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Dec 07, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-07T12:45:47.276Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Dec 07, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-07T12:45:48.389Z: Starting 5 ****s in us-central1-a...
Dec 07, 2021 12:46:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-07T12:45:59.126Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Dec 07, 2021 12:46:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-07T12:46:32.753Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Dec 07, 2021 12:47:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-07T12:47:29.975Z: Workers have started successfully.
Dec 07, 2021 12:47:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-07T12:47:30.004Z: Workers have started successfully.
Dec 07, 2021 4:00:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-07T16:00:37.032Z: Cancel request is committed for workflow job: 2021-12-07_04_45_29-6626945001967699222.
Dec 07, 2021 4:00:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-07T16:00:37.695Z: Cleaning up.
Dec 07, 2021 4:00:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-07T16:00:37.793Z: Stopping **** pool...
Dec 07, 2021 4:00:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-07T16:00:37.872Z: Stopping **** pool...
Dec 07, 2021 4:02:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-07T16:02:57.835Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Dec 07, 2021 4:02:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-07T16:02:57.865Z: Worker pool stopped.
Dec 07, 2021 4:03:04 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-12-07_04_45_29-6626945001967699222 finished with status CANCELLED.
Load test results for test (ID): dd30530b-7af9-4c29-b78e-45b2439a0ef0 and timestamp: 2021-12-07T12:45:24.625000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11547.852
dataflow_v2_java11_total_bytes_count             1.40517518E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211207124330
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:c57efefbfb8148143d6649d313cfe5976a74525602f128add8238605ea64fcdf
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211207124330]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:c57efefbfb8148143d6649d313cfe5976a74525602f128add8238605ea64fcdf]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211207124330] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:c57efefbfb8148143d6649d313cfe5976a74525602f128add8238605ea64fcdf])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:c57efefbfb8148143d6649d313cfe5976a74525602f128add8238605ea64fcdf
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:c57efefbfb8148143d6649d313cfe5976a74525602f128add8238605ea64fcdf
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:c57efefbfb8148143d6649d313cfe5976a74525602f128add8238605ea64fcdf].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 51s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/tq5dwqa5ycmgy

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #172

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/172/display/redirect?page=changes>

Changes:

[noreply] Merge pull request #16118 from [BEAM-13362][Playground] python ci cd


------------------------------------------
[...truncated 49.22 KB...]
97ff91347834: Preparing
9b06cfb03e9d: Preparing
9663fdc07379: Preparing
4496f816e1b9: Preparing
0e80a617bb77: Preparing
47924046174a: Preparing
745b84e806b4: Preparing
5c81f9330d99: Preparing
927f9fcef4cf: Preparing
a81f1846a0d2: Preparing
3b441d7cb46b: Preparing
d3710de04cb3: Preparing
91f7336bbfff: Preparing
97ff91347834: Waiting
e2e8c39e0f77: Preparing
4496f816e1b9: Waiting
0e80a617bb77: Waiting
47924046174a: Waiting
9b06cfb03e9d: Waiting
745b84e806b4: Waiting
9663fdc07379: Waiting
5c81f9330d99: Waiting
d3710de04cb3: Waiting
91f7336bbfff: Waiting
a81f1846a0d2: Waiting
3b441d7cb46b: Waiting
e2e8c39e0f77: Waiting
b6f5d0825c23: Pushed
7bf7eab0c69d: Pushed
f00185611714: Pushed
9afec5a4e288: Pushed
6a0c9ee8153e: Pushed
97ff91347834: Pushed
9663fdc07379: Pushed
4496f816e1b9: Pushed
5c81f9330d99: Layer already exists
927f9fcef4cf: Layer already exists
a81f1846a0d2: Layer already exists
3b441d7cb46b: Layer already exists
d3710de04cb3: Layer already exists
47924046174a: Pushed
91f7336bbfff: Layer already exists
e2e8c39e0f77: Layer already exists
9b06cfb03e9d: Pushed
745b84e806b4: Pushed
0e80a617bb77: Pushed
20211206124332: digest: sha256:572b1a30b1310735b262a0c8f30816ee780cc6e86400786523db0855a3b54617 size: 4311

> Task :sdks:java:testing:load-tests:run
Dec 06, 2021 12:45:20 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Dec 06, 2021 12:45:20 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 197 files. Enable logging at DEBUG level to see which files will be staged.
Dec 06, 2021 12:45:21 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Dec 06, 2021 12:45:21 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Dec 06, 2021 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 197 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Dec 06, 2021 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 197 files cached, 0 files newly uploaded in 0 seconds
Dec 06, 2021 12:45:24 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Dec 06, 2021 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <111769 bytes, hash 6a52a15ac2806921d65edc8b2b04c35f27c179449ed75e09b1e568b06cff53d2> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-alKhWsKAaSHWXtyLKwTDXyfBeUSe114JseVosGz_U9I.pb
Dec 06, 2021 12:45:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Dec 06, 2021 12:45:26 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@480ad82c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4d18b73a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@177515d1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@52ff99cd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4c2af006, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@44032fde, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b676112, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5578be42, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4e49ce2b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@136965e3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@53c6f96d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@435cc7f9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4364712f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1b7a52dd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7f93dd4e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5ad5be4a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ad85136, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@737d100a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12e5da86, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6535117e]
Dec 06, 2021 12:45:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Dec 06, 2021 12:45:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Dec 06, 2021 12:45:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Dec 06, 2021 12:45:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Dec 06, 2021 12:45:26 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58437801, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6af5bbd0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@76464795, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@b34832b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@48f4713c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@f1868c9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4ba6ec50, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@642413d4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@fb2e3fd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43a09ce2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3f183caa, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b66322e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@63538bb4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@24534cb0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5a50d9fc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@106d77da, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6f9c5048, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5114b7c7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@dd71b20, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@767f6ee7]
Dec 06, 2021 12:45:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Dec 06, 2021 12:45:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Dec 06, 2021 12:45:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Dec 06, 2021 12:45:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Dec 06, 2021 12:45:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Dec 06, 2021 12:45:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Dec 06, 2021 12:45:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Dec 06, 2021 12:45:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Dec 06, 2021 12:45:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Dec 06, 2021 12:45:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Dec 06, 2021 12:45:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Dec 06, 2021 12:45:26 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.36.0-SNAPSHOT
Dec 06, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-12-06_04_45_26-1700801981759437675?project=apache-beam-testing
Dec 06, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-12-06_04_45_26-1700801981759437675
Dec 06, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-12-06_04_45_26-1700801981759437675
Dec 06, 2021 12:45:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-12-06T12:45:33.536Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-12-jbsc. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Dec 06, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-06T12:45:38.362Z: Worker configuration: e2-standard-2 in us-central1-a.
Dec 06, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-06T12:45:39.037Z: Expanding SplittableParDo operations into optimizable parts.
Dec 06, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-06T12:45:39.068Z: Expanding CollectionToSingleton operations into optimizable parts.
Dec 06, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-06T12:45:39.117Z: Expanding CoGroupByKey operations into optimizable parts.
Dec 06, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-06T12:45:39.178Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Dec 06, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-06T12:45:39.207Z: Expanding GroupByKey operations into streaming Read/Write steps
Dec 06, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-06T12:45:39.275Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Dec 06, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-06T12:45:39.393Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Dec 06, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-06T12:45:39.418Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Dec 06, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-06T12:45:39.449Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Dec 06, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-06T12:45:39.483Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Dec 06, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-06T12:45:39.506Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Dec 06, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-06T12:45:39.534Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Dec 06, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-06T12:45:39.563Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Dec 06, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-06T12:45:39.598Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Dec 06, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-06T12:45:39.631Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Dec 06, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-06T12:45:39.654Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Dec 06, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-06T12:45:39.681Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Dec 06, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-06T12:45:39.703Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Dec 06, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-06T12:45:39.734Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Dec 06, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-06T12:45:39.769Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Dec 06, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-06T12:45:39.804Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Dec 06, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-06T12:45:39.838Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Dec 06, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-06T12:45:39.873Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Dec 06, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-06T12:45:39.907Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Dec 06, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-06T12:45:39.942Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Dec 06, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-06T12:45:39.969Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Dec 06, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-06T12:45:39.999Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Dec 06, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-06T12:45:40.037Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Dec 06, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-06T12:45:40.068Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Dec 06, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-06T12:45:40.385Z: Starting 5 ****s in us-central1-a...
Dec 06, 2021 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-06T12:45:57.339Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Dec 06, 2021 12:46:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-06T12:46:25.054Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Dec 06, 2021 12:47:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-06T12:47:23.696Z: Workers have started successfully.
Dec 06, 2021 12:47:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-06T12:47:23.742Z: Workers have started successfully.
Dec 06, 2021 3:57:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-06T15:57:41.624Z: Staged package classgraph-4.8.104-ePvfM6WdiJMBzk05ytbzmopew--T1rqGqw_PYcvi6h0.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/classgraph-4.8.104-ePvfM6WdiJMBzk05ytbzmopew--T1rqGqw_PYcvi6h0.jar' is inaccessible.
Dec 06, 2021 3:57:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-06T15:57:43.985Z: Staged package junit-4.13.1-wwcZ25dNZFJ5P-GRs2OKV3cAVIW64UWSQERTD_pfYSI.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/junit-4.13.1-wwcZ25dNZFJ5P-GRs2OKV3cAVIW64UWSQERTD_pfYSI.jar' is inaccessible.
Dec 06, 2021 3:57:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-12-06T15:57:44.980Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Dec 06, 2021 4:00:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-06T16:00:36.710Z: Cancel request is committed for workflow job: 2021-12-06_04_45_26-1700801981759437675.
Dec 06, 2021 4:00:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-06T16:00:36.754Z: Cleaning up.
Dec 06, 2021 4:00:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-06T16:00:36.829Z: Stopping **** pool...
Dec 06, 2021 4:00:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-06T16:00:36.895Z: Stopping **** pool...
Dec 06, 2021 4:03:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-06T16:03:01.116Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Dec 06, 2021 4:03:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-06T16:03:01.146Z: Worker pool stopped.
Dec 06, 2021 4:03:07 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-12-06_04_45_26-1700801981759437675 finished with status CANCELLED.
Load test results for test (ID): 573eb212-12de-4267-8180-bb1d133d9b99 and timestamp: 2021-12-06T12:45:20.917000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11562.718
dataflow_v2_java11_total_bytes_count             1.50949697E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211206124332
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:572b1a30b1310735b262a0c8f30816ee780cc6e86400786523db0855a3b54617
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211206124332]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:572b1a30b1310735b262a0c8f30816ee780cc6e86400786523db0855a3b54617]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211206124332] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:572b1a30b1310735b262a0c8f30816ee780cc6e86400786523db0855a3b54617])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:572b1a30b1310735b262a0c8f30816ee780cc6e86400786523db0855a3b54617
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:572b1a30b1310735b262a0c8f30816ee780cc6e86400786523db0855a3b54617
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:572b1a30b1310735b262a0c8f30816ee780cc6e86400786523db0855a3b54617].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 50s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/pwrhl75m3uyru

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #171

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/171/display/redirect>

Changes:


------------------------------------------
[...truncated 48.72 KB...]
The push refers to repository [us.gcr.io/apache-beam-testing/java-postcommit-it/java]
8f591258c8cc: Preparing
b7ee6d446b95: Preparing
4de9b9373a95: Preparing
2efcee59fe64: Preparing
83384815c3ff: Preparing
698f82620171: Preparing
da02b3bc4ded: Preparing
5b928413235f: Preparing
1cbb5ccdd4b0: Preparing
bee9533ee01d: Preparing
e62ec33cc1b2: Preparing
f92d8dd645b0: Preparing
5c81f9330d99: Preparing
927f9fcef4cf: Preparing
a81f1846a0d2: Preparing
3b441d7cb46b: Preparing
d3710de04cb3: Preparing
91f7336bbfff: Preparing
e2e8c39e0f77: Preparing
f92d8dd645b0: Waiting
d3710de04cb3: Waiting
5c81f9330d99: Waiting
927f9fcef4cf: Waiting
91f7336bbfff: Waiting
e2e8c39e0f77: Waiting
a81f1846a0d2: Waiting
5b928413235f: Waiting
3b441d7cb46b: Waiting
1cbb5ccdd4b0: Waiting
698f82620171: Waiting
da02b3bc4ded: Waiting
bee9533ee01d: Waiting
4de9b9373a95: Pushed
b7ee6d446b95: Pushed
83384815c3ff: Pushed
8f591258c8cc: Pushed
698f82620171: Pushed
2efcee59fe64: Pushed
5b928413235f: Pushed
1cbb5ccdd4b0: Pushed
5c81f9330d99: Layer already exists
927f9fcef4cf: Layer already exists
e62ec33cc1b2: Pushed
3b441d7cb46b: Layer already exists
a81f1846a0d2: Layer already exists
91f7336bbfff: Layer already exists
d3710de04cb3: Layer already exists
e2e8c39e0f77: Layer already exists
da02b3bc4ded: Pushed
f92d8dd645b0: Pushed
bee9533ee01d: Pushed
20211205124332: digest: sha256:07aefa9e0f355d4d4ef54aba3a76b22efc4eb6008d0187d37735b8f8ba6c9163 size: 4311

> Task :sdks:java:testing:load-tests:run
Dec 05, 2021 12:45:36 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Dec 05, 2021 12:45:37 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 197 files. Enable logging at DEBUG level to see which files will be staged.
Dec 05, 2021 12:45:37 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Dec 05, 2021 12:45:37 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Dec 05, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 197 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Dec 05, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 197 files cached, 0 files newly uploaded in 0 seconds
Dec 05, 2021 12:45:40 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Dec 05, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <111769 bytes, hash 9816ff88d3f4a30c7523df597543fda83e0690eff08f5abf1e53b67516fc7bdf> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-mBb_iNP0owx1I99ZdUP9qD4GkO_wj1q_HlO2dRb8e98.pb
Dec 05, 2021 12:45:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Dec 05, 2021 12:45:41 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4d18b73a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@177515d1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@52ff99cd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4c2af006, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@44032fde, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b676112, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5578be42, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4e49ce2b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@136965e3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@53c6f96d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@435cc7f9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4364712f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1b7a52dd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7f93dd4e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5ad5be4a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ad85136, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@737d100a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12e5da86, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6535117e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1d1cbd0f]
Dec 05, 2021 12:45:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Dec 05, 2021 12:45:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Dec 05, 2021 12:45:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Dec 05, 2021 12:45:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Dec 05, 2021 12:45:41 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6af5bbd0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@76464795, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@b34832b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@48f4713c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@f1868c9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4ba6ec50, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@642413d4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@fb2e3fd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43a09ce2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3f183caa, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b66322e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@63538bb4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@24534cb0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5a50d9fc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@106d77da, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6f9c5048, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5114b7c7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@dd71b20, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@767f6ee7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b6c6e70]
Dec 05, 2021 12:45:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Dec 05, 2021 12:45:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Dec 05, 2021 12:45:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Dec 05, 2021 12:45:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Dec 05, 2021 12:45:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Dec 05, 2021 12:45:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Dec 05, 2021 12:45:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Dec 05, 2021 12:45:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Dec 05, 2021 12:45:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Dec 05, 2021 12:45:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Dec 05, 2021 12:45:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Dec 05, 2021 12:45:42 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.36.0-SNAPSHOT
Dec 05, 2021 12:45:42 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-12-05_04_45_42-14422065288940656869?project=apache-beam-testing
Dec 05, 2021 12:45:42 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-12-05_04_45_42-14422065288940656869
Dec 05, 2021 12:45:42 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-12-05_04_45_42-14422065288940656869
Dec 05, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-12-05T12:45:49.356Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-12-n9e9. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Dec 05, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-05T12:45:55.458Z: Worker configuration: e2-standard-2 in us-central1-a.
Dec 05, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-05T12:45:56.097Z: Expanding SplittableParDo operations into optimizable parts.
Dec 05, 2021 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-05T12:45:56.131Z: Expanding CollectionToSingleton operations into optimizable parts.
Dec 05, 2021 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-05T12:45:56.197Z: Expanding CoGroupByKey operations into optimizable parts.
Dec 05, 2021 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-05T12:45:56.272Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Dec 05, 2021 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-05T12:45:56.309Z: Expanding GroupByKey operations into streaming Read/Write steps
Dec 05, 2021 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-05T12:45:56.377Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Dec 05, 2021 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-05T12:45:56.477Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Dec 05, 2021 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-05T12:45:56.512Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Dec 05, 2021 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-05T12:45:56.544Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Dec 05, 2021 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-05T12:45:56.576Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Dec 05, 2021 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-05T12:45:56.610Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Dec 05, 2021 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-05T12:45:56.635Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Dec 05, 2021 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-05T12:45:56.670Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Dec 05, 2021 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-05T12:45:56.704Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Dec 05, 2021 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-05T12:45:56.736Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Dec 05, 2021 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-05T12:45:56.770Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Dec 05, 2021 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-05T12:45:56.796Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Dec 05, 2021 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-05T12:45:56.829Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Dec 05, 2021 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-05T12:45:56.862Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Dec 05, 2021 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-05T12:45:56.893Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Dec 05, 2021 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-05T12:45:56.927Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Dec 05, 2021 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-05T12:45:56.972Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Dec 05, 2021 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-05T12:45:57.014Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Dec 05, 2021 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-05T12:45:57.040Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Dec 05, 2021 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-05T12:45:57.074Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Dec 05, 2021 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-05T12:45:57.108Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Dec 05, 2021 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-05T12:45:57.143Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Dec 05, 2021 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-05T12:45:57.171Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Dec 05, 2021 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-05T12:45:57.203Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Dec 05, 2021 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-05T12:45:57.549Z: Starting 5 ****s in us-central1-a...
Dec 05, 2021 12:46:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-05T12:46:12.165Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Dec 05, 2021 12:46:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-05T12:46:37.046Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Dec 05, 2021 12:47:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-05T12:47:36.725Z: Workers have started successfully.
Dec 05, 2021 12:47:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-05T12:47:36.755Z: Workers have started successfully.
Dec 05, 2021 4:00:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-05T16:00:33.271Z: Cancel request is committed for workflow job: 2021-12-05_04_45_42-14422065288940656869.
Dec 05, 2021 4:00:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-05T16:00:33.347Z: Cleaning up.
Dec 05, 2021 4:00:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-05T16:00:33.427Z: Stopping **** pool...
Dec 05, 2021 4:00:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-05T16:00:33.469Z: Stopping **** pool...
Dec 05, 2021 4:02:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-05T16:02:51.090Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Dec 05, 2021 4:02:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-05T16:02:51.132Z: Worker pool stopped.
Dec 05, 2021 4:02:57 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-12-05_04_45_42-14422065288940656869 finished with status CANCELLED.
Load test results for test (ID): e9f64209-708c-43ba-bd02-87b35c43f3f8 and timestamp: 2021-12-05T12:45:37.387000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11543.136
dataflow_v2_java11_total_bytes_count             2.53889161E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211205124332
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:07aefa9e0f355d4d4ef54aba3a76b22efc4eb6008d0187d37735b8f8ba6c9163
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211205124332]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:07aefa9e0f355d4d4ef54aba3a76b22efc4eb6008d0187d37735b8f8ba6c9163]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211205124332] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:07aefa9e0f355d4d4ef54aba3a76b22efc4eb6008d0187d37735b8f8ba6c9163])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:07aefa9e0f355d4d4ef54aba3a76b22efc4eb6008d0187d37735b8f8ba6c9163
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:07aefa9e0f355d4d4ef54aba3a76b22efc4eb6008d0187d37735b8f8ba6c9163
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:07aefa9e0f355d4d4ef54aba3a76b22efc4eb6008d0187d37735b8f8ba6c9163].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 43s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/ghja2lih3zvok

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #170

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/170/display/redirect?page=changes>

Changes:

[Robert Bradshaw] Revert "Revert "Allow wildcards for java class lookup transform

[Robert Bradshaw] Move stand-alone expansion service jar into its own project.

[noreply] Merge pull request #16117 from [BEAM-13368][Playground][Bugfix] Fix CI

[noreply] Run python SpannerIO IT with python 3.7 only to avoid overload spanner

[noreply] Merge pull request #15378 from [RFC] Define and document per-key

[noreply] Bump python containers to beam-master-20211202 (#16129)

[noreply] [BEAM-13354, BEAM-13015, BEAM-12802, BEAM-12588] Support prefetch for

[zyichi] Fix failing RecommendationAICatalogItemIT

[Valentyn Tymofieiev] Clarify instructions on how to get contributor list.

[noreply] [BEAM-11936] Fix errorprone UnusedVariable in IO (#16036)


------------------------------------------
[...truncated 49.09 KB...]
00126f09d5b7: Preparing
fd2c289dd740: Preparing
9437b340a45c: Preparing
5c81f9330d99: Preparing
927f9fcef4cf: Preparing
fd2c289dd740: Waiting
a81f1846a0d2: Preparing
3b441d7cb46b: Preparing
d3710de04cb3: Preparing
91f7336bbfff: Preparing
e2e8c39e0f77: Preparing
a0b0581a19cd: Waiting
89f7638f548c: Waiting
3b441d7cb46b: Waiting
9437b340a45c: Waiting
00126f09d5b7: Waiting
91f7336bbfff: Waiting
927f9fcef4cf: Waiting
d3710de04cb3: Waiting
e2e8c39e0f77: Waiting
5c81f9330d99: Waiting
91ab7bf1ec06: Pushed
a82ba1ba0f48: Pushed
1a9b9d06b27c: Pushed
499c60a48ffa: Pushed
19a3cff3639a: Pushed
af1cb12c8718: Pushed
a0b0581a19cd: Pushed
89f7638f548c: Pushed
5c81f9330d99: Layer already exists
927f9fcef4cf: Layer already exists
a81f1846a0d2: Layer already exists
3b441d7cb46b: Layer already exists
d3710de04cb3: Layer already exists
fd2c289dd740: Pushed
751487c16c82: Pushed
91f7336bbfff: Layer already exists
e2e8c39e0f77: Layer already exists
9437b340a45c: Pushed
00126f09d5b7: Pushed
20211204124331: digest: sha256:dc8c982f3bb510c2c8e1501a52cfad9709e289905da50aa6caa13a215156f720 size: 4311

> Task :sdks:java:testing:load-tests:run
Dec 04, 2021 12:45:22 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Dec 04, 2021 12:45:22 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 197 files. Enable logging at DEBUG level to see which files will be staged.
Dec 04, 2021 12:45:23 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Dec 04, 2021 12:45:23 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Dec 04, 2021 12:45:26 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 197 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Dec 04, 2021 12:45:26 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 197 files cached, 0 files newly uploaded in 0 seconds
Dec 04, 2021 12:45:26 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Dec 04, 2021 12:45:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <111769 bytes, hash f78bcdd0db7651084d4df8e7ec462faabf4834fa6e16411a39f2abb2c2f20e57> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-94vN0Nt2UQhNTfjn7EYvqr9INPpuFkEaOfKrssLyDlc.pb
Dec 04, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Dec 04, 2021 12:45:28 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@53d13cd4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77865933, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@480ad82c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4d18b73a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@177515d1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@52ff99cd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4c2af006, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@44032fde, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b676112, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5578be42, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4e49ce2b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@136965e3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@53c6f96d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@435cc7f9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4364712f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1b7a52dd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7f93dd4e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5ad5be4a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ad85136, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@737d100a]
Dec 04, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Dec 04, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Dec 04, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Dec 04, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Dec 04, 2021 12:45:28 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5399f6c5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4fe64d23, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58437801, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6af5bbd0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@76464795, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@b34832b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@48f4713c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@f1868c9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4ba6ec50, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@642413d4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@fb2e3fd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43a09ce2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3f183caa, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b66322e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@63538bb4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@24534cb0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5a50d9fc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@106d77da, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6f9c5048, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5114b7c7]
Dec 04, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Dec 04, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Dec 04, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Dec 04, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Dec 04, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Dec 04, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Dec 04, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Dec 04, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Dec 04, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Dec 04, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Dec 04, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Dec 04, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.36.0-SNAPSHOT
Dec 04, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-12-04_04_45_28-4642740283095629077?project=apache-beam-testing
Dec 04, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-12-04_04_45_28-4642740283095629077
Dec 04, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-12-04_04_45_28-4642740283095629077
Dec 04, 2021 12:45:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-12-04T12:45:35.313Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-12-jiav. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Dec 04, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-04T12:45:38.926Z: Worker configuration: e2-standard-2 in us-central1-a.
Dec 04, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-04T12:45:39.592Z: Expanding SplittableParDo operations into optimizable parts.
Dec 04, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-04T12:45:39.630Z: Expanding CollectionToSingleton operations into optimizable parts.
Dec 04, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-04T12:45:39.718Z: Expanding CoGroupByKey operations into optimizable parts.
Dec 04, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-04T12:45:39.790Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Dec 04, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-04T12:45:39.817Z: Expanding GroupByKey operations into streaming Read/Write steps
Dec 04, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-04T12:45:39.869Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Dec 04, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-04T12:45:39.961Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Dec 04, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-04T12:45:39.992Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Dec 04, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-04T12:45:40.026Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Dec 04, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-04T12:45:40.049Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Dec 04, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-04T12:45:40.082Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Dec 04, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-04T12:45:40.118Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Dec 04, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-04T12:45:40.144Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Dec 04, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-04T12:45:40.174Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Dec 04, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-04T12:45:40.199Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Dec 04, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-04T12:45:40.226Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Dec 04, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-04T12:45:40.251Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Dec 04, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-04T12:45:40.274Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Dec 04, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-04T12:45:40.295Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Dec 04, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-04T12:45:40.334Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Dec 04, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-04T12:45:40.364Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Dec 04, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-04T12:45:40.390Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Dec 04, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-04T12:45:40.416Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Dec 04, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-04T12:45:40.457Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Dec 04, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-04T12:45:40.485Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Dec 04, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-04T12:45:40.511Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Dec 04, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-04T12:45:40.533Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Dec 04, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-04T12:45:40.565Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Dec 04, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-04T12:45:40.591Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Dec 04, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-04T12:45:40.923Z: Starting 5 ****s in us-central1-a...
Dec 04, 2021 12:46:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-04T12:46:08.169Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Dec 04, 2021 12:46:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-04T12:46:30.993Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Dec 04, 2021 12:47:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-04T12:47:26.331Z: Workers have started successfully.
Dec 04, 2021 12:47:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-04T12:47:26.355Z: Workers have started successfully.
Dec 04, 2021 1:54:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-04T13:54:41.130Z: Staged package amazon-kinesis-producer-0.14.1-5YYT_oyDFV17VWs8Of43O7N08sRfHu0SuJeqVSFCuLg.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/amazon-kinesis-producer-0.14.1-5YYT_oyDFV17VWs8Of43O7N08sRfHu0SuJeqVSFCuLg.jar' is inaccessible.
Dec 04, 2021 1:54:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-12-04T13:54:44.602Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Dec 04, 2021 1:57:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-12-04T13:57:44.299Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Dec 04, 2021 2:00:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-04T14:00:41.124Z: Staged package amazon-kinesis-producer-0.14.1-5YYT_oyDFV17VWs8Of43O7N08sRfHu0SuJeqVSFCuLg.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/amazon-kinesis-producer-0.14.1-5YYT_oyDFV17VWs8Of43O7N08sRfHu0SuJeqVSFCuLg.jar' is inaccessible.
Dec 04, 2021 2:00:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-12-04T14:00:44.928Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Dec 04, 2021 2:03:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-12-04T14:03:44.111Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Dec 04, 2021 4:00:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-04T16:00:31.584Z: Cancel request is committed for workflow job: 2021-12-04_04_45_28-4642740283095629077.
Dec 04, 2021 4:00:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-04T16:00:31.617Z: Cleaning up.
Dec 04, 2021 4:00:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-04T16:00:31.686Z: Stopping **** pool...
Dec 04, 2021 4:00:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-04T16:00:31.750Z: Stopping **** pool...
Dec 04, 2021 4:02:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-04T16:02:55.163Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Dec 04, 2021 4:02:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-04T16:02:55.197Z: Worker pool stopped.
Dec 04, 2021 4:03:00 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-12-04_04_45_28-4642740283095629077 finished with status CANCELLED.
Load test results for test (ID): 74b0eb54-c82a-4eff-8144-1733ae9f263a and timestamp: 2021-12-04T12:45:22.779000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11503.363
dataflow_v2_java11_total_bytes_count             1.94954657E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211204124331
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:dc8c982f3bb510c2c8e1501a52cfad9709e289905da50aa6caa13a215156f720
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211204124331]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:dc8c982f3bb510c2c8e1501a52cfad9709e289905da50aa6caa13a215156f720]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211204124331] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:dc8c982f3bb510c2c8e1501a52cfad9709e289905da50aa6caa13a215156f720])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:dc8c982f3bb510c2c8e1501a52cfad9709e289905da50aa6caa13a215156f720
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:dc8c982f3bb510c2c8e1501a52cfad9709e289905da50aa6caa13a215156f720
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:dc8c982f3bb510c2c8e1501a52cfad9709e289905da50aa6caa13a215156f720].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 46s
101 actionable tasks: 71 executed, 28 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/ahs7e476yi63a

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #169

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/169/display/redirect?page=changes>

Changes:

[aydar.zaynutdinov] [BEAM-13329][Playground]

[aydar.zaynutdinov] [BEAM-13329][Playground]

[alexander.zhuravlev] [BEAM-13370] Deleted unused prints & strings

[ilya.kozyrev] Fix pylint issues and apply yapf with Beam config

[ilya.kozyrev] fix white spaces

[noreply] Don't pin a particular version of Tensorflow. (#16102)

[noreply] [BEAM-12733] Fix failing integration tests for Java Recommendation AI

[noreply] [BEAM-13288] improve logging for no rows present error (#16096)


------------------------------------------
[...truncated 48.91 KB...]
e2eb060785c6: Preparing
709a030d6410: Preparing
47ba6c76f0c9: Preparing
c1d696a4b3a1: Preparing
22c136e61d12: Preparing
d0c7bda4e558: Preparing
5c81f9330d99: Preparing
927f9fcef4cf: Preparing
a81f1846a0d2: Preparing
3b441d7cb46b: Preparing
d3710de04cb3: Preparing
91f7336bbfff: Preparing
e2e8c39e0f77: Preparing
d0c7bda4e558: Waiting
47ba6c76f0c9: Waiting
d3710de04cb3: Waiting
22c136e61d12: Waiting
91f7336bbfff: Waiting
5c81f9330d99: Waiting
c1d696a4b3a1: Waiting
e2e8c39e0f77: Waiting
3b441d7cb46b: Waiting
a81f1846a0d2: Waiting
709a030d6410: Waiting
a0c9a60fb894: Waiting
191dfd937c71: Pushed
7e9dfde64951: Pushed
a3b0537b01a1: Pushed
247e5c7bec76: Pushed
a0c9a60fb894: Pushed
f53aaaa20428: Pushed
709a030d6410: Pushed
47ba6c76f0c9: Pushed
5c81f9330d99: Layer already exists
927f9fcef4cf: Layer already exists
a81f1846a0d2: Layer already exists
3b441d7cb46b: Layer already exists
d3710de04cb3: Layer already exists
22c136e61d12: Pushed
e2eb060785c6: Pushed
91f7336bbfff: Layer already exists
e2e8c39e0f77: Layer already exists
d0c7bda4e558: Pushed
c1d696a4b3a1: Pushed
20211203124333: digest: sha256:32715126f0b40ff91a6606824fd709ff77d81b6a11c6dc08f9908833c466fff7 size: 4311

> Task :sdks:java:testing:load-tests:run
Dec 03, 2021 12:45:35 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Dec 03, 2021 12:45:35 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 197 files. Enable logging at DEBUG level to see which files will be staged.
Dec 03, 2021 12:45:36 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Dec 03, 2021 12:45:36 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Dec 03, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 197 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Dec 03, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 197 files cached, 0 files newly uploaded in 0 seconds
Dec 03, 2021 12:45:39 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Dec 03, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <111769 bytes, hash bce5e84e0c79f033e20e0b1e119be0d3d85385cdefafa9adadb64b1c121b5982> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-vOXoTgx58DPiDgseEZvg09hThc3vr6mtrbZLHBIbWYI.pb
Dec 03, 2021 12:45:40 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Dec 03, 2021 12:45:40 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@53d13cd4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77865933, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@480ad82c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4d18b73a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@177515d1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@52ff99cd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4c2af006, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@44032fde, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b676112, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5578be42, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4e49ce2b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@136965e3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@53c6f96d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@435cc7f9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4364712f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1b7a52dd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7f93dd4e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5ad5be4a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ad85136, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@737d100a]
Dec 03, 2021 12:45:40 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Dec 03, 2021 12:45:40 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Dec 03, 2021 12:45:40 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Dec 03, 2021 12:45:40 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Dec 03, 2021 12:45:40 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5399f6c5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4fe64d23, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58437801, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6af5bbd0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@76464795, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@b34832b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@48f4713c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@f1868c9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4ba6ec50, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@642413d4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@fb2e3fd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43a09ce2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3f183caa, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b66322e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@63538bb4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@24534cb0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5a50d9fc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@106d77da, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6f9c5048, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5114b7c7]
Dec 03, 2021 12:45:40 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Dec 03, 2021 12:45:40 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Dec 03, 2021 12:45:40 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Dec 03, 2021 12:45:40 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Dec 03, 2021 12:45:40 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Dec 03, 2021 12:45:40 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Dec 03, 2021 12:45:40 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Dec 03, 2021 12:45:40 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Dec 03, 2021 12:45:40 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Dec 03, 2021 12:45:40 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Dec 03, 2021 12:45:40 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Dec 03, 2021 12:45:40 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.36.0-SNAPSHOT
Dec 03, 2021 12:45:41 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-12-03_04_45_40-9024301071661498886?project=apache-beam-testing
Dec 03, 2021 12:45:41 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-12-03_04_45_40-9024301071661498886
Dec 03, 2021 12:45:41 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-12-03_04_45_40-9024301071661498886
Dec 03, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-12-03T12:45:48.216Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-12-b4fr. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Dec 03, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-03T12:45:52.848Z: Worker configuration: e2-standard-2 in us-central1-a.
Dec 03, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-03T12:45:53.407Z: Expanding SplittableParDo operations into optimizable parts.
Dec 03, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-03T12:45:53.437Z: Expanding CollectionToSingleton operations into optimizable parts.
Dec 03, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-03T12:45:53.514Z: Expanding CoGroupByKey operations into optimizable parts.
Dec 03, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-03T12:45:53.573Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Dec 03, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-03T12:45:53.679Z: Expanding GroupByKey operations into streaming Read/Write steps
Dec 03, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-03T12:45:53.728Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Dec 03, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-03T12:45:53.832Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Dec 03, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-03T12:45:53.859Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Dec 03, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-03T12:45:53.892Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Dec 03, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-03T12:45:53.924Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Dec 03, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-03T12:45:53.959Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Dec 03, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-03T12:45:53.990Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Dec 03, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-03T12:45:54.015Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Dec 03, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-03T12:45:54.050Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Dec 03, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-03T12:45:54.097Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Dec 03, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-03T12:45:54.121Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Dec 03, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-03T12:45:54.155Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Dec 03, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-03T12:45:54.202Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Dec 03, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-03T12:45:54.240Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Dec 03, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-03T12:45:54.274Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Dec 03, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-03T12:45:54.303Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Dec 03, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-03T12:45:54.331Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Dec 03, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-03T12:45:54.367Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Dec 03, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-03T12:45:54.399Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Dec 03, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-03T12:45:54.423Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Dec 03, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-03T12:45:54.478Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Dec 03, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-03T12:45:54.504Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Dec 03, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-03T12:45:54.527Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Dec 03, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-03T12:45:54.561Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Dec 03, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-03T12:45:54.864Z: Starting 5 ****s in us-central1-a...
Dec 03, 2021 12:46:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-03T12:46:24.077Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Dec 03, 2021 12:46:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-03T12:46:39.260Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Dec 03, 2021 12:47:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-03T12:47:37.519Z: Workers have started successfully.
Dec 03, 2021 12:47:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-03T12:47:37.549Z: Workers have started successfully.
Dec 03, 2021 2:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-03T14:45:55.930Z: Staged package beam-sdks-java-io-kinesis-2.36.0-SNAPSHOT-dHp3AUpnCyW8BI5_dm34xo4w2zHWNjlCUzZMmsdn-aM.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-kinesis-2.36.0-SNAPSHOT-dHp3AUpnCyW8BI5_dm34xo4w2zHWNjlCUzZMmsdn-aM.jar' is inaccessible.
Dec 03, 2021 2:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-03T14:45:56.068Z: Staged package beam-sdks-java-load-tests-2.36.0-SNAPSHOT-AWXw3244IpmjjLbDXVk6FhB1JxiEeTDrXIDH5IPOx9U.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-load-tests-2.36.0-SNAPSHOT-AWXw3244IpmjjLbDXVk6FhB1JxiEeTDrXIDH5IPOx9U.jar' is inaccessible.
Dec 03, 2021 2:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-12-03T14:45:58.172Z: Staged package jackson-dataformat-cbor-2.13.0-87i8Y8vQiEl2BVWwSbyVD2k9G-KE9nB4qPOXblapNm4.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/jackson-dataformat-cbor-2.13.0-87i8Y8vQiEl2BVWwSbyVD2k9G-KE9nB4qPOXblapNm4.jar' is inaccessible.
Dec 03, 2021 2:46:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-12-03T14:45:59.594Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Dec 03, 2021 4:00:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-03T16:00:36.377Z: Cancel request is committed for workflow job: 2021-12-03_04_45_40-9024301071661498886.
Dec 03, 2021 4:00:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-03T16:00:36.420Z: Cleaning up.
Dec 03, 2021 4:00:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-03T16:00:36.519Z: Stopping **** pool...
Dec 03, 2021 4:00:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-03T16:00:36.589Z: Stopping **** pool...
Dec 03, 2021 4:03:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-03T16:02:59.353Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Dec 03, 2021 4:03:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-03T16:02:59.381Z: Worker pool stopped.
Dec 03, 2021 4:03:06 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-12-03_04_45_40-9024301071661498886 finished with status CANCELLED.
Load test results for test (ID): 2baf55fd-dba5-4a68-95ea-bdae2153e17a and timestamp: 2021-12-03T12:45:35.937000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11541.252
dataflow_v2_java11_total_bytes_count             1.90487712E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211203124333
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:32715126f0b40ff91a6606824fd709ff77d81b6a11c6dc08f9908833c466fff7
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211203124333]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:32715126f0b40ff91a6606824fd709ff77d81b6a11c6dc08f9908833c466fff7]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211203124333] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:32715126f0b40ff91a6606824fd709ff77d81b6a11c6dc08f9908833c466fff7])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:32715126f0b40ff91a6606824fd709ff77d81b6a11c6dc08f9908833c466fff7
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:32715126f0b40ff91a6606824fd709ff77d81b6a11c6dc08f9908833c466fff7
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:32715126f0b40ff91a6606824fd709ff77d81b6a11c6dc08f9908833c466fff7].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 49s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/ws6ap6y5zswgm

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #168

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/168/display/redirect?page=changes>

Changes:

[Robert Bradshaw] Require explicit designation of proxy partitioning specification.

[Robert Bradshaw] Elementwise transforms preserve all partitionings.

[aydar.zaynutdinov] [BEAM-13360][Playground]

[noreply] Merge pull request #16011 from [BEAM-12164] Add models for Spanner

[noreply] [BEAM-13335] Use signed int64 range for DataFrame read unique indexes

[iyi] Add test case for untriggered expansion with temp tables.

[zyichi] Fix small typo in ProcessBundleHandlerTest.java

[noreply] Merge pull request #16079 from [BEAM-13241] [Playground] Frontend

[noreply] Ricardo Case Study (#16087)

[noreply] Merge pull request #16086 from [BEAM-13258][Playground] Get and store

[noreply] [BEAM-11936] Fix errorprone UnusedVariable in Runners (#16035)

[noreply] [BEAM-8123] Add cloudpickle as optional library (#15472)

[noreply] [BEAM-12572] Beam python examples continuously exercised on at least 2


------------------------------------------
[...truncated 48.95 KB...]

The push refers to repository [us.gcr.io/apache-beam-testing/java-postcommit-it/java]
c862c4aeb96b: Preparing
d61fa9a6824b: Preparing
691c0fbd62ab: Preparing
7127b4d552e0: Preparing
860d21c74383: Preparing
0d0f6a907076: Preparing
1bd26ae5df9b: Preparing
3678452d6023: Preparing
70d2d983890a: Preparing
4eebf714a030: Preparing
348562b66da8: Preparing
199c65c9b847: Preparing
5c81f9330d99: Preparing
927f9fcef4cf: Preparing
a81f1846a0d2: Preparing
3b441d7cb46b: Preparing
d3710de04cb3: Preparing
91f7336bbfff: Preparing
e2e8c39e0f77: Preparing
199c65c9b847: Waiting
1bd26ae5df9b: Waiting
d3710de04cb3: Waiting
91f7336bbfff: Waiting
5c81f9330d99: Waiting
3678452d6023: Waiting
e2e8c39e0f77: Waiting
70d2d983890a: Waiting
927f9fcef4cf: Waiting
4eebf714a030: Waiting
a81f1846a0d2: Waiting
0d0f6a907076: Waiting
860d21c74383: Pushed
691c0fbd62ab: Pushed
d61fa9a6824b: Pushed
c862c4aeb96b: Pushed
0d0f6a907076: Pushed
7127b4d552e0: Pushed
3678452d6023: Pushed
70d2d983890a: Pushed
5c81f9330d99: Layer already exists
199c65c9b847: Pushed
1bd26ae5df9b: Pushed
348562b66da8: Pushed
927f9fcef4cf: Layer already exists
a81f1846a0d2: Layer already exists
3b441d7cb46b: Layer already exists
d3710de04cb3: Layer already exists
91f7336bbfff: Layer already exists
e2e8c39e0f77: Layer already exists
4eebf714a030: Pushed
20211202124333: digest: sha256:f4dd28e5ce64b7d74204df409ea446fae9e268806ccceae69cb903a1a744811a size: 4311

> Task :sdks:java:testing:load-tests:run
Dec 02, 2021 12:45:48 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Dec 02, 2021 12:45:48 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 197 files. Enable logging at DEBUG level to see which files will be staged.
Dec 02, 2021 12:45:49 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Dec 02, 2021 12:45:49 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Dec 02, 2021 12:45:52 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 197 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Dec 02, 2021 12:45:52 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 197 files cached, 0 files newly uploaded in 0 seconds
Dec 02, 2021 12:45:52 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Dec 02, 2021 12:45:52 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <111769 bytes, hash 616e89017161d40fdcc1b931fe819a245586534650ee036fc93130a830296f3c> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-YW6JAXFh1A_cwbkx_oGaJFWGU0ZQ7gNvyTEwqDApbzw.pb
Dec 02, 2021 12:45:54 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Dec 02, 2021 12:45:54 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77865933, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@480ad82c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4d18b73a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@177515d1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@52ff99cd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4c2af006, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@44032fde, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b676112, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5578be42, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4e49ce2b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@136965e3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@53c6f96d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@435cc7f9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4364712f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1b7a52dd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7f93dd4e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5ad5be4a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ad85136, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@737d100a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12e5da86]
Dec 02, 2021 12:45:54 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Dec 02, 2021 12:45:54 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Dec 02, 2021 12:45:54 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Dec 02, 2021 12:45:54 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Dec 02, 2021 12:45:54 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4fe64d23, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58437801, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6af5bbd0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@76464795, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@b34832b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@48f4713c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@f1868c9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4ba6ec50, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@642413d4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@fb2e3fd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43a09ce2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3f183caa, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b66322e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@63538bb4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@24534cb0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5a50d9fc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@106d77da, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6f9c5048, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5114b7c7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@dd71b20]
Dec 02, 2021 12:45:54 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Dec 02, 2021 12:45:54 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Dec 02, 2021 12:45:54 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Dec 02, 2021 12:45:54 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Dec 02, 2021 12:45:54 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Dec 02, 2021 12:45:54 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Dec 02, 2021 12:45:54 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Dec 02, 2021 12:45:54 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Dec 02, 2021 12:45:54 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Dec 02, 2021 12:45:54 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Dec 02, 2021 12:45:54 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Dec 02, 2021 12:45:54 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.36.0-SNAPSHOT
Dec 02, 2021 12:45:55 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-12-02_04_45_54-11669014248312983346?project=apache-beam-testing
Dec 02, 2021 12:45:55 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-12-02_04_45_54-11669014248312983346
Dec 02, 2021 12:45:55 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-12-02_04_45_54-11669014248312983346
Dec 02, 2021 12:46:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-12-02T12:46:01.070Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-12-5viu. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Dec 02, 2021 12:46:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-02T12:46:05.318Z: Worker configuration: e2-standard-2 in us-central1-a.
Dec 02, 2021 12:46:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-02T12:46:05.838Z: Expanding SplittableParDo operations into optimizable parts.
Dec 02, 2021 12:46:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-02T12:46:05.864Z: Expanding CollectionToSingleton operations into optimizable parts.
Dec 02, 2021 12:46:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-02T12:46:05.958Z: Expanding CoGroupByKey operations into optimizable parts.
Dec 02, 2021 12:46:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-02T12:46:06.041Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Dec 02, 2021 12:46:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-02T12:46:06.068Z: Expanding GroupByKey operations into streaming Read/Write steps
Dec 02, 2021 12:46:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-02T12:46:06.137Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Dec 02, 2021 12:46:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-02T12:46:06.245Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Dec 02, 2021 12:46:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-02T12:46:06.286Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Dec 02, 2021 12:46:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-02T12:46:06.308Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Dec 02, 2021 12:46:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-02T12:46:06.343Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Dec 02, 2021 12:46:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-02T12:46:06.376Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Dec 02, 2021 12:46:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-02T12:46:06.413Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Dec 02, 2021 12:46:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-02T12:46:06.447Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Dec 02, 2021 12:46:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-02T12:46:06.497Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Dec 02, 2021 12:46:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-02T12:46:06.533Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Dec 02, 2021 12:46:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-02T12:46:06.568Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Dec 02, 2021 12:46:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-02T12:46:06.600Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Dec 02, 2021 12:46:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-02T12:46:06.636Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Dec 02, 2021 12:46:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-02T12:46:06.671Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Dec 02, 2021 12:46:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-02T12:46:06.695Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Dec 02, 2021 12:46:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-02T12:46:06.718Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Dec 02, 2021 12:46:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-02T12:46:06.744Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Dec 02, 2021 12:46:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-02T12:46:06.782Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Dec 02, 2021 12:46:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-02T12:46:06.824Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Dec 02, 2021 12:46:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-02T12:46:06.853Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Dec 02, 2021 12:46:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-02T12:46:06.887Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Dec 02, 2021 12:46:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-02T12:46:06.923Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Dec 02, 2021 12:46:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-02T12:46:06.958Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Dec 02, 2021 12:46:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-02T12:46:06.991Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Dec 02, 2021 12:46:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-02T12:46:07.349Z: Starting 5 ****s in us-central1-a...
Dec 02, 2021 12:46:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-02T12:46:16.735Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Dec 02, 2021 12:46:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-02T12:46:54.900Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Dec 02, 2021 12:47:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-02T12:47:54.497Z: Workers have started successfully.
Dec 02, 2021 12:47:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-02T12:47:54.541Z: Workers have started successfully.
Dec 02, 2021 4:00:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-02T16:00:35.714Z: Cancel request is committed for workflow job: 2021-12-02_04_45_54-11669014248312983346.
Dec 02, 2021 4:00:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-02T16:00:35.788Z: Cleaning up.
Dec 02, 2021 4:00:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-02T16:00:35.862Z: Stopping **** pool...
Dec 02, 2021 4:00:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-02T16:00:35.915Z: Stopping **** pool...
Dec 02, 2021 4:02:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-02T16:02:55.359Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Dec 02, 2021 4:02:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-02T16:02:55.397Z: Worker pool stopped.
Dec 02, 2021 4:03:01 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-12-02_04_45_54-11669014248312983346 finished with status CANCELLED.
Load test results for test (ID): 50148b1c-14be-4ae4-8a3a-291c3ce095a6 and timestamp: 2021-12-02T12:45:48.814000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11528.599
dataflow_v2_java11_total_bytes_count             2.10152696E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211202124333
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:f4dd28e5ce64b7d74204df409ea446fae9e268806ccceae69cb903a1a744811a
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211202124333]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:f4dd28e5ce64b7d74204df409ea446fae9e268806ccceae69cb903a1a744811a]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211202124333] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:f4dd28e5ce64b7d74204df409ea446fae9e268806ccceae69cb903a1a744811a])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:f4dd28e5ce64b7d74204df409ea446fae9e268806ccceae69cb903a1a744811a
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:f4dd28e5ce64b7d74204df409ea446fae9e268806ccceae69cb903a1a744811a
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:f4dd28e5ce64b7d74204df409ea446fae9e268806ccceae69cb903a1a744811a].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 45s
101 actionable tasks: 71 executed, 28 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/ulfhgwrtuua4o

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #167

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/167/display/redirect?page=changes>

Changes:

[mmack] [BEAM-11494][BEAM-11821] FileIO stops overwriting files on retries (AWS

[Pablo Estrada] [py] Supporting ignore_unknown_values for WriteToBigQuery

[noreply] Merge pull request #16072 from [BEAM-13135][Playground] Add function to

[noreply] Merge pull request #16084 from [BEAM-13347][Playground] [Bugfix] Backend

[noreply] Merge pull request #15987 from [BEAM-13242] Allow values with smaller

[noreply] [BEAM-11936] Fix errorprone UnusedVariable in Extensions (#16033)

[noreply] Merge pull request #15936 from [BEAM-13351] FhirIO GetPatientEverything

[avilovpavel6] Add fmt preparator for go sdk

[noreply] Merge pull request #16073 from [BEAM-13267][Playground] Implement

[ningkang0957] Updated screen diff test for Interactive Beam

[noreply] [BEAM-13282][Playground] Create go dockerfile #16049

[noreply] Merge pull request #16085 from [Beam-13336][Playground] Refactor

[noreply] [BEAM-11936] Fix errorprone UnusedVariable in Extensions-sql (#16034)

[noreply] [BEAM-13193] Support

[noreply] Merge pull request #16090: [BEAM-13352] Add transient qualifier to


------------------------------------------
[...truncated 48.65 KB...]
348f9624d5ed: Preparing
f0c0b4242b1f: Preparing
d88332d7efe5: Preparing
b0ccf29f0a00: Preparing
53ed7ada91af: Preparing
abb00800e18b: Preparing
311cde9b71dd: Preparing
d479baaba261: Preparing
a9e4c9343539: Preparing
47ee2d19f81a: Preparing
ab9d251e27cb: Preparing
8a5844586fdb: Preparing
a4aba4e59b40: Preparing
5499f2905579: Preparing
a36ba9e322f7: Preparing
ab9d251e27cb: Waiting
d479baaba261: Waiting
d88332d7efe5: Waiting
8a5844586fdb: Waiting
47ee2d19f81a: Waiting
b0ccf29f0a00: Waiting
a4aba4e59b40: Waiting
abb00800e18b: Waiting
f0c0b4242b1f: Waiting
53ed7ada91af: Waiting
a9e4c9343539: Waiting
311cde9b71dd: Waiting
5499f2905579: Waiting
a36ba9e322f7: Waiting
9d7b54cf0721: Pushed
348f9624d5ed: Pushed
110d54eba4c1: Pushed
f0c0b4242b1f: Pushed
6164f9663732: Pushed
2a7497ef9079: Pushed
b0ccf29f0a00: Pushed
53ed7ada91af: Pushed
a9e4c9343539: Layer already exists
311cde9b71dd: Pushed
47ee2d19f81a: Layer already exists
ab9d251e27cb: Layer already exists
8a5844586fdb: Layer already exists
a4aba4e59b40: Layer already exists
5499f2905579: Layer already exists
a36ba9e322f7: Layer already exists
d88332d7efe5: Pushed
d479baaba261: Pushed
abb00800e18b: Pushed
20211201124336: digest: sha256:7842836cfde2dc38a12000047f23497aa28d5ec4c47954f0d2ab50d0c5e5ceb5 size: 4311

> Task :sdks:java:testing:load-tests:run
Dec 01, 2021 12:45:51 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Dec 01, 2021 12:45:52 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 197 files. Enable logging at DEBUG level to see which files will be staged.
Dec 01, 2021 12:45:52 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Dec 01, 2021 12:45:52 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Dec 01, 2021 12:45:54 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 197 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Dec 01, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 197 files cached, 0 files newly uploaded in 0 seconds
Dec 01, 2021 12:45:55 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Dec 01, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <111769 bytes, hash a9885a91185c78b8becce7d562022023aedcee54516191af986e1c17ca34cef0> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-qYhakRhceLi-zOfVYgIgI67c7lRRYZGvmG4cF8o0zvA.pb
Dec 01, 2021 12:45:57 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Dec 01, 2021 12:45:57 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@44032fde, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b676112, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5578be42, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4e49ce2b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@136965e3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@53c6f96d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@435cc7f9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4364712f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1b7a52dd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7f93dd4e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5ad5be4a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ad85136, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@737d100a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12e5da86, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6535117e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1d1cbd0f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6fa13e6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3af7d855, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77049094, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@f88bfbe]
Dec 01, 2021 12:45:57 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Dec 01, 2021 12:45:57 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Dec 01, 2021 12:45:57 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Dec 01, 2021 12:45:57 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Dec 01, 2021 12:45:57 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@f1868c9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4ba6ec50, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@642413d4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@fb2e3fd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43a09ce2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3f183caa, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b66322e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@63538bb4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@24534cb0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5a50d9fc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@106d77da, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6f9c5048, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5114b7c7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@dd71b20, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@767f6ee7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b6c6e70, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@f324455, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3a894088, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@370c1968, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2f0bfe17]
Dec 01, 2021 12:45:57 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Dec 01, 2021 12:45:57 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Dec 01, 2021 12:45:57 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Dec 01, 2021 12:45:57 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Dec 01, 2021 12:45:57 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Dec 01, 2021 12:45:57 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Dec 01, 2021 12:45:57 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Dec 01, 2021 12:45:57 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Dec 01, 2021 12:45:57 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Dec 01, 2021 12:45:57 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Dec 01, 2021 12:45:57 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Dec 01, 2021 12:45:57 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.36.0-SNAPSHOT
Dec 01, 2021 12:45:58 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-12-01_04_45_57-7573785157800842187?project=apache-beam-testing
Dec 01, 2021 12:45:58 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-12-01_04_45_57-7573785157800842187
Dec 01, 2021 12:45:58 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-12-01_04_45_57-7573785157800842187
Dec 01, 2021 12:46:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-12-01T12:46:03.096Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-12-11lj. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Dec 01, 2021 12:46:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-01T12:46:08.050Z: Worker configuration: e2-standard-2 in us-central1-a.
Dec 01, 2021 12:46:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-01T12:46:08.830Z: Expanding SplittableParDo operations into optimizable parts.
Dec 01, 2021 12:46:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-01T12:46:08.861Z: Expanding CollectionToSingleton operations into optimizable parts.
Dec 01, 2021 12:46:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-01T12:46:08.938Z: Expanding CoGroupByKey operations into optimizable parts.
Dec 01, 2021 12:46:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-01T12:46:09.002Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Dec 01, 2021 12:46:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-01T12:46:09.037Z: Expanding GroupByKey operations into streaming Read/Write steps
Dec 01, 2021 12:46:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-01T12:46:09.090Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Dec 01, 2021 12:46:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-01T12:46:09.197Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Dec 01, 2021 12:46:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-01T12:46:09.220Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Dec 01, 2021 12:46:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-01T12:46:09.251Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Dec 01, 2021 12:46:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-01T12:46:09.284Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Dec 01, 2021 12:46:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-01T12:46:09.317Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Dec 01, 2021 12:46:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-01T12:46:09.351Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Dec 01, 2021 12:46:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-01T12:46:09.375Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Dec 01, 2021 12:46:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-01T12:46:09.400Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Dec 01, 2021 12:46:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-01T12:46:09.421Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Dec 01, 2021 12:46:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-01T12:46:09.464Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Dec 01, 2021 12:46:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-01T12:46:09.489Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Dec 01, 2021 12:46:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-01T12:46:09.516Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Dec 01, 2021 12:46:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-01T12:46:09.548Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Dec 01, 2021 12:46:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-01T12:46:09.573Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Dec 01, 2021 12:46:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-01T12:46:09.593Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Dec 01, 2021 12:46:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-01T12:46:09.626Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Dec 01, 2021 12:46:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-01T12:46:09.651Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Dec 01, 2021 12:46:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-01T12:46:09.678Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Dec 01, 2021 12:46:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-01T12:46:09.711Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Dec 01, 2021 12:46:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-01T12:46:09.750Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Dec 01, 2021 12:46:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-01T12:46:09.777Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Dec 01, 2021 12:46:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-01T12:46:09.806Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Dec 01, 2021 12:46:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-01T12:46:09.832Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Dec 01, 2021 12:46:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-01T12:46:10.161Z: Starting 5 ****s in us-central1-a...
Dec 01, 2021 12:46:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-01T12:46:42.377Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Dec 01, 2021 12:46:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-01T12:46:54.797Z: Autoscaling: Raised the number of ****s to 3 so that the pipeline can catch up with its backlog and keep up with its input rate.
Dec 01, 2021 12:46:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-01T12:46:54.826Z: Resized **** pool to 3, though goal was 5.  This could be a quota issue.
Dec 01, 2021 12:47:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-01T12:47:05.071Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Dec 01, 2021 12:47:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-01T12:47:58.150Z: Workers have started successfully.
Dec 01, 2021 12:47:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-01T12:47:58.178Z: Workers have started successfully.
Dec 01, 2021 4:00:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-01T16:00:41.723Z: Cancel request is committed for workflow job: 2021-12-01_04_45_57-7573785157800842187.
Dec 01, 2021 4:00:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-01T16:00:41.828Z: Cleaning up.
Dec 01, 2021 4:00:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-01T16:00:41.894Z: Stopping **** pool...
Dec 01, 2021 4:00:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-01T16:00:41.938Z: Stopping **** pool...
Dec 01, 2021 4:03:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-01T16:03:05.110Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Dec 01, 2021 4:03:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-12-01T16:03:05.143Z: Worker pool stopped.
Dec 01, 2021 4:03:10 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-12-01_04_45_57-7573785157800842187 finished with status CANCELLED.
Load test results for test (ID): 9214887a-ec4e-4e4c-ac1e-69a0c7f71c17 and timestamp: 2021-12-01T12:45:52.514000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11539.196
dataflow_v2_java11_total_bytes_count             1.48836149E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211201124336
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:7842836cfde2dc38a12000047f23497aa28d5ec4c47954f0d2ab50d0c5e5ceb5
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211201124336]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:7842836cfde2dc38a12000047f23497aa28d5ec4c47954f0d2ab50d0c5e5ceb5]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211201124336] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:7842836cfde2dc38a12000047f23497aa28d5ec4c47954f0d2ab50d0c5e5ceb5])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:7842836cfde2dc38a12000047f23497aa28d5ec4c47954f0d2ab50d0c5e5ceb5
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:7842836cfde2dc38a12000047f23497aa28d5ec4c47954f0d2ab50d0c5e5ceb5
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:7842836cfde2dc38a12000047f23497aa28d5ec4c47954f0d2ab50d0c5e5ceb5].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 57s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/mlunobhdvfsf6

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #166

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/166/display/redirect?page=changes>

Changes:

[kileysok] Increase timeout for Dataflow Streaming VR

[Kyle Weaver] [BEAM-13220] Update release instructions.

[noreply] Merge pull request #16050 from [BEAM-13307][Playground] Support Python

[noreply] Merge pull request #16071 from [BEAM-13306][Playground] Add using of

[noreply] [BEAM-11943] Fix wrong Klio logo being displayed (#16078)

[noreply] Merge pull request #15954 from [BEAM-960][BEAM-1675] Improvements to

[noreply] [BEAM-13284] Respect expiration time for all Redis write methods.

[noreply] [BEAM-12697] Add SBE module and initial classes (#15733)

[noreply] [Playground][Beam-13316][Bugfix] Fix Playground Frontend Precommit

[Kyle Weaver] [BEAM-13337] Periodically delete stale spanner databases.

[noreply] [BEAM-13115][Playground] Security – Mock Storage (#16047)

[noreply] [BEAM-13335] Use shorter, numerical indices for dataframe reads.

[noreply] Be more conservative about rebatching. (#16058)

[noreply] [BEAM-12587] Allow None in Python's Any logical type. (#16055)

[melissapa] [BEAM-11758] Update basics page: Trigger, State and timers


------------------------------------------
[...truncated 48.49 KB...]
22cc9d8dc485: Preparing
fd30fb18479b: Preparing
119eb50fa5ed: Preparing
357535433b5f: Preparing
c3b07ef09fc4: Preparing
de1267421cb2: Preparing
49a87df89925: Preparing
b949590960bd: Preparing
452981d78c45: Preparing
01d2ecf6213f: Preparing
d0f0e5accd86: Preparing
37b27ba05de4: Preparing
a9e4c9343539: Preparing
47ee2d19f81a: Preparing
ab9d251e27cb: Preparing
8a5844586fdb: Preparing
a4aba4e59b40: Preparing
5499f2905579: Preparing
a36ba9e322f7: Preparing
8a5844586fdb: Waiting
49a87df89925: Waiting
d0f0e5accd86: Waiting
a4aba4e59b40: Waiting
b949590960bd: Waiting
5499f2905579: Waiting
37b27ba05de4: Waiting
452981d78c45: Waiting
a36ba9e322f7: Waiting
a9e4c9343539: Waiting
47ee2d19f81a: Waiting
ab9d251e27cb: Waiting
01d2ecf6213f: Waiting
de1267421cb2: Waiting
119eb50fa5ed: Pushed
fd30fb18479b: Pushed
c3b07ef09fc4: Pushed
de1267421cb2: Pushed
22cc9d8dc485: Pushed
b949590960bd: Pushed
357535433b5f: Pushed
452981d78c45: Pushed
49a87df89925: Pushed
a9e4c9343539: Layer already exists
47ee2d19f81a: Layer already exists
ab9d251e27cb: Layer already exists
8a5844586fdb: Layer already exists
a4aba4e59b40: Layer already exists
5499f2905579: Layer already exists
a36ba9e322f7: Layer already exists
37b27ba05de4: Pushed
d0f0e5accd86: Pushed
01d2ecf6213f: Pushed
20211130124331: digest: sha256:6bff8f3c3ac22df4dbc41278d493852c44ac2c05039e9161a851b0ed33504cb7 size: 4311

> Task :sdks:java:testing:load-tests:run
Nov 30, 2021 12:45:24 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Nov 30, 2021 12:45:25 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 197 files. Enable logging at DEBUG level to see which files will be staged.
Nov 30, 2021 12:45:25 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Nov 30, 2021 12:45:25 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Nov 30, 2021 12:45:27 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 197 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Nov 30, 2021 12:45:28 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 197 files cached, 0 files newly uploaded in 0 seconds
Nov 30, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Nov 30, 2021 12:45:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <111716 bytes, hash a425a33e3e0a65638cfc02c2a061d7e62c3e97e07fb53fb26a02d975c2fc60bc> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-pCWjPj4KZWOM_ALCoGHX5iw-l-B_tT-yagLZdcL8YLw.pb
Nov 30, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Nov 30, 2021 12:45:30 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@177515d1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@52ff99cd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4c2af006, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@44032fde, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b676112, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5578be42, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4e49ce2b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@136965e3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@53c6f96d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@435cc7f9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4364712f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1b7a52dd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7f93dd4e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5ad5be4a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ad85136, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@737d100a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12e5da86, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6535117e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1d1cbd0f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6fa13e6]
Nov 30, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Nov 30, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Nov 30, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Nov 30, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Nov 30, 2021 12:45:30 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3f183caa, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b66322e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@63538bb4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@24534cb0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5a50d9fc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@106d77da, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6f9c5048, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5114b7c7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@dd71b20, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@767f6ee7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b6c6e70, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@f324455, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3a894088, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@370c1968, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2f0bfe17, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3206174f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@42fb8c87, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@15eb0ae9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@65e0b505, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@67de7a99]
Nov 30, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Nov 30, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Nov 30, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Nov 30, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Nov 30, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Nov 30, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Nov 30, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Nov 30, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Nov 30, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Nov 30, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Nov 30, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Nov 30, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.36.0-SNAPSHOT
Nov 30, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-11-30_04_45_30-14949378249761327338?project=apache-beam-testing
Nov 30, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-11-30_04_45_30-14949378249761327338
Nov 30, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-11-30_04_45_30-14949378249761327338
Nov 30, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-11-30T12:45:37.313Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-11-snhw. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Nov 30, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-30T12:45:41.916Z: Worker configuration: e2-standard-2 in us-central1-a.
Nov 30, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-30T12:45:42.651Z: Expanding SplittableParDo operations into optimizable parts.
Nov 30, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-30T12:45:42.679Z: Expanding CollectionToSingleton operations into optimizable parts.
Nov 30, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-30T12:45:42.740Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 30, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-30T12:45:42.818Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Nov 30, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-30T12:45:42.844Z: Expanding GroupByKey operations into streaming Read/Write steps
Nov 30, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-30T12:45:42.912Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 30, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-30T12:45:43.017Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 30, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-30T12:45:43.070Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Nov 30, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-30T12:45:43.103Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Nov 30, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-30T12:45:43.136Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Nov 30, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-30T12:45:43.158Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Nov 30, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-30T12:45:43.188Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Nov 30, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-30T12:45:43.216Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Nov 30, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-30T12:45:43.253Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Nov 30, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-30T12:45:43.292Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Nov 30, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-30T12:45:43.317Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Nov 30, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-30T12:45:43.339Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Nov 30, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-30T12:45:43.370Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Nov 30, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-30T12:45:43.398Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Nov 30, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-30T12:45:43.427Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Nov 30, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-30T12:45:43.456Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Nov 30, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-30T12:45:43.487Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Nov 30, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-30T12:45:43.514Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Nov 30, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-30T12:45:43.540Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Nov 30, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-30T12:45:43.599Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Nov 30, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-30T12:45:43.631Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Nov 30, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-30T12:45:43.661Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Nov 30, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-30T12:45:43.696Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Nov 30, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-30T12:45:43.726Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Nov 30, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-30T12:45:44.110Z: Starting 5 ****s in us-central1-a...
Nov 30, 2021 12:46:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-30T12:46:06.068Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Nov 30, 2021 12:46:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-30T12:46:35.750Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Nov 30, 2021 12:47:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-30T12:47:30.008Z: Workers have started successfully.
Nov 30, 2021 12:47:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-30T12:47:30.041Z: Workers have started successfully.
Nov 30, 2021 4:00:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-30T16:00:33.918Z: Cancel request is committed for workflow job: 2021-11-30_04_45_30-14949378249761327338.
Nov 30, 2021 4:00:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-30T16:00:33.983Z: Cleaning up.
Nov 30, 2021 4:00:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-30T16:00:34.053Z: Stopping **** pool...
Nov 30, 2021 4:00:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-30T16:00:34.105Z: Stopping **** pool...
Nov 30, 2021 4:02:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-30T16:02:48.202Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Nov 30, 2021 4:02:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-30T16:02:48.242Z: Worker pool stopped.
Nov 30, 2021 4:02:53 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-11-30_04_45_30-14949378249761327338 finished with status CANCELLED.
Load test results for test (ID): b5cfb8fa-28a9-4705-89d1-b4863702b70c and timestamp: 2021-11-30T12:45:25.569000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11548.839
dataflow_v2_java11_total_bytes_count             1.80352863E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211130124331
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:6bff8f3c3ac22df4dbc41278d493852c44ac2c05039e9161a851b0ed33504cb7
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211130124331]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:6bff8f3c3ac22df4dbc41278d493852c44ac2c05039e9161a851b0ed33504cb7]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211130124331] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:6bff8f3c3ac22df4dbc41278d493852c44ac2c05039e9161a851b0ed33504cb7])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:6bff8f3c3ac22df4dbc41278d493852c44ac2c05039e9161a851b0ed33504cb7
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:6bff8f3c3ac22df4dbc41278d493852c44ac2c05039e9161a851b0ed33504cb7
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:6bff8f3c3ac22df4dbc41278d493852c44ac2c05039e9161a851b0ed33504cb7].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 39s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/ajlrw3jjo5cge

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #165

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/165/display/redirect>

Changes:


------------------------------------------
[...truncated 48.47 KB...]
The push refers to repository [us.gcr.io/apache-beam-testing/java-postcommit-it/java]
042af599cd41: Preparing
8f5380278bf5: Preparing
09ee2abfddec: Preparing
bfcba587c762: Preparing
db24a6eb056c: Preparing
0b018574049b: Preparing
c64bdd999388: Preparing
4357dbc49323: Preparing
ff3ded16ccfd: Preparing
42dc884742a5: Preparing
3318df26b12a: Preparing
096944600e60: Preparing
a9e4c9343539: Preparing
47ee2d19f81a: Preparing
ab9d251e27cb: Preparing
8a5844586fdb: Preparing
a4aba4e59b40: Preparing
5499f2905579: Preparing
a36ba9e322f7: Preparing
c64bdd999388: Waiting
ff3ded16ccfd: Waiting
8a5844586fdb: Waiting
5499f2905579: Waiting
0b018574049b: Waiting
a9e4c9343539: Waiting
47ee2d19f81a: Waiting
4357dbc49323: Waiting
ab9d251e27cb: Waiting
3318df26b12a: Waiting
096944600e60: Waiting
42dc884742a5: Waiting
a4aba4e59b40: Waiting
09ee2abfddec: Pushed
db24a6eb056c: Pushed
8f5380278bf5: Pushed
042af599cd41: Pushed
0b018574049b: Pushed
bfcba587c762: Pushed
4357dbc49323: Pushed
3318df26b12a: Pushed
ff3ded16ccfd: Pushed
47ee2d19f81a: Layer already exists
c64bdd999388: Pushed
ab9d251e27cb: Layer already exists
a9e4c9343539: Layer already exists
8a5844586fdb: Layer already exists
a4aba4e59b40: Layer already exists
5499f2905579: Layer already exists
a36ba9e322f7: Layer already exists
096944600e60: Pushed
42dc884742a5: Pushed
20211129124331: digest: sha256:5afc79ff4826b7c35eb584305732884be6469ea09f5d18ebc32d647783d8cb11 size: 4311

> Task :sdks:java:testing:load-tests:run
Nov 29, 2021 12:45:33 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Nov 29, 2021 12:45:33 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 197 files. Enable logging at DEBUG level to see which files will be staged.
Nov 29, 2021 12:45:34 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Nov 29, 2021 12:45:34 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Nov 29, 2021 12:45:36 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 197 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Nov 29, 2021 12:45:37 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 197 files cached, 0 files newly uploaded in 0 seconds
Nov 29, 2021 12:45:37 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Nov 29, 2021 12:45:37 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <111716 bytes, hash a6ebe71c86a4d0b91fba17c93acd7e4c1394caac55c55086f20eddc03eac1852> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-puvnHIak0LkfuhfJOs1-TBOUyqxVxVCG8g7dwD6sGFI.pb
Nov 29, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Nov 29, 2021 12:45:38 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4c2af006, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@44032fde, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b676112, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5578be42, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4e49ce2b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@136965e3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@53c6f96d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@435cc7f9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4364712f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1b7a52dd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7f93dd4e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5ad5be4a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ad85136, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@737d100a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12e5da86, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6535117e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1d1cbd0f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6fa13e6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3af7d855, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77049094]
Nov 29, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Nov 29, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Nov 29, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Nov 29, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Nov 29, 2021 12:45:38 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@48f4713c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@f1868c9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4ba6ec50, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@642413d4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@fb2e3fd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43a09ce2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3f183caa, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b66322e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@63538bb4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@24534cb0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5a50d9fc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@106d77da, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6f9c5048, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5114b7c7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@dd71b20, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@767f6ee7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b6c6e70, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@f324455, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3a894088, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@370c1968]
Nov 29, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Nov 29, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Nov 29, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Nov 29, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Nov 29, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Nov 29, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Nov 29, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Nov 29, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Nov 29, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Nov 29, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Nov 29, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Nov 29, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.36.0-SNAPSHOT
Nov 29, 2021 12:45:39 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-11-29_04_45_38-16428249400959612030?project=apache-beam-testing
Nov 29, 2021 12:45:39 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-11-29_04_45_38-16428249400959612030
Nov 29, 2021 12:45:39 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-11-29_04_45_38-16428249400959612030
Nov 29, 2021 12:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-11-29T12:45:46.529Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-11-vlu2. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Nov 29, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-29T12:45:52.248Z: Worker configuration: e2-standard-2 in us-central1-a.
Nov 29, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-29T12:45:53.144Z: Expanding SplittableParDo operations into optimizable parts.
Nov 29, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-29T12:45:53.179Z: Expanding CollectionToSingleton operations into optimizable parts.
Nov 29, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-29T12:45:53.250Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 29, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-29T12:45:53.400Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Nov 29, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-29T12:45:53.447Z: Expanding GroupByKey operations into streaming Read/Write steps
Nov 29, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-29T12:45:53.516Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 29, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-29T12:45:53.606Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 29, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-29T12:45:53.661Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Nov 29, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-29T12:45:53.696Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Nov 29, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-29T12:45:53.725Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Nov 29, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-29T12:45:53.754Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Nov 29, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-29T12:45:53.781Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Nov 29, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-29T12:45:53.808Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Nov 29, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-29T12:45:53.836Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Nov 29, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-29T12:45:53.865Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Nov 29, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-29T12:45:53.891Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Nov 29, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-29T12:45:53.925Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Nov 29, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-29T12:45:53.959Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Nov 29, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-29T12:45:53.992Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Nov 29, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-29T12:45:54.048Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Nov 29, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-29T12:45:54.076Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Nov 29, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-29T12:45:54.110Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Nov 29, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-29T12:45:54.145Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Nov 29, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-29T12:45:54.177Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Nov 29, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-29T12:45:54.209Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Nov 29, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-29T12:45:54.243Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Nov 29, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-29T12:45:54.275Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Nov 29, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-29T12:45:54.302Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Nov 29, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-29T12:45:54.330Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Nov 29, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-29T12:45:54.794Z: Starting 5 ****s in us-central1-a...
Nov 29, 2021 12:46:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-29T12:46:18.736Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Nov 29, 2021 12:46:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-29T12:46:39.962Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Nov 29, 2021 12:47:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-29T12:47:34.972Z: Workers have started successfully.
Nov 29, 2021 12:47:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-29T12:47:35.006Z: Workers have started successfully.
Nov 29, 2021 4:00:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-29T16:00:32.287Z: Cancel request is committed for workflow job: 2021-11-29_04_45_38-16428249400959612030.
Nov 29, 2021 4:00:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-29T16:00:32.430Z: Cleaning up.
Nov 29, 2021 4:00:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-29T16:00:32.526Z: Stopping **** pool...
Nov 29, 2021 4:00:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-29T16:00:32.572Z: Stopping **** pool...
Nov 29, 2021 4:02:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-29T16:02:49.254Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Nov 29, 2021 4:02:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-29T16:02:49.287Z: Worker pool stopped.
Nov 29, 2021 4:02:56 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-11-29_04_45_38-16428249400959612030 finished with status CANCELLED.
Load test results for test (ID): d92f23c9-7209-4584-9499-fd264ef7418a and timestamp: 2021-11-29T12:45:34.180000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11544.278
dataflow_v2_java11_total_bytes_count             2.52293675E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211129124331
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:5afc79ff4826b7c35eb584305732884be6469ea09f5d18ebc32d647783d8cb11
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211129124331]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:5afc79ff4826b7c35eb584305732884be6469ea09f5d18ebc32d647783d8cb11]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211129124331] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:5afc79ff4826b7c35eb584305732884be6469ea09f5d18ebc32d647783d8cb11])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:5afc79ff4826b7c35eb584305732884be6469ea09f5d18ebc32d647783d8cb11
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:5afc79ff4826b7c35eb584305732884be6469ea09f5d18ebc32d647783d8cb11
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:5afc79ff4826b7c35eb584305732884be6469ea09f5d18ebc32d647783d8cb11].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 44s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/u4qqon5swkz4u

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #164

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/164/display/redirect?page=changes>

Changes:

[vitaly.ivanov] [BEAM-13269] Data exceeds database column capacity error while inserting

[vitaly.ivanov] [BEAM-13269] Data exceeds database column capacity error while inserting

[noreply] Merge pull request #16074 from [BEAM-13323][Playground] [Bugfix] Fix CI


------------------------------------------
[...truncated 48.49 KB...]
1780a986a391: Preparing
260779d49dfb: Preparing
d91179b94dcc: Preparing
255e6d95ccc8: Preparing
7d006af8d29d: Preparing
25cd40d8dbc1: Preparing
73d0dba98448: Preparing
603a9b8b9b10: Preparing
66cd4191185b: Preparing
05207a53e914: Preparing
6cadd39f5402: Preparing
4e3f4c4e2855: Preparing
a9e4c9343539: Preparing
47ee2d19f81a: Preparing
ab9d251e27cb: Preparing
8a5844586fdb: Preparing
a4aba4e59b40: Preparing
5499f2905579: Preparing
a36ba9e322f7: Preparing
603a9b8b9b10: Waiting
25cd40d8dbc1: Waiting
73d0dba98448: Waiting
66cd4191185b: Waiting
ab9d251e27cb: Waiting
5499f2905579: Waiting
a36ba9e322f7: Waiting
05207a53e914: Waiting
8a5844586fdb: Waiting
a4aba4e59b40: Waiting
a9e4c9343539: Waiting
4e3f4c4e2855: Waiting
6cadd39f5402: Waiting
47ee2d19f81a: Waiting
7d006af8d29d: Pushed
d91179b94dcc: Pushed
1780a986a391: Pushed
25cd40d8dbc1: Pushed
255e6d95ccc8: Pushed
66cd4191185b: Pushed
73d0dba98448: Pushed
603a9b8b9b10: Pushed
a9e4c9343539: Layer already exists
47ee2d19f81a: Layer already exists
ab9d251e27cb: Layer already exists
8a5844586fdb: Layer already exists
a4aba4e59b40: Layer already exists
5499f2905579: Layer already exists
a36ba9e322f7: Layer already exists
6cadd39f5402: Pushed
4e3f4c4e2855: Pushed
05207a53e914: Pushed
260779d49dfb: Pushed
20211128124334: digest: sha256:61273d3b426f6d6c7f2f0eaf0310d35b1a45d480d3805d3e0e155670a45e8969 size: 4311

> Task :sdks:java:testing:load-tests:run
Nov 28, 2021 12:45:50 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Nov 28, 2021 12:45:51 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 197 files. Enable logging at DEBUG level to see which files will be staged.
Nov 28, 2021 12:45:51 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Nov 28, 2021 12:45:51 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Nov 28, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 197 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Nov 28, 2021 12:45:54 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 197 files cached, 0 files newly uploaded in 0 seconds
Nov 28, 2021 12:45:54 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Nov 28, 2021 12:45:54 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <111716 bytes, hash 5c6cb0c19fbfbb769aa40edcacb5d42a701f556d8a2b61a76d71c56beab9479f> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-XGywwZ-_u3aapA7crLXUKnAfVW2KK2GnbXHFa-q5R58.pb
Nov 28, 2021 12:45:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Nov 28, 2021 12:45:56 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4c2af006, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@44032fde, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b676112, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5578be42, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4e49ce2b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@136965e3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@53c6f96d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@435cc7f9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4364712f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1b7a52dd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7f93dd4e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5ad5be4a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ad85136, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@737d100a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12e5da86, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6535117e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1d1cbd0f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6fa13e6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3af7d855, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77049094]
Nov 28, 2021 12:45:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Nov 28, 2021 12:45:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Nov 28, 2021 12:45:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Nov 28, 2021 12:45:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Nov 28, 2021 12:45:56 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@48f4713c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@f1868c9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4ba6ec50, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@642413d4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@fb2e3fd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43a09ce2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3f183caa, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b66322e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@63538bb4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@24534cb0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5a50d9fc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@106d77da, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6f9c5048, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5114b7c7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@dd71b20, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@767f6ee7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b6c6e70, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@f324455, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3a894088, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@370c1968]
Nov 28, 2021 12:45:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Nov 28, 2021 12:45:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Nov 28, 2021 12:45:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Nov 28, 2021 12:45:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Nov 28, 2021 12:45:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Nov 28, 2021 12:45:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Nov 28, 2021 12:45:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Nov 28, 2021 12:45:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Nov 28, 2021 12:45:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Nov 28, 2021 12:45:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Nov 28, 2021 12:45:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Nov 28, 2021 12:45:56 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.36.0-SNAPSHOT
Nov 28, 2021 12:45:57 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-11-28_04_45_56-13413062277742559395?project=apache-beam-testing
Nov 28, 2021 12:45:57 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-11-28_04_45_56-13413062277742559395
Nov 28, 2021 12:45:57 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-11-28_04_45_56-13413062277742559395
Nov 28, 2021 12:46:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-11-28T12:46:03.731Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-11-vbej. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Nov 28, 2021 12:46:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-28T12:46:08.503Z: Worker configuration: e2-standard-2 in us-central1-a.
Nov 28, 2021 12:46:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-28T12:46:09.007Z: Expanding SplittableParDo operations into optimizable parts.
Nov 28, 2021 12:46:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-28T12:46:09.027Z: Expanding CollectionToSingleton operations into optimizable parts.
Nov 28, 2021 12:46:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-28T12:46:09.080Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 28, 2021 12:46:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-28T12:46:09.139Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Nov 28, 2021 12:46:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-28T12:46:09.195Z: Expanding GroupByKey operations into streaming Read/Write steps
Nov 28, 2021 12:46:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-28T12:46:09.260Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 28, 2021 12:46:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-28T12:46:09.357Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 28, 2021 12:46:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-28T12:46:09.390Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Nov 28, 2021 12:46:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-28T12:46:09.435Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Nov 28, 2021 12:46:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-28T12:46:09.467Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Nov 28, 2021 12:46:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-28T12:46:09.502Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Nov 28, 2021 12:46:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-28T12:46:09.533Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Nov 28, 2021 12:46:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-28T12:46:09.566Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Nov 28, 2021 12:46:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-28T12:46:09.598Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Nov 28, 2021 12:46:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-28T12:46:09.620Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Nov 28, 2021 12:46:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-28T12:46:09.653Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Nov 28, 2021 12:46:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-28T12:46:09.686Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Nov 28, 2021 12:46:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-28T12:46:09.718Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Nov 28, 2021 12:46:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-28T12:46:09.760Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Nov 28, 2021 12:46:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-28T12:46:09.806Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Nov 28, 2021 12:46:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-28T12:46:09.839Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Nov 28, 2021 12:46:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-28T12:46:09.864Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Nov 28, 2021 12:46:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-28T12:46:09.884Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Nov 28, 2021 12:46:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-28T12:46:09.915Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Nov 28, 2021 12:46:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-28T12:46:09.949Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Nov 28, 2021 12:46:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-28T12:46:09.982Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Nov 28, 2021 12:46:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-28T12:46:10.013Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Nov 28, 2021 12:46:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-28T12:46:10.057Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Nov 28, 2021 12:46:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-28T12:46:10.089Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Nov 28, 2021 12:46:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-28T12:46:10.438Z: Starting 5 ****s in us-central1-a...
Nov 28, 2021 12:46:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-28T12:46:38.523Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Nov 28, 2021 12:46:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-28T12:46:50.430Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Nov 28, 2021 12:47:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-28T12:47:44.482Z: Workers have started successfully.
Nov 28, 2021 12:47:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-28T12:47:44.516Z: Workers have started successfully.
Nov 28, 2021 4:00:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-28T16:00:31.528Z: Cancel request is committed for workflow job: 2021-11-28_04_45_56-13413062277742559395.
Nov 28, 2021 4:00:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-28T16:00:31.945Z: Cleaning up.
Nov 28, 2021 4:00:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-28T16:00:32.037Z: Stopping **** pool...
Nov 28, 2021 4:00:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-28T16:00:32.220Z: Stopping **** pool...
Nov 28, 2021 4:02:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-28T16:02:47.442Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Nov 28, 2021 4:02:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-28T16:02:47.479Z: Worker pool stopped.
Nov 28, 2021 4:02:54 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-11-28_04_45_56-13413062277742559395 finished with status CANCELLED.
Load test results for test (ID): 3a652a54-0a2b-46da-8ad7-9d83a78e2d24 and timestamp: 2021-11-28T12:45:51.496000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11501.868
dataflow_v2_java11_total_bytes_count             2.36728306E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211128124334
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:61273d3b426f6d6c7f2f0eaf0310d35b1a45d480d3805d3e0e155670a45e8969
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211128124334]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:61273d3b426f6d6c7f2f0eaf0310d35b1a45d480d3805d3e0e155670a45e8969]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211128124334] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:61273d3b426f6d6c7f2f0eaf0310d35b1a45d480d3805d3e0e155670a45e8969])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:61273d3b426f6d6c7f2f0eaf0310d35b1a45d480d3805d3e0e155670a45e8969
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:61273d3b426f6d6c7f2f0eaf0310d35b1a45d480d3805d3e0e155670a45e8969
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:61273d3b426f6d6c7f2f0eaf0310d35b1a45d480d3805d3e0e155670a45e8969].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 37s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/wt2txti7cn7xi

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #163

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/163/display/redirect>

Changes:


------------------------------------------
[...truncated 48.08 KB...]
1c6a92bd04bd: Preparing
2e84b2cd629a: Preparing
3233a81acadc: Preparing
5461a9724d64: Preparing
bdcb92e8950b: Preparing
e12deef2713a: Preparing
a9e4c9343539: Preparing
47ee2d19f81a: Preparing
ab9d251e27cb: Preparing
2e84b2cd629a: Waiting
5461a9724d64: Waiting
8a5844586fdb: Preparing
3233a81acadc: Waiting
a4aba4e59b40: Preparing
5499f2905579: Preparing
bdcb92e8950b: Waiting
a9e4c9343539: Waiting
47ee2d19f81a: Waiting
a36ba9e322f7: Preparing
8a5844586fdb: Waiting
a4aba4e59b40: Waiting
5499f2905579: Waiting
a5d8ec5405f9: Waiting
a36ba9e322f7: Waiting
ab9d251e27cb: Waiting
8da5421b804b: Pushed
909ecb9172e0: Pushed
2be90649daa8: Pushed
a5d8ec5405f9: Pushed
dd91861233dd: Pushed
98901f8af3ee: Pushed
2e84b2cd629a: Pushed
3233a81acadc: Pushed
a9e4c9343539: Layer already exists
47ee2d19f81a: Layer already exists
ab9d251e27cb: Layer already exists
8a5844586fdb: Layer already exists
a4aba4e59b40: Layer already exists
bdcb92e8950b: Pushed
5499f2905579: Layer already exists
a36ba9e322f7: Layer already exists
e12deef2713a: Pushed
1c6a92bd04bd: Pushed
5461a9724d64: Pushed
20211127124332: digest: sha256:c44ba3d6e18b4542f8294f3b50b46035b75f98347237d9d9b8b821045e9ca16d size: 4311

> Task :sdks:java:testing:load-tests:run
Nov 27, 2021 12:45:17 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Nov 27, 2021 12:45:17 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 197 files. Enable logging at DEBUG level to see which files will be staged.
Nov 27, 2021 12:45:18 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Nov 27, 2021 12:45:18 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Nov 27, 2021 12:45:20 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 197 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Nov 27, 2021 12:45:20 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 197 files cached, 0 files newly uploaded in 0 seconds
Nov 27, 2021 12:45:21 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Nov 27, 2021 12:45:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <111716 bytes, hash 43eb46937e01796d53e29ba9dd6cc2a2e764819e4b77a5fe014d88429ba4b374> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-Q-tGk34BeW1T4pup3WzCoudkgZ5Ld6X-AU2IQpuks3Q.pb
Nov 27, 2021 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Nov 27, 2021 12:45:22 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@177515d1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@52ff99cd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4c2af006, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@44032fde, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b676112, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5578be42, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4e49ce2b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@136965e3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@53c6f96d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@435cc7f9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4364712f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1b7a52dd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7f93dd4e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5ad5be4a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ad85136, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@737d100a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12e5da86, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6535117e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1d1cbd0f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6fa13e6]
Nov 27, 2021 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Nov 27, 2021 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Nov 27, 2021 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Nov 27, 2021 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Nov 27, 2021 12:45:22 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3f183caa, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b66322e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@63538bb4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@24534cb0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5a50d9fc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@106d77da, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6f9c5048, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5114b7c7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@dd71b20, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@767f6ee7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b6c6e70, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@f324455, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3a894088, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@370c1968, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2f0bfe17, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3206174f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@42fb8c87, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@15eb0ae9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@65e0b505, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@67de7a99]
Nov 27, 2021 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Nov 27, 2021 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Nov 27, 2021 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Nov 27, 2021 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Nov 27, 2021 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Nov 27, 2021 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Nov 27, 2021 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Nov 27, 2021 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Nov 27, 2021 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Nov 27, 2021 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Nov 27, 2021 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Nov 27, 2021 12:45:22 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.36.0-SNAPSHOT
Nov 27, 2021 12:45:23 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-11-27_04_45_23-6887480424228758737?project=apache-beam-testing
Nov 27, 2021 12:45:23 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-11-27_04_45_23-6887480424228758737
Nov 27, 2021 12:45:23 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-11-27_04_45_23-6887480424228758737
Nov 27, 2021 12:45:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-11-27T12:45:29.329Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-11-tsvn. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Nov 27, 2021 12:45:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-27T12:45:34.097Z: Worker configuration: e2-standard-2 in us-central1-a.
Nov 27, 2021 12:45:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-27T12:45:34.661Z: Expanding SplittableParDo operations into optimizable parts.
Nov 27, 2021 12:45:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-27T12:45:34.702Z: Expanding CollectionToSingleton operations into optimizable parts.
Nov 27, 2021 12:45:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-27T12:45:34.774Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 27, 2021 12:45:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-27T12:45:34.842Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Nov 27, 2021 12:45:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-27T12:45:34.873Z: Expanding GroupByKey operations into streaming Read/Write steps
Nov 27, 2021 12:45:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-27T12:45:34.950Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 27, 2021 12:45:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-27T12:45:35.061Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 27, 2021 12:45:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-27T12:45:35.093Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Nov 27, 2021 12:45:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-27T12:45:35.119Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Nov 27, 2021 12:45:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-27T12:45:35.151Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Nov 27, 2021 12:45:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-27T12:45:35.189Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Nov 27, 2021 12:45:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-27T12:45:35.222Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Nov 27, 2021 12:45:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-27T12:45:35.257Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Nov 27, 2021 12:45:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-27T12:45:35.290Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Nov 27, 2021 12:45:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-27T12:45:35.336Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Nov 27, 2021 12:45:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-27T12:45:35.374Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Nov 27, 2021 12:45:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-27T12:45:35.403Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Nov 27, 2021 12:45:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-27T12:45:35.437Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Nov 27, 2021 12:45:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-27T12:45:35.471Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Nov 27, 2021 12:45:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-27T12:45:35.506Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Nov 27, 2021 12:45:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-27T12:45:35.554Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Nov 27, 2021 12:45:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-27T12:45:35.580Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Nov 27, 2021 12:45:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-27T12:45:35.612Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Nov 27, 2021 12:45:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-27T12:45:35.643Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Nov 27, 2021 12:45:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-27T12:45:35.682Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Nov 27, 2021 12:45:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-27T12:45:35.717Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Nov 27, 2021 12:45:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-27T12:45:35.755Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Nov 27, 2021 12:45:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-27T12:45:35.784Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Nov 27, 2021 12:45:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-27T12:45:35.818Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Nov 27, 2021 12:45:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-27T12:45:36.243Z: Starting 5 ****s in us-central1-a...
Nov 27, 2021 12:46:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-27T12:46:02.233Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Nov 27, 2021 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-27T12:46:19.385Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Nov 27, 2021 12:47:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-27T12:47:17.737Z: Workers have started successfully.
Nov 27, 2021 12:47:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-27T12:47:17.769Z: Workers have started successfully.
Nov 27, 2021 2:12:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-27T14:12:36.310Z: Staged package amazon-kinesis-client-1.14.2-qtBh3OuKuLE7S26ITTJzYsoYfG-mWjqbGmv3VJPhec8.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/amazon-kinesis-client-1.14.2-qtBh3OuKuLE7S26ITTJzYsoYfG-mWjqbGmv3VJPhec8.jar' is inaccessible.
Nov 27, 2021 2:12:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-27T14:12:42.146Z: Staged package jakarta.activation-api-1.2.2-oYepORA671hJp6-EvX4nvi0SDEEK8pFDc3X_4GH08J0.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/jakarta.activation-api-1.2.2-oYepORA671hJp6-EvX4nvi0SDEEK8pFDc3X_4GH08J0.jar' is inaccessible.
Nov 27, 2021 2:12:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-27T14:12:42.188Z: Staged package jakarta.xml.bind-api-2.3.3-wEU59HLppt0MdoXqgtZ3KCJpq457rKLhRQDjgeDGzsU.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/jakarta.xml.bind-api-2.3.3-wEU59HLppt0MdoXqgtZ3KCJpq457rKLhRQDjgeDGzsU.jar' is inaccessible.
Nov 27, 2021 2:12:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-11-27T14:12:43.367Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Nov 27, 2021 4:00:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-27T16:00:34.493Z: Cancel request is committed for workflow job: 2021-11-27_04_45_23-6887480424228758737.
Nov 27, 2021 4:00:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-27T16:00:34.521Z: Cleaning up.
Nov 27, 2021 4:00:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-27T16:00:34.590Z: Stopping **** pool...
Nov 27, 2021 4:00:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-27T16:00:34.656Z: Stopping **** pool...
Nov 27, 2021 4:02:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-27T16:02:55.989Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Nov 27, 2021 4:02:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-27T16:02:56.023Z: Worker pool stopped.
Nov 27, 2021 4:03:01 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-11-27_04_45_23-6887480424228758737 finished with status CANCELLED.
Load test results for test (ID): 55206b0c-aefb-4f05-a172-41d13136a7e2 and timestamp: 2021-11-27T12:45:18.190000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11571.355
dataflow_v2_java11_total_bytes_count             2.07269107E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211127124332
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:c44ba3d6e18b4542f8294f3b50b46035b75f98347237d9d9b8b821045e9ca16d
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211127124332]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:c44ba3d6e18b4542f8294f3b50b46035b75f98347237d9d9b8b821045e9ca16d]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211127124332] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:c44ba3d6e18b4542f8294f3b50b46035b75f98347237d9d9b8b821045e9ca16d])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:c44ba3d6e18b4542f8294f3b50b46035b75f98347237d9d9b8b821045e9ca16d
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:c44ba3d6e18b4542f8294f3b50b46035b75f98347237d9d9b8b821045e9ca16d
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:c44ba3d6e18b4542f8294f3b50b46035b75f98347237d9d9b8b821045e9ca16d].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 45s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/ixk722wtnlutc

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #162

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/162/display/redirect>

Changes:


------------------------------------------
[...truncated 48.35 KB...]

The push refers to repository [us.gcr.io/apache-beam-testing/java-postcommit-it/java]
ab28e4d9ed50: Preparing
e841475cba0f: Preparing
fbf5c5c2b0e8: Preparing
c7567abe3a99: Preparing
5eb68597851d: Preparing
441d45f2bf20: Preparing
4ddfb9f670e8: Preparing
7534a823251f: Preparing
fb28fc033933: Preparing
65e7c91c4fed: Preparing
11c4b41de4a2: Preparing
9a148d5c3fba: Preparing
a9e4c9343539: Preparing
47ee2d19f81a: Preparing
441d45f2bf20: Waiting
ab9d251e27cb: Preparing
8a5844586fdb: Preparing
4ddfb9f670e8: Waiting
a4aba4e59b40: Preparing
5499f2905579: Preparing
7534a823251f: Waiting
fb28fc033933: Waiting
a36ba9e322f7: Preparing
9a148d5c3fba: Waiting
a4aba4e59b40: Waiting
11c4b41de4a2: Waiting
ab9d251e27cb: Waiting
8a5844586fdb: Waiting
5499f2905579: Waiting
a36ba9e322f7: Waiting
65e7c91c4fed: Waiting
5eb68597851d: Pushed
e841475cba0f: Pushed
fbf5c5c2b0e8: Pushed
ab28e4d9ed50: Pushed
441d45f2bf20: Pushed
c7567abe3a99: Pushed
7534a823251f: Pushed
fb28fc033933: Pushed
a9e4c9343539: Layer already exists
47ee2d19f81a: Layer already exists
ab9d251e27cb: Layer already exists
11c4b41de4a2: Pushed
8a5844586fdb: Layer already exists
a4aba4e59b40: Layer already exists
4ddfb9f670e8: Pushed
5499f2905579: Layer already exists
a36ba9e322f7: Layer already exists
9a148d5c3fba: Pushed
65e7c91c4fed: Pushed
20211126124333: digest: sha256:2d7c6b9211345efc3330d846b7254c9cb037a2915090c98db71c8194739c82bc size: 4311

> Task :sdks:java:testing:load-tests:run
Nov 26, 2021 12:45:32 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Nov 26, 2021 12:45:32 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 197 files. Enable logging at DEBUG level to see which files will be staged.
Nov 26, 2021 12:45:33 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Nov 26, 2021 12:45:33 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Nov 26, 2021 12:45:36 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 197 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Nov 26, 2021 12:45:36 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 197 files cached, 0 files newly uploaded in 0 seconds
Nov 26, 2021 12:45:36 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Nov 26, 2021 12:45:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <111716 bytes, hash c956899fc4a0767482982fe4bb216f478b48aefc2814e53d4e941357e1c2fd4c> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-yVaJn8SgdnSCmC_kuyFvR4tIrvwoFOU9TpQTV-HC_Uw.pb
Nov 26, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Nov 26, 2021 12:45:38 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4c2af006, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@44032fde, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b676112, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5578be42, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4e49ce2b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@136965e3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@53c6f96d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@435cc7f9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4364712f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1b7a52dd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7f93dd4e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5ad5be4a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ad85136, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@737d100a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12e5da86, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6535117e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1d1cbd0f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6fa13e6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3af7d855, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77049094]
Nov 26, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Nov 26, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Nov 26, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Nov 26, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Nov 26, 2021 12:45:38 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@48f4713c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@f1868c9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4ba6ec50, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@642413d4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@fb2e3fd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43a09ce2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3f183caa, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b66322e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@63538bb4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@24534cb0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5a50d9fc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@106d77da, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6f9c5048, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5114b7c7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@dd71b20, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@767f6ee7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b6c6e70, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@f324455, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3a894088, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@370c1968]
Nov 26, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Nov 26, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Nov 26, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Nov 26, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Nov 26, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Nov 26, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Nov 26, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Nov 26, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Nov 26, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Nov 26, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Nov 26, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Nov 26, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.36.0-SNAPSHOT
Nov 26, 2021 12:45:39 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-11-26_04_45_38-3752755376803788383?project=apache-beam-testing
Nov 26, 2021 12:45:39 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-11-26_04_45_38-3752755376803788383
Nov 26, 2021 12:45:39 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-11-26_04_45_38-3752755376803788383
Nov 26, 2021 12:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-11-26T12:45:45.719Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-11-r6oa. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Nov 26, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-26T12:45:48.773Z: Worker configuration: e2-standard-2 in us-central1-a.
Nov 26, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-26T12:45:49.431Z: Expanding SplittableParDo operations into optimizable parts.
Nov 26, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-26T12:45:49.463Z: Expanding CollectionToSingleton operations into optimizable parts.
Nov 26, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-26T12:45:49.532Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 26, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-26T12:45:49.594Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Nov 26, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-26T12:45:49.616Z: Expanding GroupByKey operations into streaming Read/Write steps
Nov 26, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-26T12:45:49.677Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 26, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-26T12:45:49.774Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 26, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-26T12:45:49.796Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Nov 26, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-26T12:45:49.823Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Nov 26, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-26T12:45:49.879Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Nov 26, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-26T12:45:49.901Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Nov 26, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-26T12:45:49.931Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Nov 26, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-26T12:45:49.969Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Nov 26, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-26T12:45:49.999Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Nov 26, 2021 12:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-26T12:45:50.041Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Nov 26, 2021 12:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-26T12:45:50.064Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Nov 26, 2021 12:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-26T12:45:50.095Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Nov 26, 2021 12:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-26T12:45:50.137Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Nov 26, 2021 12:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-26T12:45:50.168Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Nov 26, 2021 12:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-26T12:45:50.199Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Nov 26, 2021 12:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-26T12:45:50.224Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Nov 26, 2021 12:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-26T12:45:50.259Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Nov 26, 2021 12:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-26T12:45:50.294Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Nov 26, 2021 12:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-26T12:45:50.320Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Nov 26, 2021 12:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-26T12:45:50.356Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Nov 26, 2021 12:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-26T12:45:50.387Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Nov 26, 2021 12:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-26T12:45:50.416Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Nov 26, 2021 12:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-26T12:45:50.456Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Nov 26, 2021 12:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-26T12:45:50.479Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Nov 26, 2021 12:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-26T12:45:50.840Z: Starting 5 ****s in us-central1-a...
Nov 26, 2021 12:46:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-26T12:46:00.606Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Nov 26, 2021 12:46:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-26T12:46:33.804Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Nov 26, 2021 12:47:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-26T12:47:29.439Z: Workers have started successfully.
Nov 26, 2021 12:47:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-26T12:47:29.472Z: Workers have started successfully.
Nov 26, 2021 4:00:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-26T16:00:34.322Z: Cancel request is committed for workflow job: 2021-11-26_04_45_38-3752755376803788383.
Nov 26, 2021 4:00:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-26T16:00:34.387Z: Cleaning up.
Nov 26, 2021 4:00:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-26T16:00:34.458Z: Stopping **** pool...
Nov 26, 2021 4:00:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-26T16:00:34.519Z: Stopping **** pool...
Nov 26, 2021 4:03:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-26T16:03:01.664Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Nov 26, 2021 4:03:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-26T16:03:01.698Z: Worker pool stopped.
Nov 26, 2021 4:03:08 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-11-26_04_45_38-3752755376803788383 finished with status CANCELLED.
Load test results for test (ID): 7f01b3c9-ec6b-4d6e-94fe-c54db1e830d3 and timestamp: 2021-11-26T12:45:32.959000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                  11548.83
dataflow_v2_java11_total_bytes_count              1.8461165E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211126124333
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:2d7c6b9211345efc3330d846b7254c9cb037a2915090c98db71c8194739c82bc
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211126124333]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:2d7c6b9211345efc3330d846b7254c9cb037a2915090c98db71c8194739c82bc]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211126124333] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:2d7c6b9211345efc3330d846b7254c9cb037a2915090c98db71c8194739c82bc])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:2d7c6b9211345efc3330d846b7254c9cb037a2915090c98db71c8194739c82bc
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:2d7c6b9211345efc3330d846b7254c9cb037a2915090c98db71c8194739c82bc
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:2d7c6b9211345efc3330d846b7254c9cb037a2915090c98db71c8194739c82bc].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 53s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/hdvg77gib4llm

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #161

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/161/display/redirect?page=changes>

Changes:

[Luke Cwik] [BEAM-13015] Simplify

[Luke Cwik] [BEAM-13015] Simplify DataStreamsDecoder by moving hasNext logic to next


------------------------------------------
[...truncated 52.04 KB...]
INFO: Uploading <111716 bytes, hash 4f49a4178dc3c51fd74d9171ced484a1e1bd6838f435c258d043b0bd84ff08b5> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-T0mkF43DxR_XTZFxztSEoeG9aDj0NcJY0EOwvYT_CLU.pb
Nov 25, 2021 12:45:36 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Nov 25, 2021 12:45:36 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b676112, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5578be42, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4e49ce2b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@136965e3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@53c6f96d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@435cc7f9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4364712f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1b7a52dd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7f93dd4e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5ad5be4a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ad85136, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@737d100a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12e5da86, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6535117e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1d1cbd0f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6fa13e6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3af7d855, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77049094, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@f88bfbe, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@59bbe88a]
Nov 25, 2021 12:45:36 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Nov 25, 2021 12:45:36 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Nov 25, 2021 12:45:36 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Nov 25, 2021 12:45:36 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Nov 25, 2021 12:45:36 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4ba6ec50, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@642413d4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@fb2e3fd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43a09ce2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3f183caa, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b66322e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@63538bb4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@24534cb0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5a50d9fc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@106d77da, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6f9c5048, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5114b7c7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@dd71b20, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@767f6ee7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b6c6e70, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@f324455, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3a894088, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@370c1968, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2f0bfe17, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3206174f]
Nov 25, 2021 12:45:36 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Nov 25, 2021 12:45:36 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Nov 25, 2021 12:45:36 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Nov 25, 2021 12:45:36 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Nov 25, 2021 12:45:36 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Nov 25, 2021 12:45:36 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Nov 25, 2021 12:45:36 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Nov 25, 2021 12:45:36 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Nov 25, 2021 12:45:36 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Nov 25, 2021 12:45:36 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Nov 25, 2021 12:45:36 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Nov 25, 2021 12:45:36 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.36.0-SNAPSHOT
Nov 25, 2021 12:45:37 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-11-25_04_45_36-14387302763703776575?project=apache-beam-testing
Nov 25, 2021 12:45:37 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-11-25_04_45_36-14387302763703776575
Nov 25, 2021 12:45:37 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-11-25_04_45_36-14387302763703776575
Nov 25, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-11-25T12:45:44.000Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-11-xij8. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Nov 25, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-25T12:45:48.540Z: Worker configuration: e2-standard-2 in us-central1-a.
Nov 25, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-25T12:45:49.222Z: Expanding SplittableParDo operations into optimizable parts.
Nov 25, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-25T12:45:49.274Z: Expanding CollectionToSingleton operations into optimizable parts.
Nov 25, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-25T12:45:49.371Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 25, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-25T12:45:49.458Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Nov 25, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-25T12:45:49.496Z: Expanding GroupByKey operations into streaming Read/Write steps
Nov 25, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-25T12:45:49.563Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 25, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-25T12:45:49.668Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 25, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-25T12:45:49.700Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Nov 25, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-25T12:45:49.734Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Nov 25, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-25T12:45:49.759Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Nov 25, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-25T12:45:49.793Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Nov 25, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-25T12:45:49.828Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Nov 25, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-25T12:45:49.859Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Nov 25, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-25T12:45:49.896Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Nov 25, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-25T12:45:49.921Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Nov 25, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-25T12:45:49.948Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Nov 25, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-25T12:45:49.974Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Nov 25, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-25T12:45:50.008Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Nov 25, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-25T12:45:50.037Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Nov 25, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-25T12:45:50.062Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Nov 25, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-25T12:45:50.097Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Nov 25, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-25T12:45:50.121Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Nov 25, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-25T12:45:50.149Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Nov 25, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-25T12:45:50.238Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Nov 25, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-25T12:45:50.379Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Nov 25, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-25T12:45:50.444Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Nov 25, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-25T12:45:50.542Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Nov 25, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-25T12:45:50.590Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Nov 25, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-25T12:45:50.636Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Nov 25, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-25T12:45:51.053Z: Starting 5 ****s in us-central1-a...
Nov 25, 2021 12:46:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-25T12:46:15.449Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Nov 25, 2021 12:46:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-25T12:46:35.424Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Nov 25, 2021 12:47:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-25T12:47:32.822Z: Workers have started successfully.
Nov 25, 2021 12:47:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-25T12:47:32.862Z: Workers have started successfully.
Nov 25, 2021 1:24:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-25T13:24:52.094Z: Staged package commons-compress-1.21-auz9VFlyillWAc-gcljRMZcv_Dm0kutIvdWWV3ovJEo.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/commons-compress-1.21-auz9VFlyillWAc-gcljRMZcv_Dm0kutIvdWWV3ovJEo.jar' is inaccessible.
Nov 25, 2021 1:24:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-25T13:24:52.309Z: Staged package conscrypt-openjdk-uber-2.5.1-AfnHQstZKhUeLmK9U5eomAYoqWcAH82s1KpHRGeGhfM.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/conscrypt-openjdk-uber-2.5.1-AfnHQstZKhUeLmK9U5eomAYoqWcAH82s1KpHRGeGhfM.jar' is inaccessible.
Nov 25, 2021 1:24:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-25T13:24:54.048Z: Staged package netty-buffer-4.1.52.Final-QcoQNa7m4PgXdZfQp2EKWqLG5vrHRKyB_eTl5Y9jlnU.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-buffer-4.1.52.Final-QcoQNa7m4PgXdZfQp2EKWqLG5vrHRKyB_eTl5Y9jlnU.jar' is inaccessible.
Nov 25, 2021 1:24:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-25T13:24:54.095Z: Staged package netty-codec-4.1.52.Final-PH543M5-NT2GrH26LCUatc3dTrhuqoic_kIpeCuCjlI.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-codec-4.1.52.Final-PH543M5-NT2GrH26LCUatc3dTrhuqoic_kIpeCuCjlI.jar' is inaccessible.
Nov 25, 2021 1:24:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-25T13:24:54.137Z: Staged package netty-codec-http-4.1.52.Final-tdoo0OqHiwfTERrPXk-vQSiO8jArJkFBkP3Rp9lzyAw.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-codec-http-4.1.52.Final-tdoo0OqHiwfTERrPXk-vQSiO8jArJkFBkP3Rp9lzyAw.jar' is inaccessible.
Nov 25, 2021 1:24:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-25T13:24:54.188Z: Staged package netty-codec-http2-4.1.52.Final-61J6QtIonRLe5tR1PKJxWXije7tBxxkVzk6VOcSfk6E.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-codec-http2-4.1.52.Final-61J6QtIonRLe5tR1PKJxWXije7tBxxkVzk6VOcSfk6E.jar' is inaccessible.
Nov 25, 2021 1:24:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-25T13:24:54.236Z: Staged package netty-codec-socks-4.1.52.Final-_sxXfgu3ch1naM_BhG7hcpEdFquA372ZldllelUdTqc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-codec-socks-4.1.52.Final-_sxXfgu3ch1naM_BhG7hcpEdFquA372ZldllelUdTqc.jar' is inaccessible.
Nov 25, 2021 1:24:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-25T13:24:54.289Z: Staged package netty-common-4.1.52.Final-DR8QHk231TDtBNfKvVfZXAdQ8Fm7az-t_tDIDUc0xxc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-common-4.1.52.Final-DR8QHk231TDtBNfKvVfZXAdQ8Fm7az-t_tDIDUc0xxc.jar' is inaccessible.
Nov 25, 2021 1:24:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-25T13:24:54.339Z: Staged package netty-handler-4.1.52.Final-LNwh-_rgSUDIWSkDl3nJP1H2GPXvshYHkxnc_PMqVcE.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-handler-4.1.52.Final-LNwh-_rgSUDIWSkDl3nJP1H2GPXvshYHkxnc_PMqVcE.jar' is inaccessible.
Nov 25, 2021 1:24:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-25T13:24:54.389Z: Staged package netty-handler-proxy-4.1.52.Final-6NB4Y860vUgG4lWh1TLIZQ8POy1NAGrW0hgS6UG0L9w.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-handler-proxy-4.1.52.Final-6NB4Y860vUgG4lWh1TLIZQ8POy1NAGrW0hgS6UG0L9w.jar' is inaccessible.
Nov 25, 2021 1:24:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-25T13:24:54.437Z: Staged package netty-resolver-4.1.52.Final-butyojOXnLRbC01tRsWkEzLFfAz4qAOx78YNu-y6CaY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-resolver-4.1.52.Final-butyojOXnLRbC01tRsWkEzLFfAz4qAOx78YNu-y6CaY.jar' is inaccessible.
Nov 25, 2021 1:24:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-25T13:24:54.502Z: Staged package netty-transport-4.1.52.Final-mj5vjA5V3jY-seoQ_ngXl-yjlOYhht8q4LTrK84LRUE.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-transport-4.1.52.Final-mj5vjA5V3jY-seoQ_ngXl-yjlOYhht8q4LTrK84LRUE.jar' is inaccessible.
Nov 25, 2021 1:24:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-25T13:24:54.546Z: Staged package opencensus-api-0.28.0-DBcj8_bTBhMjhFzouIs1_dpQCBLgp1uOtfzErYyHGpU.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/opencensus-api-0.28.0-DBcj8_bTBhMjhFzouIs1_dpQCBLgp1uOtfzErYyHGpU.jar' is inaccessible.
Nov 25, 2021 1:24:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-25T13:24:54.636Z: Staged package paranamer-2.8-aIyxGKYCHYGROOhVIIyVYDFoi-S0eiS7YVvsxjrO3wc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/paranamer-2.8-aIyxGKYCHYGROOhVIIyVYDFoi-S0eiS7YVvsxjrO3wc.jar' is inaccessible.
Nov 25, 2021 1:24:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-25T13:24:54.680Z: Staged package perfmark-api-0.23.0-xwW1wQwY_zAyuegXQrwvaw5WB_am38DIrQz_ddSRMEI.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/perfmark-api-0.23.0-xwW1wQwY_zAyuegXQrwvaw5WB_am38DIrQz_ddSRMEI.jar' is inaccessible.
Nov 25, 2021 1:24:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-25T13:24:55.041Z: Staged package snappy-java-1.1.8.4-JMTR_B6J4HgzGrj0AamcrWhZm95KLkUWBCy1SMUbHD4.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/snappy-java-1.1.8.4-JMTR_B6J4HgzGrj0AamcrWhZm95KLkUWBCy1SMUbHD4.jar' is inaccessible.
Nov 25, 2021 1:24:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-11-25T13:24:55.199Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Nov 25, 2021 1:27:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-11-25T13:27:55.781Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Nov 25, 2021 1:30:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-25T13:30:52.167Z: Staged package commons-compress-1.21-auz9VFlyillWAc-gcljRMZcv_Dm0kutIvdWWV3ovJEo.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/commons-compress-1.21-auz9VFlyillWAc-gcljRMZcv_Dm0kutIvdWWV3ovJEo.jar' is inaccessible.
Nov 25, 2021 1:30:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-25T13:30:52.292Z: Staged package conscrypt-openjdk-uber-2.5.1-AfnHQstZKhUeLmK9U5eomAYoqWcAH82s1KpHRGeGhfM.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/conscrypt-openjdk-uber-2.5.1-AfnHQstZKhUeLmK9U5eomAYoqWcAH82s1KpHRGeGhfM.jar' is inaccessible.
Nov 25, 2021 1:30:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-25T13:30:53.931Z: Staged package netty-buffer-4.1.52.Final-QcoQNa7m4PgXdZfQp2EKWqLG5vrHRKyB_eTl5Y9jlnU.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-buffer-4.1.52.Final-QcoQNa7m4PgXdZfQp2EKWqLG5vrHRKyB_eTl5Y9jlnU.jar' is inaccessible.
Nov 25, 2021 1:30:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-25T13:30:54.026Z: Staged package netty-codec-4.1.52.Final-PH543M5-NT2GrH26LCUatc3dTrhuqoic_kIpeCuCjlI.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-codec-4.1.52.Final-PH543M5-NT2GrH26LCUatc3dTrhuqoic_kIpeCuCjlI.jar' is inaccessible.
Nov 25, 2021 1:30:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-25T13:30:54.078Z: Staged package netty-codec-http-4.1.52.Final-tdoo0OqHiwfTERrPXk-vQSiO8jArJkFBkP3Rp9lzyAw.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-codec-http-4.1.52.Final-tdoo0OqHiwfTERrPXk-vQSiO8jArJkFBkP3Rp9lzyAw.jar' is inaccessible.
Nov 25, 2021 1:30:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-25T13:30:54.132Z: Staged package netty-codec-http2-4.1.52.Final-61J6QtIonRLe5tR1PKJxWXije7tBxxkVzk6VOcSfk6E.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-codec-http2-4.1.52.Final-61J6QtIonRLe5tR1PKJxWXije7tBxxkVzk6VOcSfk6E.jar' is inaccessible.
Nov 25, 2021 1:30:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-25T13:30:54.178Z: Staged package netty-codec-socks-4.1.52.Final-_sxXfgu3ch1naM_BhG7hcpEdFquA372ZldllelUdTqc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-codec-socks-4.1.52.Final-_sxXfgu3ch1naM_BhG7hcpEdFquA372ZldllelUdTqc.jar' is inaccessible.
Nov 25, 2021 1:30:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-25T13:30:54.240Z: Staged package netty-common-4.1.52.Final-DR8QHk231TDtBNfKvVfZXAdQ8Fm7az-t_tDIDUc0xxc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-common-4.1.52.Final-DR8QHk231TDtBNfKvVfZXAdQ8Fm7az-t_tDIDUc0xxc.jar' is inaccessible.
Nov 25, 2021 1:30:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-25T13:30:54.288Z: Staged package netty-handler-4.1.52.Final-LNwh-_rgSUDIWSkDl3nJP1H2GPXvshYHkxnc_PMqVcE.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-handler-4.1.52.Final-LNwh-_rgSUDIWSkDl3nJP1H2GPXvshYHkxnc_PMqVcE.jar' is inaccessible.
Nov 25, 2021 1:30:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-25T13:30:54.336Z: Staged package netty-handler-proxy-4.1.52.Final-6NB4Y860vUgG4lWh1TLIZQ8POy1NAGrW0hgS6UG0L9w.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-handler-proxy-4.1.52.Final-6NB4Y860vUgG4lWh1TLIZQ8POy1NAGrW0hgS6UG0L9w.jar' is inaccessible.
Nov 25, 2021 1:30:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-25T13:30:54.376Z: Staged package netty-resolver-4.1.52.Final-butyojOXnLRbC01tRsWkEzLFfAz4qAOx78YNu-y6CaY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-resolver-4.1.52.Final-butyojOXnLRbC01tRsWkEzLFfAz4qAOx78YNu-y6CaY.jar' is inaccessible.
Nov 25, 2021 1:30:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-25T13:30:54.439Z: Staged package netty-transport-4.1.52.Final-mj5vjA5V3jY-seoQ_ngXl-yjlOYhht8q4LTrK84LRUE.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-transport-4.1.52.Final-mj5vjA5V3jY-seoQ_ngXl-yjlOYhht8q4LTrK84LRUE.jar' is inaccessible.
Nov 25, 2021 1:30:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-25T13:30:54.479Z: Staged package opencensus-api-0.28.0-DBcj8_bTBhMjhFzouIs1_dpQCBLgp1uOtfzErYyHGpU.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/opencensus-api-0.28.0-DBcj8_bTBhMjhFzouIs1_dpQCBLgp1uOtfzErYyHGpU.jar' is inaccessible.
Nov 25, 2021 1:30:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-25T13:30:54.547Z: Staged package paranamer-2.8-aIyxGKYCHYGROOhVIIyVYDFoi-S0eiS7YVvsxjrO3wc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/paranamer-2.8-aIyxGKYCHYGROOhVIIyVYDFoi-S0eiS7YVvsxjrO3wc.jar' is inaccessible.
Nov 25, 2021 1:30:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-25T13:30:54.594Z: Staged package perfmark-api-0.23.0-xwW1wQwY_zAyuegXQrwvaw5WB_am38DIrQz_ddSRMEI.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/perfmark-api-0.23.0-xwW1wQwY_zAyuegXQrwvaw5WB_am38DIrQz_ddSRMEI.jar' is inaccessible.
Nov 25, 2021 1:30:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-25T13:30:55.010Z: Staged package snappy-java-1.1.8.4-JMTR_B6J4HgzGrj0AamcrWhZm95KLkUWBCy1SMUbHD4.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/snappy-java-1.1.8.4-JMTR_B6J4HgzGrj0AamcrWhZm95KLkUWBCy1SMUbHD4.jar' is inaccessible.
Nov 25, 2021 1:30:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-11-25T13:30:55.152Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Nov 25, 2021 4:00:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-25T16:00:32.346Z: Cancel request is committed for workflow job: 2021-11-25_04_45_36-14387302763703776575.
Nov 25, 2021 4:00:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-25T16:00:32.386Z: Cleaning up.
Nov 25, 2021 4:00:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-25T16:00:32.456Z: Stopping **** pool...
Nov 25, 2021 4:00:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-25T16:00:32.516Z: Stopping **** pool...
Nov 25, 2021 4:02:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-25T16:02:51.241Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Nov 25, 2021 4:02:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-25T16:02:51.280Z: Worker pool stopped.
Nov 25, 2021 4:02:57 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-11-25_04_45_36-14387302763703776575 finished with status CANCELLED.
Load test results for test (ID): 630cae55-715d-429c-a625-304c78a81027 and timestamp: 2021-11-25T12:45:30.711000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11475.977
dataflow_v2_java11_total_bytes_count             1.94129639E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211125124332
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:b24df267d99449fe3394d08dfd4d79b8991c418e36cbadf82cc63e48619a8582
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211125124332]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:b24df267d99449fe3394d08dfd4d79b8991c418e36cbadf82cc63e48619a8582]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211125124332] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:b24df267d99449fe3394d08dfd4d79b8991c418e36cbadf82cc63e48619a8582])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:b24df267d99449fe3394d08dfd4d79b8991c418e36cbadf82cc63e48619a8582
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:b24df267d99449fe3394d08dfd4d79b8991c418e36cbadf82cc63e48619a8582
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:b24df267d99449fe3394d08dfd4d79b8991c418e36cbadf82cc63e48619a8582].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 43s
101 actionable tasks: 71 executed, 28 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/prwnunraitcqc

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #160

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/160/display/redirect?page=changes>

Changes:

[noreply] Increase the timeout for LoadTest Jenkins jobs 4h->12h.

[noreply] [BEAM-13073] UseParallelGC on Java container

[noreply] Merge pull request #16002 from [BEAM-13101] [Playground] load examples

[noreply] Merge pull request #16016 from [BEAM-13089][Playground] Enable

[noreply] Merge pull request #16046 from [BEAM-13295] [Playground] [Bugfix]

[noreply] Merge pull request #16048 from [Beam-13256] [Playground] Implement an

[noreply] Merge pull request #16026 from [BEAM-13208][Playground] Separate stderr

[noreply] Append to state in chunks. (#15983)

[Luke Cwik] [BEAM-13313] Fix WindowingStrategy proto to use snake case field names.

[noreply] [BEAM-12829] Never copy .gogradle/** for Python. (#16059)


------------------------------------------
[...truncated 48.37 KB...]

The push refers to repository [us.gcr.io/apache-beam-testing/java-postcommit-it/java]
51bea171d5f6: Preparing
838006e56d1c: Preparing
eaf55fdd28d2: Preparing
c70e269b2ef0: Preparing
cb59a9e06435: Preparing
5763c90fc677: Preparing
10a991f910d4: Preparing
536966cdeac6: Preparing
0416872d8098: Preparing
fddc5bcb125e: Preparing
19397597ab6b: Preparing
80463cec54a1: Preparing
a9e4c9343539: Preparing
47ee2d19f81a: Preparing
ab9d251e27cb: Preparing
8a5844586fdb: Preparing
a4aba4e59b40: Preparing
5499f2905579: Preparing
a36ba9e322f7: Preparing
80463cec54a1: Waiting
a9e4c9343539: Waiting
10a991f910d4: Waiting
47ee2d19f81a: Waiting
536966cdeac6: Waiting
ab9d251e27cb: Waiting
19397597ab6b: Waiting
fddc5bcb125e: Waiting
5763c90fc677: Waiting
8a5844586fdb: Waiting
a4aba4e59b40: Waiting
5499f2905579: Waiting
eaf55fdd28d2: Pushed
cb59a9e06435: Pushed
838006e56d1c: Pushed
5763c90fc677: Pushed
51bea171d5f6: Pushed
c70e269b2ef0: Pushed
536966cdeac6: Pushed
0416872d8098: Pushed
a9e4c9343539: Layer already exists
47ee2d19f81a: Layer already exists
10a991f910d4: Pushed
ab9d251e27cb: Layer already exists
8a5844586fdb: Layer already exists
19397597ab6b: Pushed
a4aba4e59b40: Layer already exists
80463cec54a1: Pushed
a36ba9e322f7: Layer already exists
5499f2905579: Layer already exists
fddc5bcb125e: Pushed
20211124124332: digest: sha256:ec7ad05fef57bf2159bc12994cea371557758d553489bcf5a5cb864fe099730d size: 4311

> Task :sdks:java:testing:load-tests:run
Nov 24, 2021 12:50:49 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Nov 24, 2021 12:50:49 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 197 files. Enable logging at DEBUG level to see which files will be staged.
Nov 24, 2021 12:50:50 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Nov 24, 2021 12:50:50 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Nov 24, 2021 12:50:52 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 197 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Nov 24, 2021 12:50:53 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 197 files cached, 0 files newly uploaded in 0 seconds
Nov 24, 2021 12:50:53 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Nov 24, 2021 12:50:53 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <111716 bytes, hash d87390877bb0bde0a3029ff92ab2bd5adbeeb9738bed6cb0fd51f3e5e0bb7052> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-2HOQh3uwveCjAp_5KrK9WtvuuXOL7Wyw_VHz5eC7cFI.pb
Nov 24, 2021 12:50:54 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Nov 24, 2021 12:50:54 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@177515d1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@52ff99cd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4c2af006, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@44032fde, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b676112, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5578be42, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4e49ce2b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@136965e3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@53c6f96d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@435cc7f9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4364712f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1b7a52dd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7f93dd4e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5ad5be4a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ad85136, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@737d100a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12e5da86, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6535117e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1d1cbd0f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6fa13e6]
Nov 24, 2021 12:50:54 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Nov 24, 2021 12:50:54 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Nov 24, 2021 12:50:54 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Nov 24, 2021 12:50:54 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Nov 24, 2021 12:50:54 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3f183caa, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b66322e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@63538bb4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@24534cb0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5a50d9fc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@106d77da, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6f9c5048, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5114b7c7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@dd71b20, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@767f6ee7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b6c6e70, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@f324455, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3a894088, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@370c1968, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2f0bfe17, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3206174f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@42fb8c87, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@15eb0ae9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@65e0b505, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@67de7a99]
Nov 24, 2021 12:50:54 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Nov 24, 2021 12:50:54 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Nov 24, 2021 12:50:54 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Nov 24, 2021 12:50:54 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Nov 24, 2021 12:50:54 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Nov 24, 2021 12:50:54 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Nov 24, 2021 12:50:54 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Nov 24, 2021 12:50:54 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Nov 24, 2021 12:50:54 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Nov 24, 2021 12:50:54 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Nov 24, 2021 12:50:54 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Nov 24, 2021 12:50:55 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.36.0-SNAPSHOT
Nov 24, 2021 12:50:55 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-11-24_04_50_55-1265207853464830809?project=apache-beam-testing
Nov 24, 2021 12:50:55 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-11-24_04_50_55-1265207853464830809
Nov 24, 2021 12:50:55 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-11-24_04_50_55-1265207853464830809
Nov 24, 2021 12:51:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-11-24T12:51:02.033Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-11-jih7. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Nov 24, 2021 12:51:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-24T12:51:06.151Z: Worker configuration: e2-standard-2 in us-central1-a.
Nov 24, 2021 12:51:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-24T12:51:06.579Z: Expanding SplittableParDo operations into optimizable parts.
Nov 24, 2021 12:51:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-24T12:51:06.626Z: Expanding CollectionToSingleton operations into optimizable parts.
Nov 24, 2021 12:51:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-24T12:51:06.683Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 24, 2021 12:51:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-24T12:51:06.767Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Nov 24, 2021 12:51:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-24T12:51:06.795Z: Expanding GroupByKey operations into streaming Read/Write steps
Nov 24, 2021 12:51:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-24T12:51:06.861Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 24, 2021 12:51:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-24T12:51:06.973Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 24, 2021 12:51:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-24T12:51:07.006Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Nov 24, 2021 12:51:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-24T12:51:07.048Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Nov 24, 2021 12:51:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-24T12:51:07.085Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Nov 24, 2021 12:51:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-24T12:51:07.114Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Nov 24, 2021 12:51:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-24T12:51:07.148Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Nov 24, 2021 12:51:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-24T12:51:07.181Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Nov 24, 2021 12:51:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-24T12:51:07.217Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Nov 24, 2021 12:51:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-24T12:51:07.249Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Nov 24, 2021 12:51:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-24T12:51:07.284Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Nov 24, 2021 12:51:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-24T12:51:07.315Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Nov 24, 2021 12:51:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-24T12:51:07.348Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Nov 24, 2021 12:51:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-24T12:51:07.383Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Nov 24, 2021 12:51:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-24T12:51:07.405Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Nov 24, 2021 12:51:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-24T12:51:07.429Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Nov 24, 2021 12:51:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-24T12:51:07.451Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Nov 24, 2021 12:51:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-24T12:51:07.477Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Nov 24, 2021 12:51:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-24T12:51:07.508Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Nov 24, 2021 12:51:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-24T12:51:07.534Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Nov 24, 2021 12:51:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-24T12:51:07.560Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Nov 24, 2021 12:51:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-24T12:51:07.589Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Nov 24, 2021 12:51:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-24T12:51:07.613Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Nov 24, 2021 12:51:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-24T12:51:07.657Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Nov 24, 2021 12:51:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-24T12:51:07.982Z: Starting 5 ****s in us-central1-a...
Nov 24, 2021 12:51:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-24T12:51:41.877Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Nov 24, 2021 12:51:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-24T12:51:52.623Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Nov 24, 2021 12:52:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-24T12:52:46.146Z: Workers have started successfully.
Nov 24, 2021 12:52:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-24T12:52:46.184Z: Workers have started successfully.
Nov 24, 2021 4:00:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-24T16:00:24.257Z: Cancel request is committed for workflow job: 2021-11-24_04_50_55-1265207853464830809.
Nov 24, 2021 4:00:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-24T16:00:24.336Z: Cleaning up.
Nov 24, 2021 4:00:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-24T16:00:24.408Z: Stopping **** pool...
Nov 24, 2021 4:00:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-24T16:00:24.466Z: Stopping **** pool...
Nov 24, 2021 4:02:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-24T16:02:51.787Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Nov 24, 2021 4:02:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-24T16:02:51.824Z: Worker pool stopped.
Nov 24, 2021 4:02:59 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-11-24_04_50_55-1265207853464830809 finished with status CANCELLED.
Load test results for test (ID): 5ebf6421-0287-43dc-9698-1208f36d72e8 and timestamp: 2021-11-24T12:50:50.252000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11217.102
dataflow_v2_java11_total_bytes_count             1.71173265E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211124124332
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ec7ad05fef57bf2159bc12994cea371557758d553489bcf5a5cb864fe099730d
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211124124332]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ec7ad05fef57bf2159bc12994cea371557758d553489bcf5a5cb864fe099730d]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211124124332] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ec7ad05fef57bf2159bc12994cea371557758d553489bcf5a5cb864fe099730d])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ec7ad05fef57bf2159bc12994cea371557758d553489bcf5a5cb864fe099730d
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ec7ad05fef57bf2159bc12994cea371557758d553489bcf5a5cb864fe099730d
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ec7ad05fef57bf2159bc12994cea371557758d553489bcf5a5cb864fe099730d].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 44s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/rrb4menh2jgx4

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #159

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/159/display/redirect?page=changes>

Changes:

[Robert Bradshaw] Make sure implicit inputs are noted in the portable graph.

[benjamin.gonzalez] [BEAM-13003] Integration Test for DebeziumIO that connects to

[noreply] Merge pull request #16032 from [BEAM-13297][Playground][Bugfix]

[noreply] Merge pull request #16031 from [BEAM-13296][Playground][Bugfix] remove

[noreply] Merge pull request #16030 from [BEAM-13219][Playground] Initialise

[noreply] Merge pull request #15996 from [BEAM-13156][Playground] Implement


------------------------------------------
[...truncated 48.24 KB...]
b74b745bfc34: Preparing
b1f85dad940c: Preparing
a9e4c9343539: Preparing
47ee2d19f81a: Preparing
ab9d251e27cb: Preparing
8a5844586fdb: Preparing
a4aba4e59b40: Preparing
5499f2905579: Preparing
a36ba9e322f7: Preparing
d7604a28e3c8: Waiting
a4aba4e59b40: Waiting
8a5844586fdb: Waiting
6014bd277026: Waiting
b1f85dad940c: Waiting
5499f2905579: Waiting
311671353f63: Waiting
260d6aa87acb: Waiting
a9e4c9343539: Waiting
a36ba9e322f7: Waiting
231770730dba: Waiting
47ee2d19f81a: Waiting
b74b745bfc34: Waiting
ab9d251e27cb: Waiting
c0b43a29c131: Pushed
b2f605300a8b: Pushed
0f1a4dcd5c41: Pushed
7afb0064df19: Pushed
d7604a28e3c8: Pushed
1fd7139284c3: Pushed
311671353f63: Pushed
260d6aa87acb: Pushed
a9e4c9343539: Layer already exists
47ee2d19f81a: Layer already exists
ab9d251e27cb: Layer already exists
b74b745bfc34: Pushed
8a5844586fdb: Layer already exists
a4aba4e59b40: Layer already exists
5499f2905579: Layer already exists
a36ba9e322f7: Layer already exists
b1f85dad940c: Pushed
6014bd277026: Pushed
231770730dba: Pushed
20211123124333: digest: sha256:66eef5700c211a3c64af38ce5c6146d89a8fc5ead4592cc8b04857b781f5846d size: 4311

> Task :sdks:java:testing:load-tests:run
Nov 23, 2021 12:45:48 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Nov 23, 2021 12:45:49 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 197 files. Enable logging at DEBUG level to see which files will be staged.
Nov 23, 2021 12:45:49 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Nov 23, 2021 12:45:49 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Nov 23, 2021 12:45:52 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 197 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Nov 23, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 197 files cached, 0 files newly uploaded in 0 seconds
Nov 23, 2021 12:45:53 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Nov 23, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <111716 bytes, hash 6c0ed3d7b543d743c14eaf66a9beb0dde45cb5f451d0ecd04dd2ee20b5d80417> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-bA7T17VD10PBTq9mqb6w3eRctfRR0OzQTdLuILXYBBc.pb
Nov 23, 2021 12:45:55 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Nov 23, 2021 12:45:55 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@44032fde, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b676112, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5578be42, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4e49ce2b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@136965e3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@53c6f96d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@435cc7f9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4364712f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1b7a52dd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7f93dd4e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5ad5be4a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ad85136, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@737d100a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12e5da86, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6535117e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1d1cbd0f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6fa13e6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3af7d855, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77049094, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@f88bfbe]
Nov 23, 2021 12:45:55 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Nov 23, 2021 12:45:55 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Nov 23, 2021 12:45:55 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Nov 23, 2021 12:45:55 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Nov 23, 2021 12:45:55 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@f1868c9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4ba6ec50, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@642413d4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@fb2e3fd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43a09ce2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3f183caa, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b66322e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@63538bb4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@24534cb0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5a50d9fc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@106d77da, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6f9c5048, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5114b7c7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@dd71b20, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@767f6ee7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b6c6e70, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@f324455, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3a894088, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@370c1968, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2f0bfe17]
Nov 23, 2021 12:45:55 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Nov 23, 2021 12:45:55 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Nov 23, 2021 12:45:55 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Nov 23, 2021 12:45:55 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Nov 23, 2021 12:45:55 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Nov 23, 2021 12:45:55 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Nov 23, 2021 12:45:55 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Nov 23, 2021 12:45:55 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Nov 23, 2021 12:45:55 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Nov 23, 2021 12:45:55 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Nov 23, 2021 12:45:55 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Nov 23, 2021 12:45:55 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.36.0-SNAPSHOT
Nov 23, 2021 12:45:56 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-11-23_04_45_55-15625709898218145271?project=apache-beam-testing
Nov 23, 2021 12:45:56 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-11-23_04_45_55-15625709898218145271
Nov 23, 2021 12:45:56 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-11-23_04_45_55-15625709898218145271
Nov 23, 2021 12:46:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-11-23T12:46:03.975Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-11-g293. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Nov 23, 2021 12:46:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-23T12:46:09.770Z: Worker configuration: e2-standard-2 in us-central1-a.
Nov 23, 2021 12:46:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-23T12:46:10.410Z: Expanding SplittableParDo operations into optimizable parts.
Nov 23, 2021 12:46:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-23T12:46:10.441Z: Expanding CollectionToSingleton operations into optimizable parts.
Nov 23, 2021 12:46:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-23T12:46:10.508Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 23, 2021 12:46:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-23T12:46:10.579Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Nov 23, 2021 12:46:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-23T12:46:10.638Z: Expanding GroupByKey operations into streaming Read/Write steps
Nov 23, 2021 12:46:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-23T12:46:10.699Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 23, 2021 12:46:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-23T12:46:10.814Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 23, 2021 12:46:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-23T12:46:10.857Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Nov 23, 2021 12:46:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-23T12:46:10.892Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Nov 23, 2021 12:46:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-23T12:46:10.925Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Nov 23, 2021 12:46:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-23T12:46:10.960Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Nov 23, 2021 12:46:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-23T12:46:10.985Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Nov 23, 2021 12:46:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-23T12:46:11.009Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Nov 23, 2021 12:46:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-23T12:46:11.026Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Nov 23, 2021 12:46:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-23T12:46:11.058Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Nov 23, 2021 12:46:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-23T12:46:11.088Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Nov 23, 2021 12:46:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-23T12:46:11.132Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Nov 23, 2021 12:46:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-23T12:46:11.168Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Nov 23, 2021 12:46:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-23T12:46:11.197Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Nov 23, 2021 12:46:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-23T12:46:11.227Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Nov 23, 2021 12:46:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-23T12:46:11.260Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Nov 23, 2021 12:46:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-23T12:46:11.292Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Nov 23, 2021 12:46:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-23T12:46:11.320Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Nov 23, 2021 12:46:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-23T12:46:11.343Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Nov 23, 2021 12:46:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-23T12:46:11.378Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Nov 23, 2021 12:46:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-23T12:46:11.408Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Nov 23, 2021 12:46:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-23T12:46:11.443Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Nov 23, 2021 12:46:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-23T12:46:11.478Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Nov 23, 2021 12:46:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-23T12:46:11.513Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Nov 23, 2021 12:46:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-23T12:46:11.875Z: Starting 5 ****s in us-central1-a...
Nov 23, 2021 12:46:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-23T12:46:23.117Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Nov 23, 2021 12:47:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-23T12:47:05.428Z: Autoscaling: Raised the number of ****s to 2 so that the pipeline can catch up with its backlog and keep up with its input rate.
Nov 23, 2021 12:47:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-23T12:47:05.461Z: Resized **** pool to 2, though goal was 5.  This could be a quota issue.
Nov 23, 2021 12:47:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-23T12:47:15.655Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Nov 23, 2021 12:48:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-23T12:48:10.228Z: Workers have started successfully.
Nov 23, 2021 12:48:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-23T12:48:10.264Z: Workers have started successfully.
Nov 23, 2021 3:13:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-23T15:13:15.027Z: Staged package kafka-clients-2.4.1-or4Vrm01S3aXE64MuzGE8iWqFvyYIFseskr9n8KjYNk.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/kafka-clients-2.4.1-or4Vrm01S3aXE64MuzGE8iWqFvyYIFseskr9n8KjYNk.jar' is inaccessible.
Nov 23, 2021 3:13:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-23T15:13:15.159Z: Staged package lz4-java-1.6.0-0ilUWqKx1SA8h2YUvbz_yswwNpf0-PJvdk4dbB7S5BY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/lz4-java-1.6.0-0ilUWqKx1SA8h2YUvbz_yswwNpf0-PJvdk4dbB7S5BY.jar' is inaccessible.
Nov 23, 2021 3:13:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-11-23T15:13:16.026Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Nov 23, 2021 4:00:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-23T16:00:43.184Z: Cancel request is committed for workflow job: 2021-11-23_04_45_55-15625709898218145271.
Nov 23, 2021 4:00:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-23T16:00:43.275Z: Cleaning up.
Nov 23, 2021 4:00:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-23T16:00:43.345Z: Stopping **** pool...
Nov 23, 2021 4:00:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-23T16:00:43.413Z: Stopping **** pool...
Nov 23, 2021 4:03:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-23T16:03:07.804Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Nov 23, 2021 4:03:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-23T16:03:07.848Z: Worker pool stopped.
Nov 23, 2021 4:03:15 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-11-23_04_45_55-15625709898218145271 finished with status CANCELLED.
Load test results for test (ID): ae3c8169-6cd6-45a5-847e-940829c58929 and timestamp: 2021-11-23T12:45:49.428000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11477.109
dataflow_v2_java11_total_bytes_count             2.56585283E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211123124333
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:66eef5700c211a3c64af38ce5c6146d89a8fc5ead4592cc8b04857b781f5846d
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211123124333]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:66eef5700c211a3c64af38ce5c6146d89a8fc5ead4592cc8b04857b781f5846d]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211123124333] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:66eef5700c211a3c64af38ce5c6146d89a8fc5ead4592cc8b04857b781f5846d])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:66eef5700c211a3c64af38ce5c6146d89a8fc5ead4592cc8b04857b781f5846d
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:66eef5700c211a3c64af38ce5c6146d89a8fc5ead4592cc8b04857b781f5846d
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:66eef5700c211a3c64af38ce5c6146d89a8fc5ead4592cc8b04857b781f5846d].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 58s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/sx33if7sjajmm

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #158

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/158/display/redirect>

Changes:


------------------------------------------
[...truncated 49.22 KB...]
72e960c5fc8a: Preparing
57223c2604cb: Preparing
da2ae2375a65: Preparing
021f027cdc5e: Preparing
c1ce556ab59f: Preparing
378a7fe2cc02: Preparing
a9e4c9343539: Preparing
47ee2d19f81a: Preparing
ab9d251e27cb: Preparing
8a5844586fdb: Preparing
a4aba4e59b40: Preparing
5499f2905579: Preparing
a36ba9e322f7: Preparing
5c9fef3dc784: Waiting
47ee2d19f81a: Waiting
72e960c5fc8a: Waiting
ab9d251e27cb: Waiting
8a5844586fdb: Waiting
57223c2604cb: Waiting
a4aba4e59b40: Waiting
c1ce556ab59f: Waiting
5499f2905579: Waiting
021f027cdc5e: Waiting
378a7fe2cc02: Waiting
a36ba9e322f7: Waiting
da2ae2375a65: Waiting
a9e4c9343539: Waiting
c780e152516c: Pushed
1e96de073336: Pushed
80c10b07718b: Pushed
0c5a68def77b: Pushed
5c9fef3dc784: Pushed
f5aaec88dbeb: Pushed
da2ae2375a65: Pushed
57223c2604cb: Pushed
c1ce556ab59f: Pushed
a9e4c9343539: Layer already exists
47ee2d19f81a: Layer already exists
ab9d251e27cb: Layer already exists
8a5844586fdb: Layer already exists
a4aba4e59b40: Layer already exists
5499f2905579: Layer already exists
72e960c5fc8a: Pushed
a36ba9e322f7: Layer already exists
378a7fe2cc02: Pushed
021f027cdc5e: Pushed
20211122124329: digest: sha256:1ad68dc1a2997caca043965ce59064e446f0825db08c222b2f134f61bacc8b7b size: 4311

> Task :sdks:java:testing:load-tests:run
Nov 22, 2021 12:45:17 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Nov 22, 2021 12:45:17 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 197 files. Enable logging at DEBUG level to see which files will be staged.
Nov 22, 2021 12:45:18 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Nov 22, 2021 12:45:18 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Nov 22, 2021 12:45:20 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 197 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Nov 22, 2021 12:45:20 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 197 files cached, 0 files newly uploaded in 0 seconds
Nov 22, 2021 12:45:20 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Nov 22, 2021 12:45:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <111716 bytes, hash dd3ad8e5ac38c666aaa706ccdf72ba6766832e74c2dc1a99f10bb3bec0c1e515> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-3TrY5aw4xmaqpwbM33K6Z2aDLnTC3BqZ8QuzvsDB5RU.pb
Nov 22, 2021 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Nov 22, 2021 12:45:22 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@177515d1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@52ff99cd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4c2af006, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@44032fde, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b676112, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5578be42, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4e49ce2b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@136965e3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@53c6f96d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@435cc7f9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4364712f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1b7a52dd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7f93dd4e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5ad5be4a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ad85136, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@737d100a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12e5da86, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6535117e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1d1cbd0f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6fa13e6]
Nov 22, 2021 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Nov 22, 2021 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Nov 22, 2021 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Nov 22, 2021 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Nov 22, 2021 12:45:22 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3f183caa, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b66322e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@63538bb4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@24534cb0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5a50d9fc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@106d77da, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6f9c5048, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5114b7c7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@dd71b20, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@767f6ee7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b6c6e70, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@f324455, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3a894088, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@370c1968, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2f0bfe17, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3206174f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@42fb8c87, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@15eb0ae9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@65e0b505, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@67de7a99]
Nov 22, 2021 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Nov 22, 2021 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Nov 22, 2021 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Nov 22, 2021 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Nov 22, 2021 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Nov 22, 2021 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Nov 22, 2021 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Nov 22, 2021 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Nov 22, 2021 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Nov 22, 2021 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Nov 22, 2021 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Nov 22, 2021 12:45:22 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.36.0-SNAPSHOT
Nov 22, 2021 12:45:23 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-11-22_04_45_22-3733807565465347251?project=apache-beam-testing
Nov 22, 2021 12:45:23 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-11-22_04_45_22-3733807565465347251
Nov 22, 2021 12:45:23 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-11-22_04_45_22-3733807565465347251
Nov 22, 2021 12:45:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-11-22T12:45:28.610Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-11-13yh. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Nov 22, 2021 12:45:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-22T12:45:32.493Z: Worker configuration: e2-standard-2 in us-central1-a.
Nov 22, 2021 12:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-22T12:45:33.144Z: Expanding SplittableParDo operations into optimizable parts.
Nov 22, 2021 12:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-22T12:45:33.212Z: Expanding CollectionToSingleton operations into optimizable parts.
Nov 22, 2021 12:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-22T12:45:33.366Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 22, 2021 12:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-22T12:45:33.478Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Nov 22, 2021 12:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-22T12:45:33.507Z: Expanding GroupByKey operations into streaming Read/Write steps
Nov 22, 2021 12:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-22T12:45:33.560Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 22, 2021 12:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-22T12:45:33.700Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 22, 2021 12:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-22T12:45:33.734Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Nov 22, 2021 12:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-22T12:45:33.768Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Nov 22, 2021 12:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-22T12:45:33.799Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Nov 22, 2021 12:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-22T12:45:33.821Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Nov 22, 2021 12:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-22T12:45:33.856Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Nov 22, 2021 12:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-22T12:45:33.889Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Nov 22, 2021 12:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-22T12:45:33.920Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Nov 22, 2021 12:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-22T12:45:33.966Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Nov 22, 2021 12:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-22T12:45:33.991Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Nov 22, 2021 12:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-22T12:45:34.014Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Nov 22, 2021 12:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-22T12:45:34.067Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Nov 22, 2021 12:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-22T12:45:34.099Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Nov 22, 2021 12:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-22T12:45:34.132Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Nov 22, 2021 12:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-22T12:45:34.164Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Nov 22, 2021 12:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-22T12:45:34.208Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Nov 22, 2021 12:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-22T12:45:34.244Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Nov 22, 2021 12:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-22T12:45:34.274Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Nov 22, 2021 12:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-22T12:45:34.308Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Nov 22, 2021 12:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-22T12:45:34.340Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Nov 22, 2021 12:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-22T12:45:34.371Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Nov 22, 2021 12:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-22T12:45:34.408Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Nov 22, 2021 12:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-22T12:45:34.438Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Nov 22, 2021 12:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-22T12:45:34.846Z: Starting 5 ****s in us-central1-a...
Nov 22, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-22T12:45:46.869Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Nov 22, 2021 12:46:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-22T12:46:20.521Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Nov 22, 2021 12:47:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-22T12:47:14.676Z: Workers have started successfully.
Nov 22, 2021 12:47:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-22T12:47:14.709Z: Workers have started successfully.
Nov 22, 2021 2:54:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-22T14:54:35.796Z: Staged package classgraph-4.8.104-ePvfM6WdiJMBzk05ytbzmopew--T1rqGqw_PYcvi6h0.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/classgraph-4.8.104-ePvfM6WdiJMBzk05ytbzmopew--T1rqGqw_PYcvi6h0.jar' is inaccessible.
Nov 22, 2021 2:54:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-22T14:54:37.408Z: Staged package junit-4.13.1-wwcZ25dNZFJ5P-GRs2OKV3cAVIW64UWSQERTD_pfYSI.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/junit-4.13.1-wwcZ25dNZFJ5P-GRs2OKV3cAVIW64UWSQERTD_pfYSI.jar' is inaccessible.
Nov 22, 2021 2:54:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-11-22T14:54:38.247Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Nov 22, 2021 4:00:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-22T16:00:34.551Z: Cancel request is committed for workflow job: 2021-11-22_04_45_22-3733807565465347251.
Nov 22, 2021 4:00:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-22T16:00:34.588Z: Cleaning up.
Nov 22, 2021 4:00:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-22T16:00:34.658Z: Stopping **** pool...
Nov 22, 2021 4:00:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-22T16:00:34.726Z: Stopping **** pool...
Nov 22, 2021 4:03:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-22T16:03:00.489Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Nov 22, 2021 4:03:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-22T16:03:00.527Z: Worker pool stopped.
Nov 22, 2021 4:03:10 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-11-22_04_45_22-3733807565465347251 finished with status CANCELLED.
Load test results for test (ID): 476b5c73-2aee-4ee4-8fb1-b7f7ef9a28ed and timestamp: 2021-11-22T12:45:18.020000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11564.866
dataflow_v2_java11_total_bytes_count             1.61944335E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211122124329
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:1ad68dc1a2997caca043965ce59064e446f0825db08c222b2f134f61bacc8b7b
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211122124329]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:1ad68dc1a2997caca043965ce59064e446f0825db08c222b2f134f61bacc8b7b]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211122124329] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:1ad68dc1a2997caca043965ce59064e446f0825db08c222b2f134f61bacc8b7b])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:1ad68dc1a2997caca043965ce59064e446f0825db08c222b2f134f61bacc8b7b
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:1ad68dc1a2997caca043965ce59064e446f0825db08c222b2f134f61bacc8b7b
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:1ad68dc1a2997caca043965ce59064e446f0825db08c222b2f134f61bacc8b7b].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 55s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/txd6con4aax3y

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #157

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/157/display/redirect>

Changes:


------------------------------------------
[...truncated 47.93 KB...]
The push refers to repository [us.gcr.io/apache-beam-testing/java-postcommit-it/java]
9a78e3d29fe7: Preparing
f46ac1e2e1f8: Preparing
ff58424771b8: Preparing
9137aacd975c: Preparing
28ece229f307: Preparing
9cc44f341e20: Preparing
390dc3f99c8d: Preparing
5ab3e64cf269: Preparing
c06dacce3e5b: Preparing
7cabaff45161: Preparing
234f762dab19: Preparing
3c6a9037b0da: Preparing
a9e4c9343539: Preparing
47ee2d19f81a: Preparing
ab9d251e27cb: Preparing
8a5844586fdb: Preparing
a4aba4e59b40: Preparing
5499f2905579: Preparing
a36ba9e322f7: Preparing
a4aba4e59b40: Waiting
7cabaff45161: Waiting
5499f2905579: Waiting
ab9d251e27cb: Waiting
8a5844586fdb: Waiting
390dc3f99c8d: Waiting
a9e4c9343539: Waiting
9cc44f341e20: Waiting
47ee2d19f81a: Waiting
5ab3e64cf269: Waiting
234f762dab19: Waiting
3c6a9037b0da: Waiting
c06dacce3e5b: Waiting
28ece229f307: Pushed
ff58424771b8: Pushed
f46ac1e2e1f8: Pushed
9a78e3d29fe7: Pushed
9cc44f341e20: Pushed
9137aacd975c: Pushed
5ab3e64cf269: Pushed
c06dacce3e5b: Pushed
a9e4c9343539: Layer already exists
47ee2d19f81a: Layer already exists
234f762dab19: Pushed
ab9d251e27cb: Layer already exists
8a5844586fdb: Layer already exists
a4aba4e59b40: Layer already exists
5499f2905579: Layer already exists
a36ba9e322f7: Layer already exists
3c6a9037b0da: Pushed
390dc3f99c8d: Pushed
7cabaff45161: Pushed
20211121124331: digest: sha256:6ba11c2bcbbb7339217c700280f3a0997f315823d564b460174915a93cf87157 size: 4311

> Task :sdks:java:testing:load-tests:run
Nov 21, 2021 12:45:22 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Nov 21, 2021 12:45:23 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 197 files. Enable logging at DEBUG level to see which files will be staged.
Nov 21, 2021 12:45:23 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Nov 21, 2021 12:45:23 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Nov 21, 2021 12:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 197 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Nov 21, 2021 12:45:26 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 197 files cached, 0 files newly uploaded in 0 seconds
Nov 21, 2021 12:45:26 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Nov 21, 2021 12:45:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <111716 bytes, hash faa5a3598be10ac04f453c63a9557fa9e21a36ec4421cf1d408426fbc23e74f5> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline--qWjWYvhCsBPRTxjqVV_qeIaNuxEIc8dQIQm-8I-dPU.pb
Nov 21, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Nov 21, 2021 12:45:28 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@177515d1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@52ff99cd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4c2af006, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@44032fde, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b676112, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5578be42, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4e49ce2b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@136965e3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@53c6f96d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@435cc7f9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4364712f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1b7a52dd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7f93dd4e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5ad5be4a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ad85136, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@737d100a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12e5da86, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6535117e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1d1cbd0f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6fa13e6]
Nov 21, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Nov 21, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Nov 21, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Nov 21, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Nov 21, 2021 12:45:28 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3f183caa, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b66322e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@63538bb4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@24534cb0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5a50d9fc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@106d77da, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6f9c5048, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5114b7c7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@dd71b20, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@767f6ee7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b6c6e70, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@f324455, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3a894088, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@370c1968, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2f0bfe17, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3206174f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@42fb8c87, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@15eb0ae9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@65e0b505, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@67de7a99]
Nov 21, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Nov 21, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Nov 21, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Nov 21, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Nov 21, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Nov 21, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Nov 21, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Nov 21, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Nov 21, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Nov 21, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Nov 21, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Nov 21, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.36.0-SNAPSHOT
Nov 21, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-11-21_04_45_28-6576307809437182887?project=apache-beam-testing
Nov 21, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-11-21_04_45_28-6576307809437182887
Nov 21, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-11-21_04_45_28-6576307809437182887
Nov 21, 2021 12:45:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-11-21T12:45:35.559Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-11-9n5m. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Nov 21, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-21T12:45:39.551Z: Worker configuration: e2-standard-2 in us-central1-a.
Nov 21, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-21T12:45:40.122Z: Expanding SplittableParDo operations into optimizable parts.
Nov 21, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-21T12:45:40.158Z: Expanding CollectionToSingleton operations into optimizable parts.
Nov 21, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-21T12:45:40.206Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 21, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-21T12:45:40.301Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Nov 21, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-21T12:45:40.342Z: Expanding GroupByKey operations into streaming Read/Write steps
Nov 21, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-21T12:45:40.409Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 21, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-21T12:45:40.576Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 21, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-21T12:45:40.626Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Nov 21, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-21T12:45:40.694Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Nov 21, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-21T12:45:40.730Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Nov 21, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-21T12:45:40.763Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Nov 21, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-21T12:45:40.790Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Nov 21, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-21T12:45:40.855Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Nov 21, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-21T12:45:40.896Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Nov 21, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-21T12:45:40.938Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Nov 21, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-21T12:45:40.971Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Nov 21, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-21T12:45:41.005Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Nov 21, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-21T12:45:41.039Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Nov 21, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-21T12:45:41.072Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Nov 21, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-21T12:45:41.112Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Nov 21, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-21T12:45:41.144Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Nov 21, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-21T12:45:41.180Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Nov 21, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-21T12:45:41.212Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Nov 21, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-21T12:45:41.233Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Nov 21, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-21T12:45:41.269Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Nov 21, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-21T12:45:41.298Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Nov 21, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-21T12:45:41.327Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Nov 21, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-21T12:45:41.357Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Nov 21, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-21T12:45:41.378Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Nov 21, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-21T12:45:41.767Z: Starting 5 ****s in us-central1-a...
Nov 21, 2021 12:46:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-21T12:46:14.978Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Nov 21, 2021 12:46:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-21T12:46:25.307Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Nov 21, 2021 12:47:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-21T12:47:20.797Z: Workers have started successfully.
Nov 21, 2021 12:47:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-21T12:47:20.834Z: Workers have started successfully.
Nov 21, 2021 4:00:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-21T16:00:23.808Z: Cancel request is committed for workflow job: 2021-11-21_04_45_28-6576307809437182887.
Nov 21, 2021 4:00:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-21T16:00:23.890Z: Cleaning up.
Nov 21, 2021 4:00:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-21T16:00:23.958Z: Stopping **** pool...
Nov 21, 2021 4:00:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-21T16:00:24.108Z: Stopping **** pool...
Nov 21, 2021 4:02:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-21T16:02:47.865Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Nov 21, 2021 4:02:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-21T16:02:47.900Z: Worker pool stopped.
Nov 21, 2021 4:02:54 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-11-21_04_45_28-6576307809437182887 finished with status CANCELLED.
Load test results for test (ID): b1d10156-b474-41a6-acde-37f06af6f1b5 and timestamp: 2021-11-21T12:45:23.552000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11535.212
dataflow_v2_java11_total_bytes_count              1.9542081E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211121124331
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:6ba11c2bcbbb7339217c700280f3a0997f315823d564b460174915a93cf87157
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211121124331]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:6ba11c2bcbbb7339217c700280f3a0997f315823d564b460174915a93cf87157]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211121124331] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:6ba11c2bcbbb7339217c700280f3a0997f315823d564b460174915a93cf87157])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:6ba11c2bcbbb7339217c700280f3a0997f315823d564b460174915a93cf87157
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:6ba11c2bcbbb7339217c700280f3a0997f315823d564b460174915a93cf87157
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:6ba11c2bcbbb7339217c700280f3a0997f315823d564b460174915a93cf87157].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 39s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/cpems35pgkj7i

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #156

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/156/display/redirect?page=changes>

Changes:

[Valentyn Tymofieiev] Add future back to Beam containers.

[noreply] [BEAM-13290] Handle disable_prime_runner_v2 flag (#16018)

[noreply] [BEAM-11936] Add configuration parameter for suppressing unusedvariable

[noreply] Merge pull request #15957 from [BEAM-13045][Playground] Implemented

[noreply] [BEAM-13286] RowJson should expect ReadableInstant for DATETIME values

[noreply] Merge pull request #16025 from [BEAM-13216] [Playground] Add client to

[ningkang0957] Updated dep and goldens for Interactive Beam screen diff tests

[ningkang0957] Removed Pillow from requirements of containers.

[noreply] Merge pull request #16013 from [BEAM-13281] [playground] Update favicon

[noreply] Merge pull request #15768 from [BEAM-13064] [Playground] Implement

[noreply] Added go code coverage report with codecov (#16020)


------------------------------------------
[...truncated 49.21 KB...]
a9e4c9343539: Preparing
47ee2d19f81a: Preparing
bff336f6281d: Waiting
ab9d251e27cb: Preparing
ebb277931c10: Waiting
8a5844586fdb: Preparing
447d12c5c5de: Waiting
e0550e470f35: Waiting
47ee2d19f81a: Waiting
13740d6db0ba: Waiting
a4aba4e59b40: Preparing
5499f2905579: Preparing
a36ba9e322f7: Preparing
8a5844586fdb: Waiting
a4aba4e59b40: Waiting
5499f2905579: Waiting
d48ea7248abd: Waiting
f4d6d610530e: Waiting
ab9d251e27cb: Waiting
e0b518950164: Pushed
fc4b07ecff9c: Pushed
e1e243790773: Pushed
d791c195e879: Pushed
e0550e470f35: Pushed
4e971804c8a0: Pushed
bff336f6281d: Pushed
ebb277931c10: Pushed
a9e4c9343539: Layer already exists
f4d6d610530e: Pushed
447d12c5c5de: Pushed
47ee2d19f81a: Layer already exists
ab9d251e27cb: Layer already exists
13740d6db0ba: Pushed
8a5844586fdb: Layer already exists
a4aba4e59b40: Layer already exists
5499f2905579: Layer already exists
a36ba9e322f7: Layer already exists
d48ea7248abd: Pushed
20211120124333: digest: sha256:b1c7edf15e98454213402b589ec0a12be1c66ebc962cd0e81f04a67d8d31cc42 size: 4311

> Task :sdks:java:testing:load-tests:run
Nov 20, 2021 12:45:30 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Nov 20, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 197 files. Enable logging at DEBUG level to see which files will be staged.
Nov 20, 2021 12:45:31 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Nov 20, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Nov 20, 2021 12:45:33 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 197 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Nov 20, 2021 12:45:33 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 197 files cached, 0 files newly uploaded in 0 seconds
Nov 20, 2021 12:45:34 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Nov 20, 2021 12:45:34 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <111716 bytes, hash 9a2b094cc6ba95bfbeb4a9c4e03d5eddaaf202c31a19e7ac4190b53e5be996b0> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-misJTMa6lb--tKnE4D1e3aryAsMaGeesQZC1PlvplrA.pb
Nov 20, 2021 12:45:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Nov 20, 2021 12:45:35 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@44032fde, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b676112, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5578be42, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4e49ce2b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@136965e3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@53c6f96d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@435cc7f9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4364712f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1b7a52dd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7f93dd4e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5ad5be4a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ad85136, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@737d100a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12e5da86, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6535117e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1d1cbd0f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6fa13e6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3af7d855, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77049094, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@f88bfbe]
Nov 20, 2021 12:45:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Nov 20, 2021 12:45:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Nov 20, 2021 12:45:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Nov 20, 2021 12:45:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Nov 20, 2021 12:45:35 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@f1868c9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4ba6ec50, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@642413d4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@fb2e3fd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43a09ce2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3f183caa, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b66322e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@63538bb4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@24534cb0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5a50d9fc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@106d77da, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6f9c5048, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5114b7c7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@dd71b20, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@767f6ee7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b6c6e70, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@f324455, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3a894088, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@370c1968, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2f0bfe17]
Nov 20, 2021 12:45:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Nov 20, 2021 12:45:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Nov 20, 2021 12:45:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Nov 20, 2021 12:45:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Nov 20, 2021 12:45:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Nov 20, 2021 12:45:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Nov 20, 2021 12:45:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Nov 20, 2021 12:45:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Nov 20, 2021 12:45:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Nov 20, 2021 12:45:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Nov 20, 2021 12:45:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Nov 20, 2021 12:45:35 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.36.0-SNAPSHOT
Nov 20, 2021 12:45:36 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-11-20_04_45_35-5493442597811009091?project=apache-beam-testing
Nov 20, 2021 12:45:36 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-11-20_04_45_35-5493442597811009091
Nov 20, 2021 12:45:36 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-11-20_04_45_35-5493442597811009091
Nov 20, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-11-20T12:45:43.018Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-11-ehn9. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Nov 20, 2021 12:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-20T12:45:47.053Z: Worker configuration: e2-standard-2 in us-central1-a.
Nov 20, 2021 12:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-20T12:45:47.601Z: Expanding SplittableParDo operations into optimizable parts.
Nov 20, 2021 12:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-20T12:45:47.631Z: Expanding CollectionToSingleton operations into optimizable parts.
Nov 20, 2021 12:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-20T12:45:47.699Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 20, 2021 12:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-20T12:45:47.768Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Nov 20, 2021 12:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-20T12:45:47.832Z: Expanding GroupByKey operations into streaming Read/Write steps
Nov 20, 2021 12:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-20T12:45:47.897Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 20, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-20T12:45:47.992Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 20, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-20T12:45:48.022Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Nov 20, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-20T12:45:48.051Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Nov 20, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-20T12:45:48.084Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Nov 20, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-20T12:45:48.120Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Nov 20, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-20T12:45:48.146Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Nov 20, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-20T12:45:48.167Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Nov 20, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-20T12:45:48.198Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Nov 20, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-20T12:45:48.223Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Nov 20, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-20T12:45:48.255Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Nov 20, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-20T12:45:48.282Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Nov 20, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-20T12:45:48.331Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Nov 20, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-20T12:45:48.365Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Nov 20, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-20T12:45:48.397Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Nov 20, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-20T12:45:48.432Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Nov 20, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-20T12:45:48.456Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Nov 20, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-20T12:45:48.483Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Nov 20, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-20T12:45:48.510Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Nov 20, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-20T12:45:48.545Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Nov 20, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-20T12:45:48.581Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Nov 20, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-20T12:45:48.608Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Nov 20, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-20T12:45:48.638Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Nov 20, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-20T12:45:48.678Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Nov 20, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-20T12:45:49.135Z: Starting 5 ****s in us-central1-a...
Nov 20, 2021 12:46:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-20T12:46:19.772Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Nov 20, 2021 12:46:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-20T12:46:28.633Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Nov 20, 2021 12:47:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-20T12:47:26.271Z: Workers have started successfully.
Nov 20, 2021 12:47:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-20T12:47:26.298Z: Workers have started successfully.
Nov 20, 2021 1:27:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-20T13:27:50.238Z: Staged package commons-io-2.6--HfTBGYKwqFC84ZbrfyXHex-1zx0fH-NXS9ROcpzZRM.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/commons-io-2.6--HfTBGYKwqFC84ZbrfyXHex-1zx0fH-NXS9ROcpzZRM.jar' is inaccessible.
Nov 20, 2021 1:27:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-20T13:27:50.316Z: Staged package commons-lang-2.6-UPEbCfh3wpTVbyRGP0fSj5Kc9QRPZIZhwPDPuumi9Jw.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/commons-lang-2.6-UPEbCfh3wpTVbyRGP0fSj5Kc9QRPZIZhwPDPuumi9Jw.jar' is inaccessible.
Nov 20, 2021 1:27:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-20T13:27:51.177Z: Staged package google-cloud-pubsublite-1.2.0-5U3FD5IEkQizHwu39BBM5AgOV1a73rwGWbaivmdyeSI.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-cloud-pubsublite-1.2.0-5U3FD5IEkQizHwu39BBM5AgOV1a73rwGWbaivmdyeSI.jar' is inaccessible.
Nov 20, 2021 1:27:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-20T13:27:51.622Z: Staged package grpc-google-cloud-pubsublite-v1-1.2.0-wV0rl2IQjaeB0Zqm6xgMut8w_3REeRHMW8XtX4sPX64.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/grpc-google-cloud-pubsublite-v1-1.2.0-wV0rl2IQjaeB0Zqm6xgMut8w_3REeRHMW8XtX4sPX64.jar' is inaccessible.
Nov 20, 2021 1:27:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-20T13:27:52.003Z: Staged package ion-java-1.0.2-DRJ7IFofzgq8KjdXoEF0hlG8ZsFc9MBZusWDOyfUcaU.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/ion-java-1.0.2-DRJ7IFofzgq8KjdXoEF0hlG8ZsFc9MBZusWDOyfUcaU.jar' is inaccessible.
Nov 20, 2021 1:27:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-20T13:27:52.901Z: Staged package proto-google-cloud-pubsublite-v1-1.2.0-vWj_eMbE5U3cmHa7Ax4j7qYSkZojafmPkGvafYKuTKs.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/proto-google-cloud-pubsublite-v1-1.2.0-vWj_eMbE5U3cmHa7Ax4j7qYSkZojafmPkGvafYKuTKs.jar' is inaccessible.
Nov 20, 2021 1:27:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-11-20T13:27:53.227Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Nov 20, 2021 4:00:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-20T16:00:25.676Z: Cancel request is committed for workflow job: 2021-11-20_04_45_35-5493442597811009091.
Nov 20, 2021 4:00:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-20T16:00:25.716Z: Cleaning up.
Nov 20, 2021 4:00:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-20T16:00:25.805Z: Stopping **** pool...
Nov 20, 2021 4:00:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-20T16:00:25.847Z: Stopping **** pool...
Nov 20, 2021 4:02:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-20T16:02:51.680Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Nov 20, 2021 4:02:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-20T16:02:51.711Z: Worker pool stopped.
Nov 20, 2021 4:02:57 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-11-20_04_45_35-5493442597811009091 finished with status CANCELLED.
Load test results for test (ID): 541f0a0a-6dd7-4376-bc9c-7f05791a3430 and timestamp: 2021-11-20T12:45:31.177000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                  11549.51
dataflow_v2_java11_total_bytes_count             2.00664536E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211120124333
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:b1c7edf15e98454213402b589ec0a12be1c66ebc962cd0e81f04a67d8d31cc42
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211120124333]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:b1c7edf15e98454213402b589ec0a12be1c66ebc962cd0e81f04a67d8d31cc42]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211120124333] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:b1c7edf15e98454213402b589ec0a12be1c66ebc962cd0e81f04a67d8d31cc42])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:b1c7edf15e98454213402b589ec0a12be1c66ebc962cd0e81f04a67d8d31cc42
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:b1c7edf15e98454213402b589ec0a12be1c66ebc962cd0e81f04a67d8d31cc42
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:b1c7edf15e98454213402b589ec0a12be1c66ebc962cd0e81f04a67d8d31cc42].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 41s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/2kh25e3ken3vw

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #155

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/155/display/redirect?page=changes>

Changes:

[Robert Bradshaw] Allow wildcards for java class lookup transform providers.

[Robert Bradshaw] Disallow getClass().forname().

[Robert Bradshaw] Docs, globbing, and a test.

[Robert Bradshaw] Defer expansion service creation for easier testing.

[Robert Bradshaw] Allow java jar service with classpath.

[avilovpavel6] Support GO sdk

[dpcollins] Fix race condition in Pub/Sub Lite SDF that causes it to error out when

[dpcollins] edit

[noreply] Merge pull request #15785 from Support for passing priority parameter in

[dpcollins] Restructure processor to access beam types from downcall thread

[dpcollins] Restructure processor to access beam types from downcall thread

[noreply] [BEAM-13283] Skip apache beam dependency installation from

[dpcollins] Restructure processor to access beam types from downcall thread

[noreply] Merge pull request #15956 from [BEAM-13210] [Playground]: support

[noreply] created quickstart guide for multi-language pipelines (Python) (#16001)

[dpcollins] use interface in logger to work around likely SLF4J bug

[dpcollins] revert logger class

[noreply] Revert "Allow wildcards for java class lookup transform providers."

[Valentyn Tymofieiev] Moving to 2.36.0-SNAPSHOT on master branch.


------------------------------------------
[...truncated 49.74 KB...]
6f42bffc4348: Pushed
5bbd48f7b5d8: Pushed
20211119124334: digest: sha256:3622d6a0a35337e17b60c710b6399c61252064c92dbfd6a314f20d7a58007ec5 size: 4311

> Task :sdks:java:testing:load-tests:run
Nov 19, 2021 12:45:32 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Nov 19, 2021 12:45:32 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 197 files. Enable logging at DEBUG level to see which files will be staged.
Nov 19, 2021 12:45:33 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Nov 19, 2021 12:45:33 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Nov 19, 2021 12:45:35 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 197 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Nov 19, 2021 12:45:36 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 197 files cached, 0 files newly uploaded in 0 seconds
Nov 19, 2021 12:45:36 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Nov 19, 2021 12:45:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <111716 bytes, hash 99ce5648231c57b1ef5566d8fdd091d391f22ebfc24bcd0ba953c1475ea5180c> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-mc5WSCMcV7HvVWbY_dCR05HyLr_CS80LqVPBR16lGAw.pb
Nov 19, 2021 12:45:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Nov 19, 2021 12:45:37 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@44032fde, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b676112, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5578be42, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4e49ce2b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@136965e3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@53c6f96d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@435cc7f9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4364712f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1b7a52dd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7f93dd4e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5ad5be4a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ad85136, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@737d100a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12e5da86, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6535117e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1d1cbd0f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6fa13e6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3af7d855, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77049094, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@f88bfbe]
Nov 19, 2021 12:45:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Nov 19, 2021 12:45:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Nov 19, 2021 12:45:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Nov 19, 2021 12:45:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Nov 19, 2021 12:45:37 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@f1868c9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4ba6ec50, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@642413d4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@fb2e3fd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43a09ce2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3f183caa, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b66322e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@63538bb4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@24534cb0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5a50d9fc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@106d77da, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6f9c5048, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5114b7c7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@dd71b20, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@767f6ee7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b6c6e70, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@f324455, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3a894088, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@370c1968, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2f0bfe17]
Nov 19, 2021 12:45:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Nov 19, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Nov 19, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Nov 19, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Nov 19, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Nov 19, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Nov 19, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Nov 19, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Nov 19, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Nov 19, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Nov 19, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Nov 19, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.36.0-SNAPSHOT
Nov 19, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-11-19_04_45_38-7329796497055665608?project=apache-beam-testing
Nov 19, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-11-19_04_45_38-7329796497055665608
Nov 19, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-11-19_04_45_38-7329796497055665608
Nov 19, 2021 12:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-11-19T12:45:45.477Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-11-84ub. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Nov 19, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-19T12:45:49.297Z: Worker configuration: e2-standard-2 in us-central1-a.
Nov 19, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-19T12:45:49.961Z: Expanding SplittableParDo operations into optimizable parts.
Nov 19, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-19T12:45:49.997Z: Expanding CollectionToSingleton operations into optimizable parts.
Nov 19, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-19T12:45:50.061Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 19, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-19T12:45:50.131Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Nov 19, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-19T12:45:50.167Z: Expanding GroupByKey operations into streaming Read/Write steps
Nov 19, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-19T12:45:50.216Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 19, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-19T12:45:50.338Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 19, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-19T12:45:50.374Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Nov 19, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-19T12:45:50.410Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Nov 19, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-19T12:45:50.431Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Nov 19, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-19T12:45:50.464Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Nov 19, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-19T12:45:50.495Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Nov 19, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-19T12:45:50.524Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Nov 19, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-19T12:45:50.574Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Nov 19, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-19T12:45:50.609Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Nov 19, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-19T12:45:50.637Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Nov 19, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-19T12:45:50.662Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Nov 19, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-19T12:45:50.685Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Nov 19, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-19T12:45:50.717Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Nov 19, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-19T12:45:50.752Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Nov 19, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-19T12:45:50.786Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Nov 19, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-19T12:45:50.826Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Nov 19, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-19T12:45:50.852Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Nov 19, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-19T12:45:50.875Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Nov 19, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-19T12:45:50.900Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Nov 19, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-19T12:45:50.933Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Nov 19, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-19T12:45:50.965Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Nov 19, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-19T12:45:50.998Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Nov 19, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-19T12:45:51.025Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Nov 19, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-19T12:45:51.408Z: Starting 5 ****s in us-central1-a...
Nov 19, 2021 12:46:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-19T12:46:12.813Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Nov 19, 2021 12:46:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-19T12:46:41.339Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Nov 19, 2021 12:47:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-19T12:47:37.392Z: Workers have started successfully.
Nov 19, 2021 12:47:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-19T12:47:37.428Z: Workers have started successfully.
Nov 19, 2021 1:26:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-19T13:26:14.502Z: Workers have started successfully.
Nov 19, 2021 1:26:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-19T13:26:17.159Z: Worker configuration: e2-standard-2 in us-central1-a.
Nov 19, 2021 1:26:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-19T13:26:29.042Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Nov 19, 2021 1:50:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-19T13:50:15.707Z: Staged package jackson-dataformat-cbor-2.13.0-87i8Y8vQiEl2BVWwSbyVD2k9G-KE9nB4qPOXblapNm4.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/jackson-dataformat-cbor-2.13.0-87i8Y8vQiEl2BVWwSbyVD2k9G-KE9nB4qPOXblapNm4.jar' is inaccessible.
Nov 19, 2021 1:50:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-11-19T13:50:16.953Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Nov 19, 2021 1:53:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-11-19T13:53:16.977Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Nov 19, 2021 1:56:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-19T13:56:15.747Z: Staged package jackson-dataformat-cbor-2.13.0-87i8Y8vQiEl2BVWwSbyVD2k9G-KE9nB4qPOXblapNm4.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/jackson-dataformat-cbor-2.13.0-87i8Y8vQiEl2BVWwSbyVD2k9G-KE9nB4qPOXblapNm4.jar' is inaccessible.
Nov 19, 2021 1:56:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-11-19T13:56:16.928Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Nov 19, 2021 1:59:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-11-19T13:59:16.822Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Nov 19, 2021 2:02:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-19T14:02:15.633Z: Staged package jackson-dataformat-cbor-2.13.0-87i8Y8vQiEl2BVWwSbyVD2k9G-KE9nB4qPOXblapNm4.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/jackson-dataformat-cbor-2.13.0-87i8Y8vQiEl2BVWwSbyVD2k9G-KE9nB4qPOXblapNm4.jar' is inaccessible.
Nov 19, 2021 2:02:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-11-19T14:02:16.913Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Nov 19, 2021 4:00:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-19T16:00:28.295Z: Cancel request is committed for workflow job: 2021-11-19_04_45_38-7329796497055665608.
Nov 19, 2021 4:00:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-19T16:00:28.335Z: Cleaning up.
Nov 19, 2021 4:00:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-19T16:00:28.418Z: Stopping **** pool...
Nov 19, 2021 4:00:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-19T16:00:28.459Z: Stopping **** pool...
Nov 19, 2021 4:02:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-19T16:02:45.252Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Nov 19, 2021 4:02:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-19T16:02:45.297Z: Worker pool stopped.
Nov 19, 2021 4:02:51 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-11-19_04_45_38-7329796497055665608 finished with status CANCELLED.
Load test results for test (ID): 4977a7da-4f55-4b89-aa18-5706c0a097db and timestamp: 2021-11-19T12:45:33.169000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11547.423
dataflow_v2_java11_total_bytes_count             2.01742936E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211119124334
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:3622d6a0a35337e17b60c710b6399c61252064c92dbfd6a314f20d7a58007ec5
Deleted: sha256:9eba0e909d2778ca8902010a04e440d732c4e577bb7fee310d52cefbc3371ad1
Deleted: sha256:6184680e7c1bbeab065fef7205c369d6f8267cbf741d86f9a2838b8765eee4e2
Deleted: sha256:9136db020dfe9aabe120d128dcbe9270b9de15c8c032133e3539499c7b9d671d
Deleted: sha256:f438aecdd2b73b67b8ac947210fb96612cc225eddc63763f7b3efad3c17e90ef
Deleted: sha256:f621043c203040adfc270951fa3cdd565def58e9891641539a2e865069453e36
Deleted: sha256:4b217410e9e2e8acebdb0c52161a9c8e74633943b406881df3de116d630a51db
Deleted: sha256:92a500d22d75367bf71c506f8a60944c8f70238da985dffb796231b63cae0bd1
Deleted: sha256:4c67424fae010648a2ac37b1b1ca0d759aced3a4b4f362c8ccb8727078fa0b8b
Deleted: sha256:4e9f9fc565c5882f886e4b0a3740a5c81478c6bbeffa8058f2fcbc06aae2576b
Deleted: sha256:7bbf4f5f1c99fba93120211dd2a26f2bb83390b2707f4210fc3678cd9fc2e81f
Deleted: sha256:01857c97d7cca3fb3cddad10543f58616c56a67b76d2a1a3c060b545bedef08c
Deleted: sha256:46fd7078d43f5abd46766eb9c04448182adbbe467ff222eb7417fabad08f76b3
Deleted: sha256:ae0bfb9f4eea4d5235c5f04ad9ca1b7c5ef2a5959d2b26284a98fad0d28a8209
Deleted: sha256:519bd39f89fa67eeba8446cb6bf4e6155d59c1dfe927cb373a714a756f845524
Deleted: sha256:7fe938f4b906d49e787396893bc6b510bb21e27e9dc9bc5178011a1458bc4e44
Deleted: sha256:c72b5f728f75805db6141ec5162f8c17f1227a7435e37ea71f7afa8312f07504
Deleted: sha256:83b1941d715acb35438eeee1334a682467957c7cf2039a8cca57b346e4b8a4d7
Deleted: sha256:42f51c0fd2822d2ec751ace46ef51c02698afe7144aad020f63f594d6d905afa
Deleted: sha256:fb02fe4285b23eb883cff62066edb6317f1d602b438a1415a19f50665bd2bd24
Deleted: sha256:6bb9732dee216db39e782562334d9d2a27eb9dd86adbcbc2ef5b8296146b6b91
Deleted: sha256:99ebc514fddeb1858e94dec53462cd579f7f31a8916d422a3ee20b962bf5a825
Deleted: sha256:5c3aad92c32eddc2e3fd4355c979d5fcfac93b2fca1197601a11b9150f0b8d59
Deleted: sha256:d3434bc12b697c4662b1336ea3dd4eeef352552642913d9be77ea4fe8422e0ea
Deleted: sha256:7686b81c5d31ff427c422fd250d62add369b440bcb52290304674edef9f1341d
Deleted: sha256:5aaf41ccbc6147b89dfc706c5c74fb02fa90dc5a7903222d8ed34f951dfa48de
Deleted: sha256:33922e9da48d23192f84f423579d621249405f8aebde02286f672ee06be736bc
Deleted: sha256:ff36df0089a9165d4f3b7fc2ea3bedf3eb2174114677fb3a896a30e29fc158a2
Deleted: sha256:d08f1e7e0327fed5fa00d03c6ac5b29d7feb147610b802167d9cbd5c3fa9b447
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211119124334]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:3622d6a0a35337e17b60c710b6399c61252064c92dbfd6a314f20d7a58007ec5]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211119124334] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:3622d6a0a35337e17b60c710b6399c61252064c92dbfd6a314f20d7a58007ec5])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:3622d6a0a35337e17b60c710b6399c61252064c92dbfd6a314f20d7a58007ec5
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:3622d6a0a35337e17b60c710b6399c61252064c92dbfd6a314f20d7a58007ec5
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:3622d6a0a35337e17b60c710b6399c61252064c92dbfd6a314f20d7a58007ec5].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 35s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/i4bgq2wt5g2om

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #154

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/154/display/redirect?page=changes>

Changes:

[samuelw] [BEAM-12818] Write windowed file output to a consistent subdirectory

[samuelw] [BEAM-13268] Change autosharding big query inserts to happen in parallel

[noreply] [BEAM-3293] Add check for error on NewIterable call (#16006)

[Brian Hulette] remove pkg_resources in run_generate_requirements.sh

[Brian Hulette] Run generatePythonRequirementsAll for pyarrow 6

[noreply] Update dev container tags used by Dataflow runner with unreleased SDKs.

[noreply] [BEAM-13265] Add withDeterministicRecordIdFn which allows for

[noreply] [BEAM-13183] [BEAM-8152] Use venv instead of virtualenv to create Python


------------------------------------------
[...truncated 46.38 KB...]
5fb5d1edcd09: Preparing
68dd8de761c9: Preparing
d1c7cb61286c: Preparing
97e67893cc15: Preparing
deeda51a2c9f: Preparing
a9e4c9343539: Preparing
47ee2d19f81a: Preparing
37d28c2bde63: Waiting
ab9d251e27cb: Preparing
8a5844586fdb: Preparing
efff75d61a0b: Waiting
a4aba4e59b40: Preparing
5499f2905579: Preparing
a36ba9e322f7: Preparing
97e67893cc15: Waiting
47ee2d19f81a: Waiting
68dd8de761c9: Waiting
a9e4c9343539: Waiting
deeda51a2c9f: Waiting
ab9d251e27cb: Waiting
d1c7cb61286c: Waiting
5499f2905579: Waiting
a36ba9e322f7: Waiting
9c7007459e3b: Pushed
56b3f2b67eef: Pushed
295cc996b349: Pushed
9d76ec276e57: Pushed
285c92fa1e72: Pushed
37d28c2bde63: Pushed
5fb5d1edcd09: Pushed
68dd8de761c9: Pushed
97e67893cc15: Pushed
a9e4c9343539: Layer already exists
47ee2d19f81a: Layer already exists
ab9d251e27cb: Layer already exists
8a5844586fdb: Layer already exists
efff75d61a0b: Pushed
deeda51a2c9f: Pushed
a4aba4e59b40: Layer already exists
5499f2905579: Layer already exists
a36ba9e322f7: Layer already exists
d1c7cb61286c: Pushed
20211118124333: digest: sha256:f777e18c24a32f8dc9446351c4388a50f2a134fb007548bb8761832699bb991c size: 4311

> Task :sdks:java:testing:load-tests:run
Nov 18, 2021 12:45:33 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Nov 18, 2021 12:45:33 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 197 files. Enable logging at DEBUG level to see which files will be staged.
Nov 18, 2021 12:45:34 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Nov 18, 2021 12:45:34 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Nov 18, 2021 12:45:36 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 197 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Nov 18, 2021 12:45:37 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 197 files cached, 0 files newly uploaded in 0 seconds
Nov 18, 2021 12:45:37 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Nov 18, 2021 12:45:37 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <111716 bytes, hash e016a3666619deb9150c16f372a0c2a7d0d41898f5300488dad7c2c2f58164b6> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-4BajZmYZ3rkVDBbzcqDCp9DUGJj1MASI2tfCwvWBZLY.pb
Nov 18, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Nov 18, 2021 12:45:38 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@177515d1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@52ff99cd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4c2af006, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@44032fde, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b676112, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5578be42, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4e49ce2b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@136965e3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@53c6f96d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@435cc7f9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4364712f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1b7a52dd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7f93dd4e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5ad5be4a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ad85136, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@737d100a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12e5da86, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6535117e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1d1cbd0f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6fa13e6]
Nov 18, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Nov 18, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Nov 18, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Nov 18, 2021 12:45:39 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Nov 18, 2021 12:45:39 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3f183caa, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b66322e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@63538bb4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@24534cb0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5a50d9fc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@106d77da, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6f9c5048, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5114b7c7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@dd71b20, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@767f6ee7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b6c6e70, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@f324455, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3a894088, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@370c1968, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2f0bfe17, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3206174f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@42fb8c87, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@15eb0ae9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@65e0b505, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@67de7a99]
Nov 18, 2021 12:45:39 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Nov 18, 2021 12:45:39 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Nov 18, 2021 12:45:39 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Nov 18, 2021 12:45:39 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Nov 18, 2021 12:45:39 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Nov 18, 2021 12:45:39 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Nov 18, 2021 12:45:39 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Nov 18, 2021 12:45:39 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Nov 18, 2021 12:45:39 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Nov 18, 2021 12:45:39 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Nov 18, 2021 12:45:39 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Nov 18, 2021 12:45:39 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.35.0-SNAPSHOT
Nov 18, 2021 12:45:39 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-11-18_04_45_39-10476051663685778151?project=apache-beam-testing
Nov 18, 2021 12:45:39 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-11-18_04_45_39-10476051663685778151
Nov 18, 2021 12:45:39 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-11-18_04_45_39-10476051663685778151
Nov 18, 2021 12:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-11-18T12:45:46.528Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-11-wypa. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Nov 18, 2021 12:45:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-18T12:45:55.152Z: Worker configuration: e2-standard-2 in us-central1-a.
Nov 18, 2021 12:45:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-18T12:45:55.979Z: Expanding SplittableParDo operations into optimizable parts.
Nov 18, 2021 12:45:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-18T12:45:56.016Z: Expanding CollectionToSingleton operations into optimizable parts.
Nov 18, 2021 12:45:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-18T12:45:56.100Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 18, 2021 12:45:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-18T12:45:56.187Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Nov 18, 2021 12:45:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-18T12:45:56.231Z: Expanding GroupByKey operations into streaming Read/Write steps
Nov 18, 2021 12:45:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-18T12:45:56.317Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 18, 2021 12:45:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-18T12:45:56.446Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 18, 2021 12:45:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-18T12:45:56.490Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Nov 18, 2021 12:45:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-18T12:45:56.537Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Nov 18, 2021 12:45:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-18T12:45:56.571Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Nov 18, 2021 12:45:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-18T12:45:56.610Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Nov 18, 2021 12:45:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-18T12:45:56.656Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Nov 18, 2021 12:45:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-18T12:45:56.705Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Nov 18, 2021 12:45:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-18T12:45:56.730Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Nov 18, 2021 12:45:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-18T12:45:56.785Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Nov 18, 2021 12:45:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-18T12:45:56.813Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Nov 18, 2021 12:45:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-18T12:45:56.856Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Nov 18, 2021 12:45:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-18T12:45:56.907Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Nov 18, 2021 12:45:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-18T12:45:56.955Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Nov 18, 2021 12:45:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-18T12:45:56.991Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Nov 18, 2021 12:45:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-18T12:45:57.040Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Nov 18, 2021 12:45:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-18T12:45:57.113Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Nov 18, 2021 12:45:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-18T12:45:57.146Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Nov 18, 2021 12:45:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-18T12:45:57.185Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Nov 18, 2021 12:45:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-18T12:45:57.226Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Nov 18, 2021 12:45:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-18T12:45:57.287Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Nov 18, 2021 12:45:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-18T12:45:57.318Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Nov 18, 2021 12:45:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-18T12:45:57.350Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Nov 18, 2021 12:45:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-18T12:45:57.395Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Nov 18, 2021 12:45:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-18T12:45:57.878Z: Starting 5 ****s in us-central1-a...
Nov 18, 2021 12:46:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-18T12:46:12.418Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Nov 18, 2021 12:46:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-18T12:46:40.907Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Nov 18, 2021 12:47:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-18T12:47:37.313Z: Workers have started successfully.
Nov 18, 2021 12:47:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-18T12:47:37.369Z: Workers have started successfully.
Nov 18, 2021 3:54:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-18T15:54:58.410Z: Staged package beam-model-job-management-2.35.0-SNAPSHOT-8QB2g9ssFGaKGT56IbOMrjcQcE9XP-LgiRhvff3RHeg.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.35.0-SNAPSHOT-8QB2g9ssFGaKGT56IbOMrjcQcE9XP-LgiRhvff3RHeg.jar' is inaccessible.
Nov 18, 2021 3:54:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-18T15:54:58.694Z: Staged package beam-sdks-java-extensions-arrow-2.35.0-SNAPSHOT--6o0gUFJinkN4upVAkTzRdayVoYWlnjhjcsg2E3us1A.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-arrow-2.35.0-SNAPSHOT--6o0gUFJinkN4upVAkTzRdayVoYWlnjhjcsg2E3us1A.jar' is inaccessible.
Nov 18, 2021 3:55:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-18T15:54:58.904Z: Staged package beam-sdks-java-load-tests-2.35.0-SNAPSHOT-AWXw3244IpmjjLbDXVk6FhB1JxiEeTDrXIDH5IPOx9U.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-load-tests-2.35.0-SNAPSHOT-AWXw3244IpmjjLbDXVk6FhB1JxiEeTDrXIDH5IPOx9U.jar' is inaccessible.
Nov 18, 2021 3:55:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-11-18T15:55:02.326Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Nov 18, 2021 3:58:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-11-18T15:58:02.900Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Nov 18, 2021 4:00:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-18T16:00:35.930Z: Cancel request is committed for workflow job: 2021-11-18_04_45_39-10476051663685778151.
Nov 18, 2021 4:00:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-18T16:00:35.958Z: Cleaning up.
Nov 18, 2021 4:00:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-18T16:00:36.032Z: Stopping **** pool...
Nov 18, 2021 4:00:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-18T16:00:36.086Z: Stopping **** pool...
Nov 18, 2021 4:02:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-18T16:02:56.856Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Nov 18, 2021 4:02:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-18T16:02:56.893Z: Worker pool stopped.
Nov 18, 2021 4:03:02 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-11-18_04_45_39-10476051663685778151 finished with status CANCELLED.
Load test results for test (ID): 28fa5b47-7660-4b8f-8ef1-056619b8a707 and timestamp: 2021-11-18T12:45:33.790000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                  11547.62
dataflow_v2_java11_total_bytes_count             2.07161488E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211118124333
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:f777e18c24a32f8dc9446351c4388a50f2a134fb007548bb8761832699bb991c
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211118124333]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:f777e18c24a32f8dc9446351c4388a50f2a134fb007548bb8761832699bb991c]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211118124333] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:f777e18c24a32f8dc9446351c4388a50f2a134fb007548bb8761832699bb991c])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:f777e18c24a32f8dc9446351c4388a50f2a134fb007548bb8761832699bb991c
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:f777e18c24a32f8dc9446351c4388a50f2a134fb007548bb8761832699bb991c
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:f777e18c24a32f8dc9446351c4388a50f2a134fb007548bb8761832699bb991c].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 46s
101 actionable tasks: 73 executed, 26 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/4fzlmo7wxjdtu

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #153

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/153/display/redirect?page=changes>

Changes:

[aydar.zaynutdinov] [BEAM-13178][Playground]

[Pablo Estrada] [BEAM-8033] Throwing an error if a dataSourceProviderFn is defined twice

[minherz] fix: update dlp dependency version range

[noreply] [BEAM-3293] Finish E2E implementation of MultiMap side inputs, add

[noreply] Merge pull request #15740 from [BEAM-12936][Playground] Code editor -

[noreply] Merge pull request #15976 from [Playground][BEAM-12941][Bugfix] Fix

[noreply] Merge pull request #15991 [BEAM-13251][Playground] [Bugfix] Lint Fails

[noreply] Merge pull request #15990 from [BEAM-13177][Playground] Change using of

[noreply] [BEAM-13262] Forward for metrics.SingleResult (#15993)

[noreply] Generate Python container dependencies in an automated way. (#15927)

[noreply] [BEAM-13264] Allow pyarrow up to 6.x (#15995)

[noreply] Update the go get command with v2 (#15945)

[noreply] Go Quickstart: switch to Spark 3 JobServer for better out-of-box

[noreply] Golint fixes for recent Go SDK import (#15999)

[noreply] Merge pull request #15997: [BEAM-8688] Upgrading gcsio library to latest


------------------------------------------
[...truncated 47.50 KB...]
7644e3242d71: Preparing
030d38dc9a3a: Preparing
7d9c85bf174e: Preparing
f3f68f0c3ef9: Preparing
9e10b8ced0e8: Preparing
e2c2d94ac4e7: Preparing
796fe27cf4f1: Preparing
5256a55da86b: Preparing
59c563b0823b: Preparing
42a75344940e: Preparing
5cf507d14524: Preparing
34d0a025f540: Preparing
a9e4c9343539: Preparing
47ee2d19f81a: Preparing
ab9d251e27cb: Preparing
8a5844586fdb: Preparing
a4aba4e59b40: Preparing
5499f2905579: Preparing
a36ba9e322f7: Preparing
59c563b0823b: Waiting
42a75344940e: Waiting
5cf507d14524: Waiting
a4aba4e59b40: Waiting
8a5844586fdb: Waiting
34d0a025f540: Waiting
5499f2905579: Waiting
e2c2d94ac4e7: Waiting
796fe27cf4f1: Waiting
a9e4c9343539: Waiting
a36ba9e322f7: Waiting
47ee2d19f81a: Waiting
5256a55da86b: Waiting
ab9d251e27cb: Waiting
9e10b8ced0e8: Pushed
030d38dc9a3a: Pushed
7d9c85bf174e: Pushed
f3f68f0c3ef9: Pushed
e2c2d94ac4e7: Pushed
5256a55da86b: Pushed
7644e3242d71: Pushed
59c563b0823b: Pushed
a9e4c9343539: Layer already exists
47ee2d19f81a: Layer already exists
ab9d251e27cb: Layer already exists
8a5844586fdb: Layer already exists
796fe27cf4f1: Pushed
a4aba4e59b40: Layer already exists
5499f2905579: Layer already exists
a36ba9e322f7: Layer already exists
5cf507d14524: Pushed
34d0a025f540: Pushed
42a75344940e: Pushed
20211117124333: digest: sha256:09aa9ff628f29be3fba2566fb2621496a95b5101f95c9256289a61bc609488db size: 4311

> Task :sdks:java:testing:load-tests:run
Nov 17, 2021 12:45:29 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Nov 17, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 197 files. Enable logging at DEBUG level to see which files will be staged.
Nov 17, 2021 12:45:30 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Nov 17, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Nov 17, 2021 12:45:32 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 197 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Nov 17, 2021 12:45:33 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 197 files cached, 0 files newly uploaded in 0 seconds
Nov 17, 2021 12:45:33 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Nov 17, 2021 12:45:33 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <111716 bytes, hash cdf42de559fdb8dde31baddcb183f8d83de2ba73962ee3a8ceb6ca3d2846e1cb> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-zfQt5Vn9uN3jG63csYP42D3iunOWLuOozrbKPShG4cs.pb
Nov 17, 2021 12:45:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Nov 17, 2021 12:45:35 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@52ff99cd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4c2af006, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@44032fde, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b676112, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5578be42, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4e49ce2b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@136965e3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@53c6f96d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@435cc7f9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4364712f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1b7a52dd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7f93dd4e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5ad5be4a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ad85136, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@737d100a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12e5da86, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6535117e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1d1cbd0f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6fa13e6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3af7d855]
Nov 17, 2021 12:45:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Nov 17, 2021 12:45:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Nov 17, 2021 12:45:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Nov 17, 2021 12:45:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Nov 17, 2021 12:45:35 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@b34832b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@48f4713c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@f1868c9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4ba6ec50, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@642413d4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@fb2e3fd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43a09ce2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3f183caa, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b66322e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@63538bb4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@24534cb0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5a50d9fc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@106d77da, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6f9c5048, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5114b7c7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@dd71b20, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@767f6ee7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b6c6e70, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@f324455, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3a894088]
Nov 17, 2021 12:45:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Nov 17, 2021 12:45:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Nov 17, 2021 12:45:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Nov 17, 2021 12:45:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Nov 17, 2021 12:45:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Nov 17, 2021 12:45:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Nov 17, 2021 12:45:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Nov 17, 2021 12:45:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Nov 17, 2021 12:45:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Nov 17, 2021 12:45:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Nov 17, 2021 12:45:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Nov 17, 2021 12:45:35 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.35.0-SNAPSHOT
Nov 17, 2021 12:45:36 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-11-17_04_45_35-8958723133952108707?project=apache-beam-testing
Nov 17, 2021 12:45:36 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-11-17_04_45_35-8958723133952108707
Nov 17, 2021 12:45:36 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-11-17_04_45_35-8958723133952108707
Nov 17, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-11-17T12:45:42.148Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-11-26at. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Nov 17, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-17T12:45:46.754Z: Worker configuration: e2-standard-2 in us-central1-a.
Nov 17, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-17T12:45:47.374Z: Expanding SplittableParDo operations into optimizable parts.
Nov 17, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-17T12:45:47.427Z: Expanding CollectionToSingleton operations into optimizable parts.
Nov 17, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-17T12:45:47.497Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 17, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-17T12:45:47.569Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Nov 17, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-17T12:45:47.604Z: Expanding GroupByKey operations into streaming Read/Write steps
Nov 17, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-17T12:45:47.685Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 17, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-17T12:45:47.796Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 17, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-17T12:45:47.837Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Nov 17, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-17T12:45:47.878Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Nov 17, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-17T12:45:47.907Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Nov 17, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-17T12:45:47.936Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Nov 17, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-17T12:45:47.997Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Nov 17, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-17T12:45:48.065Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Nov 17, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-17T12:45:48.101Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Nov 17, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-17T12:45:48.131Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Nov 17, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-17T12:45:48.178Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Nov 17, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-17T12:45:48.212Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Nov 17, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-17T12:45:48.251Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Nov 17, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-17T12:45:48.286Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Nov 17, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-17T12:45:48.320Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Nov 17, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-17T12:45:48.358Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Nov 17, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-17T12:45:48.389Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Nov 17, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-17T12:45:48.424Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Nov 17, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-17T12:45:48.460Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Nov 17, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-17T12:45:48.488Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Nov 17, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-17T12:45:48.521Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Nov 17, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-17T12:45:48.558Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Nov 17, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-17T12:45:48.602Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Nov 17, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-17T12:45:48.641Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Nov 17, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-17T12:45:48.989Z: Starting 5 ****s in us-central1-a...
Nov 17, 2021 12:46:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-17T12:46:07.330Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Nov 17, 2021 12:46:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-17T12:46:32.928Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Nov 17, 2021 12:47:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-17T12:47:26.593Z: Workers have started successfully.
Nov 17, 2021 12:47:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-17T12:47:26.626Z: Workers have started successfully.
Nov 17, 2021 4:00:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-17T16:00:24.862Z: Cancel request is committed for workflow job: 2021-11-17_04_45_35-8958723133952108707.
Nov 17, 2021 4:00:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-17T16:00:25.174Z: Cleaning up.
Nov 17, 2021 4:00:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-17T16:00:25.265Z: Stopping **** pool...
Nov 17, 2021 4:00:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-17T16:00:25.313Z: Stopping **** pool...
Nov 17, 2021 4:02:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-17T16:02:54.477Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Nov 17, 2021 4:02:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-17T16:02:54.518Z: Worker pool stopped.
Nov 17, 2021 4:03:01 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-11-17_04_45_35-8958723133952108707 finished with status CANCELLED.
Load test results for test (ID): d4d2d075-5db1-4fe5-88fa-45fb70c09531 and timestamp: 2021-11-17T12:45:30.544000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11544.927
dataflow_v2_java11_total_bytes_count             2.24061703E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211117124333
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:09aa9ff628f29be3fba2566fb2621496a95b5101f95c9256289a61bc609488db
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211117124333]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:09aa9ff628f29be3fba2566fb2621496a95b5101f95c9256289a61bc609488db]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211117124333] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:09aa9ff628f29be3fba2566fb2621496a95b5101f95c9256289a61bc609488db])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:09aa9ff628f29be3fba2566fb2621496a95b5101f95c9256289a61bc609488db
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:09aa9ff628f29be3fba2566fb2621496a95b5101f95c9256289a61bc609488db
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:09aa9ff628f29be3fba2566fb2621496a95b5101f95c9256289a61bc609488db].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 45s
101 actionable tasks: 71 executed, 28 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/wdpql4z5cgxcc

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #152

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/152/display/redirect?page=changes>

Changes:

[brachipa] [BEAM-12737] error handling for sql calc error

[brachipa] [BEAM-12737] error handling for sql calc error

[brachipa] [BEAM-12737] error handling for sql calc error

[brachipa] [BEAM-12737] add test

[brachipa] [BEAM-12737] fix build

[brachipa] [BEAM-12737] fix build

[brachipa] revert

[brachipa] [beam-12737] cr fix

[brachipa] [beam-12737] cr fixes

[brachipa] [beam-12737] send errorTransformer to `buildPTransform`

[brachipa] [beam-12737] send errorTransformer to `buildPTransform`

[brachipa] [beam-12737] fix type `? extends POutput`

[Robert Bradshaw] Fully resolve type variables after PTransform type inference.

[brachipa] [beam-12737] add complex test, change error row type

[noreply] [BEAM-13226] Batch metrics parsing errors (#15972)

[noreply] [BEAM-13239] pass through io.EOF when decoding row (#15971)

[noreply] [BEAM-5378] Register types & funcs for examples. (#15970)

[Robert Bradshaw] Key-inferable type hints for CombinePerKey.

[Robert Bradshaw] Implement type variable binding for union types.

[Robert Bradshaw] Add comment about wildcard.

[noreply] [BEAM-3293] Add initial MultiMap side input backing type (#15975)

[Robert Bradshaw] Type inference deterministic coder fix for old Python versions.

[Robert Bradshaw] Also implement type matching for union types.

[noreply] [BEAM-13238] Bump all Dataflow Java VR timeouts (#15979)

[noreply] Merge pull request #15978: [BEAM-8691] Upgrading Java deps compatible

[noreply] [BEAM-13221] Don't drop last data. (#15939)


------------------------------------------
[...truncated 48.15 KB...]

The push refers to repository [us.gcr.io/apache-beam-testing/java-postcommit-it/java]
a19b100d9727: Preparing
3e7bbb9a0ae1: Preparing
c3e81b40e5a4: Preparing
e416520b2a16: Preparing
47d80867c3f6: Preparing
a983ffe86b29: Preparing
06b1822f9592: Preparing
b639d92bb7ed: Preparing
537a22569349: Preparing
ef3341e27015: Preparing
fa1a9ea6ce41: Preparing
839c0169a061: Preparing
78700b6b35d0: Preparing
62a5b8741e83: Preparing
36e0782f1159: Preparing
ba6e5ff31f23: Preparing
9f9f651e9303: Preparing
0b3c02b5d746: Preparing
62a747bf1719: Preparing
a983ffe86b29: Waiting
ba6e5ff31f23: Waiting
9f9f651e9303: Waiting
537a22569349: Waiting
fa1a9ea6ce41: Waiting
78700b6b35d0: Waiting
62a5b8741e83: Waiting
b639d92bb7ed: Waiting
0b3c02b5d746: Waiting
36e0782f1159: Waiting
839c0169a061: Waiting
06b1822f9592: Waiting
47d80867c3f6: Pushed
c3e81b40e5a4: Pushed
3e7bbb9a0ae1: Pushed
a19b100d9727: Pushed
a983ffe86b29: Pushed
e416520b2a16: Pushed
b639d92bb7ed: Pushed
537a22569349: Pushed
78700b6b35d0: Layer already exists
62a5b8741e83: Layer already exists
fa1a9ea6ce41: Pushed
36e0782f1159: Layer already exists
ba6e5ff31f23: Layer already exists
9f9f651e9303: Layer already exists
0b3c02b5d746: Layer already exists
839c0169a061: Pushed
62a747bf1719: Layer already exists
06b1822f9592: Pushed
ef3341e27015: Pushed
20211116124334: digest: sha256:fbb71c2337c295639bb3f17335ecb1a4c1e08f8a7161cc8227a859dd0841ee67 size: 4311

> Task :sdks:java:testing:load-tests:run
Nov 16, 2021 12:45:28 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Nov 16, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 197 files. Enable logging at DEBUG level to see which files will be staged.
Nov 16, 2021 12:45:29 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Nov 16, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Nov 16, 2021 12:45:32 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 197 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Nov 16, 2021 12:45:33 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 197 files cached, 0 files newly uploaded in 0 seconds
Nov 16, 2021 12:45:33 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Nov 16, 2021 12:45:33 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <111704 bytes, hash 5add93afc2ae7a0b7ef5a3a1540b4c011a754a5fb4b6dde69bc4cfa0f1cb5f15> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-Wt2Tr8Kuegt-9aOhVAtMARp1Sl-0tt3mm8TPoPHLXxU.pb
Nov 16, 2021 12:45:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Nov 16, 2021 12:45:34 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b676112, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5578be42, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4e49ce2b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@136965e3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@53c6f96d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@435cc7f9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4364712f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1b7a52dd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7f93dd4e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5ad5be4a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ad85136, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@737d100a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12e5da86, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6535117e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1d1cbd0f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6fa13e6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3af7d855, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77049094, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@f88bfbe, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@59bbe88a]
Nov 16, 2021 12:45:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Nov 16, 2021 12:45:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Nov 16, 2021 12:45:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Nov 16, 2021 12:45:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Nov 16, 2021 12:45:35 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4ba6ec50, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@642413d4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@fb2e3fd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43a09ce2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3f183caa, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b66322e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@63538bb4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@24534cb0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5a50d9fc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@106d77da, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6f9c5048, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5114b7c7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@dd71b20, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@767f6ee7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b6c6e70, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@f324455, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3a894088, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@370c1968, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2f0bfe17, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3206174f]
Nov 16, 2021 12:45:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Nov 16, 2021 12:45:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Nov 16, 2021 12:45:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Nov 16, 2021 12:45:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Nov 16, 2021 12:45:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Nov 16, 2021 12:45:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Nov 16, 2021 12:45:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Nov 16, 2021 12:45:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Nov 16, 2021 12:45:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Nov 16, 2021 12:45:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Nov 16, 2021 12:45:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Nov 16, 2021 12:45:35 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.35.0-SNAPSHOT
Nov 16, 2021 12:45:35 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-11-16_04_45_35-14900297558574420371?project=apache-beam-testing
Nov 16, 2021 12:45:35 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-11-16_04_45_35-14900297558574420371
Nov 16, 2021 12:45:35 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-11-16_04_45_35-14900297558574420371
Nov 16, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-11-16T12:45:42.692Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-11-o50r. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Nov 16, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-16T12:45:49.003Z: Worker configuration: e2-standard-2 in us-central1-a.
Nov 16, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-16T12:45:49.613Z: Expanding SplittableParDo operations into optimizable parts.
Nov 16, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-16T12:45:49.663Z: Expanding CollectionToSingleton operations into optimizable parts.
Nov 16, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-16T12:45:49.732Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 16, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-16T12:45:49.807Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Nov 16, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-16T12:45:49.886Z: Expanding GroupByKey operations into streaming Read/Write steps
Nov 16, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-16T12:45:49.950Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 16, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-16T12:45:50.073Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 16, 2021 12:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-16T12:45:50.107Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Nov 16, 2021 12:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-16T12:45:50.144Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Nov 16, 2021 12:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-16T12:45:50.189Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Nov 16, 2021 12:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-16T12:45:50.224Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Nov 16, 2021 12:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-16T12:45:50.282Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Nov 16, 2021 12:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-16T12:45:50.316Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Nov 16, 2021 12:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-16T12:45:50.363Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Nov 16, 2021 12:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-16T12:45:50.413Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Nov 16, 2021 12:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-16T12:45:50.468Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Nov 16, 2021 12:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-16T12:45:50.492Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Nov 16, 2021 12:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-16T12:45:50.527Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Nov 16, 2021 12:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-16T12:45:50.570Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Nov 16, 2021 12:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-16T12:45:50.635Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Nov 16, 2021 12:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-16T12:45:50.669Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Nov 16, 2021 12:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-16T12:45:50.700Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Nov 16, 2021 12:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-16T12:45:50.738Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Nov 16, 2021 12:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-16T12:45:50.777Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Nov 16, 2021 12:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-16T12:45:50.827Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Nov 16, 2021 12:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-16T12:45:50.849Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Nov 16, 2021 12:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-16T12:45:50.885Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Nov 16, 2021 12:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-16T12:45:50.919Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Nov 16, 2021 12:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-16T12:45:50.971Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Nov 16, 2021 12:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-16T12:45:51.475Z: Starting 5 ****s in us-central1-a...
Nov 16, 2021 12:46:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-16T12:46:22.130Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Nov 16, 2021 12:46:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-16T12:46:41.609Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Nov 16, 2021 12:47:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-16T12:47:38.691Z: Workers have started successfully.
Nov 16, 2021 12:47:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-16T12:47:38.739Z: Workers have started successfully.
Nov 16, 2021 4:00:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-16T16:00:25.549Z: Cancel request is committed for workflow job: 2021-11-16_04_45_35-14900297558574420371.
Nov 16, 2021 4:00:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-16T16:00:25.722Z: Cleaning up.
Nov 16, 2021 4:00:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-16T16:00:25.862Z: Stopping **** pool...
Nov 16, 2021 4:00:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-16T16:00:25.945Z: Stopping **** pool...
Nov 16, 2021 4:02:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-16T16:02:53.819Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Nov 16, 2021 4:02:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-16T16:02:53.854Z: Worker pool stopped.
Nov 16, 2021 4:03:02 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-11-16_04_45_35-14900297558574420371 finished with status CANCELLED.
Load test results for test (ID): 42417e3b-7856-41e9-982f-748d232ac7ce and timestamp: 2021-11-16T12:45:29.471000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11540.437
dataflow_v2_java11_total_bytes_count             2.22921607E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211116124334
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:fbb71c2337c295639bb3f17335ecb1a4c1e08f8a7161cc8227a859dd0841ee67
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211116124334]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:fbb71c2337c295639bb3f17335ecb1a4c1e08f8a7161cc8227a859dd0841ee67]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211116124334] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:fbb71c2337c295639bb3f17335ecb1a4c1e08f8a7161cc8227a859dd0841ee67])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:fbb71c2337c295639bb3f17335ecb1a4c1e08f8a7161cc8227a859dd0841ee67
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:fbb71c2337c295639bb3f17335ecb1a4c1e08f8a7161cc8227a859dd0841ee67
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:fbb71c2337c295639bb3f17335ecb1a4c1e08f8a7161cc8227a859dd0841ee67].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 45s
101 actionable tasks: 72 executed, 27 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/dqjkko7wblfz4

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #151

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/151/display/redirect>

Changes:


------------------------------------------
[...truncated 48.84 KB...]
13166cc94378: Preparing
bc6c3f04f977: Preparing
eddac7d9be2a: Preparing
f93a0f611dd1: Preparing
2603b8f84eb7: Preparing
58383ef9b4e7: Preparing
622f37308080: Preparing
550a9a3f0f1b: Preparing
15a758d156a7: Preparing
e8434966d4f0: Preparing
cf265fc91198: Preparing
cc0b6de21ba4: Preparing
78700b6b35d0: Preparing
62a5b8741e83: Preparing
36e0782f1159: Preparing
ba6e5ff31f23: Preparing
9f9f651e9303: Preparing
0b3c02b5d746: Preparing
58383ef9b4e7: Waiting
62a747bf1719: Preparing
622f37308080: Waiting
550a9a3f0f1b: Waiting
15a758d156a7: Waiting
0b3c02b5d746: Waiting
36e0782f1159: Waiting
cc0b6de21ba4: Waiting
cf265fc91198: Waiting
62a747bf1719: Waiting
78700b6b35d0: Waiting
ba6e5ff31f23: Waiting
e8434966d4f0: Waiting
62a5b8741e83: Waiting
9f9f651e9303: Waiting
eddac7d9be2a: Pushed
2603b8f84eb7: Pushed
bc6c3f04f977: Pushed
58383ef9b4e7: Pushed
13166cc94378: Pushed
550a9a3f0f1b: Pushed
f93a0f611dd1: Pushed
15a758d156a7: Pushed
78700b6b35d0: Layer already exists
62a5b8741e83: Layer already exists
36e0782f1159: Layer already exists
ba6e5ff31f23: Layer already exists
9f9f651e9303: Layer already exists
cf265fc91198: Pushed
0b3c02b5d746: Layer already exists
62a747bf1719: Layer already exists
622f37308080: Pushed
cc0b6de21ba4: Pushed
e8434966d4f0: Pushed
20211115124331: digest: sha256:d03c90db7a93e49c3766f1eed1b5bb9c36ddd7e4e17a4976f87418cf3475382c size: 4311

> Task :sdks:java:testing:load-tests:run
Nov 15, 2021 12:45:27 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Nov 15, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 197 files. Enable logging at DEBUG level to see which files will be staged.
Nov 15, 2021 12:45:29 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Nov 15, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Nov 15, 2021 12:45:31 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 197 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Nov 15, 2021 12:45:32 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 197 files cached, 0 files newly uploaded in 0 seconds
Nov 15, 2021 12:45:32 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Nov 15, 2021 12:45:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <111704 bytes, hash bd4a691a6f84633dfd221c889347d95e44f57c39bd7250ee525fe5b07788ee16> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-vUppGm-EYz39IhyIk0fZXkT1fDm9clDuUl_lsHeI7hY.pb
Nov 15, 2021 12:45:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Nov 15, 2021 12:45:34 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5578be42, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4e49ce2b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@136965e3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@53c6f96d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@435cc7f9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4364712f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1b7a52dd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7f93dd4e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5ad5be4a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ad85136, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@737d100a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12e5da86, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6535117e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1d1cbd0f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6fa13e6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3af7d855, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77049094, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@f88bfbe, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@59bbe88a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5d8ab698]
Nov 15, 2021 12:45:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Nov 15, 2021 12:45:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Nov 15, 2021 12:45:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Nov 15, 2021 12:45:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Nov 15, 2021 12:45:34 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@642413d4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@fb2e3fd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43a09ce2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3f183caa, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b66322e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@63538bb4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@24534cb0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5a50d9fc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@106d77da, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6f9c5048, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5114b7c7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@dd71b20, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@767f6ee7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b6c6e70, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@f324455, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3a894088, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@370c1968, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2f0bfe17, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3206174f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@42fb8c87]
Nov 15, 2021 12:45:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Nov 15, 2021 12:45:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Nov 15, 2021 12:45:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Nov 15, 2021 12:45:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Nov 15, 2021 12:45:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Nov 15, 2021 12:45:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Nov 15, 2021 12:45:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Nov 15, 2021 12:45:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Nov 15, 2021 12:45:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Nov 15, 2021 12:45:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Nov 15, 2021 12:45:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Nov 15, 2021 12:45:34 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.35.0-SNAPSHOT
Nov 15, 2021 12:45:35 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-11-15_04_45_34-10727282003328944491?project=apache-beam-testing
Nov 15, 2021 12:45:35 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-11-15_04_45_34-10727282003328944491
Nov 15, 2021 12:45:35 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-11-15_04_45_34-10727282003328944491
Nov 15, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-11-15T12:45:41.108Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-11-qxax. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Nov 15, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-15T12:45:45.543Z: Worker configuration: e2-standard-2 in us-central1-a.
Nov 15, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-15T12:45:46.328Z: Expanding SplittableParDo operations into optimizable parts.
Nov 15, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-15T12:45:46.353Z: Expanding CollectionToSingleton operations into optimizable parts.
Nov 15, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-15T12:45:46.438Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 15, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-15T12:45:46.527Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Nov 15, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-15T12:45:46.552Z: Expanding GroupByKey operations into streaming Read/Write steps
Nov 15, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-15T12:45:46.621Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 15, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-15T12:45:46.717Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 15, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-15T12:45:46.762Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Nov 15, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-15T12:45:46.797Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Nov 15, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-15T12:45:46.826Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Nov 15, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-15T12:45:46.844Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Nov 15, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-15T12:45:46.867Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Nov 15, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-15T12:45:46.894Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Nov 15, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-15T12:45:46.930Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Nov 15, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-15T12:45:46.961Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Nov 15, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-15T12:45:46.990Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Nov 15, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-15T12:45:47.022Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Nov 15, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-15T12:45:47.056Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Nov 15, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-15T12:45:47.092Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Nov 15, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-15T12:45:47.129Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Nov 15, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-15T12:45:47.162Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Nov 15, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-15T12:45:47.195Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Nov 15, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-15T12:45:47.227Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Nov 15, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-15T12:45:47.266Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Nov 15, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-15T12:45:47.299Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Nov 15, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-15T12:45:47.331Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Nov 15, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-15T12:45:47.359Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Nov 15, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-15T12:45:47.390Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Nov 15, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-15T12:45:47.434Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Nov 15, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-15T12:45:47.925Z: Starting 5 ****s in us-central1-a...
Nov 15, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-15T12:45:54.146Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Nov 15, 2021 12:46:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-15T12:46:28.754Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Nov 15, 2021 12:47:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-15T12:47:21.749Z: Workers have started successfully.
Nov 15, 2021 12:47:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-15T12:47:21.794Z: Workers have started successfully.
Nov 15, 2021 4:00:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-15T16:00:29.029Z: Cancel request is committed for workflow job: 2021-11-15_04_45_34-10727282003328944491.
Nov 15, 2021 4:00:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-15T16:00:29.100Z: Cleaning up.
Nov 15, 2021 4:00:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-15T16:00:29.194Z: Stopping **** pool...
Nov 15, 2021 4:00:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-15T16:00:29.235Z: Stopping **** pool...
Nov 15, 2021 4:02:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-15T16:02:57.344Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Nov 15, 2021 4:02:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-15T16:02:57.378Z: Worker pool stopped.
Nov 15, 2021 4:03:03 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-11-15_04_45_34-10727282003328944491 finished with status CANCELLED.
Load test results for test (ID): c3c0ac09-c8f0-4933-9972-5638b16ef9ab and timestamp: 2021-11-15T12:45:28.678000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11478.655
dataflow_v2_java11_total_bytes_count               3.8981957E9
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211115124331
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:d03c90db7a93e49c3766f1eed1b5bb9c36ddd7e4e17a4976f87418cf3475382c
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211115124331]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:d03c90db7a93e49c3766f1eed1b5bb9c36ddd7e4e17a4976f87418cf3475382c]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211115124331] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:d03c90db7a93e49c3766f1eed1b5bb9c36ddd7e4e17a4976f87418cf3475382c])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:d03c90db7a93e49c3766f1eed1b5bb9c36ddd7e4e17a4976f87418cf3475382c
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:d03c90db7a93e49c3766f1eed1b5bb9c36ddd7e4e17a4976f87418cf3475382c
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:d03c90db7a93e49c3766f1eed1b5bb9c36ddd7e4e17a4976f87418cf3475382c].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 49s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/wpusiuywl442u

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #150

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/150/display/redirect>

Changes:


------------------------------------------
[...truncated 48.26 KB...]
f10c84e98196: Preparing
647a916d6752: Preparing
ab4be44fdb6c: Preparing
c6b7c0b24692: Preparing
0ec397045672: Preparing
e9bad2710870: Preparing
f152ea72d34f: Preparing
7d73bbcb2f7f: Preparing
c511f8453bc6: Preparing
2a974265d5bc: Preparing
96bc97fcb9ff: Preparing
f84924b774ce: Preparing
78700b6b35d0: Preparing
62a5b8741e83: Preparing
36e0782f1159: Preparing
ba6e5ff31f23: Preparing
9f9f651e9303: Preparing
e9bad2710870: Waiting
0b3c02b5d746: Preparing
62a747bf1719: Preparing
f152ea72d34f: Waiting
c511f8453bc6: Waiting
7d73bbcb2f7f: Waiting
36e0782f1159: Waiting
78700b6b35d0: Waiting
0b3c02b5d746: Waiting
2a974265d5bc: Waiting
62a5b8741e83: Waiting
9f9f651e9303: Waiting
ba6e5ff31f23: Waiting
62a747bf1719: Waiting
96bc97fcb9ff: Waiting
f84924b774ce: Waiting
ab4be44fdb6c: Pushed
647a916d6752: Pushed
0ec397045672: Pushed
c6b7c0b24692: Pushed
f10c84e98196: Pushed
e9bad2710870: Pushed
7d73bbcb2f7f: Pushed
c511f8453bc6: Pushed
78700b6b35d0: Layer already exists
62a5b8741e83: Layer already exists
36e0782f1159: Layer already exists
ba6e5ff31f23: Layer already exists
96bc97fcb9ff: Pushed
9f9f651e9303: Layer already exists
0b3c02b5d746: Layer already exists
62a747bf1719: Layer already exists
f152ea72d34f: Pushed
f84924b774ce: Pushed
2a974265d5bc: Pushed
20211114124333: digest: sha256:43762bcbecb5e049d54e9a08b7b25073a5144d6eef40c8aab0a235cff2c924de size: 4311

> Task :sdks:java:testing:load-tests:run
Nov 14, 2021 12:45:30 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Nov 14, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 197 files. Enable logging at DEBUG level to see which files will be staged.
Nov 14, 2021 12:45:31 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Nov 14, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Nov 14, 2021 12:45:33 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 197 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Nov 14, 2021 12:45:34 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 197 files cached, 0 files newly uploaded in 0 seconds
Nov 14, 2021 12:45:34 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Nov 14, 2021 12:45:34 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <111704 bytes, hash df40614a24002a92873d6f51887c5033d985c7c09fefa065dc851745151d8449> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-30BhSiQAKpKHPW9RiHxQM9mFx8Cf76Bl3IUXRRUdhEk.pb
Nov 14, 2021 12:45:36 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Nov 14, 2021 12:45:36 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b676112, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5578be42, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4e49ce2b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@136965e3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@53c6f96d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@435cc7f9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4364712f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1b7a52dd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7f93dd4e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5ad5be4a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ad85136, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@737d100a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12e5da86, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6535117e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1d1cbd0f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6fa13e6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3af7d855, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77049094, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@f88bfbe, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@59bbe88a]
Nov 14, 2021 12:45:36 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Nov 14, 2021 12:45:36 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Nov 14, 2021 12:45:36 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Nov 14, 2021 12:45:36 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Nov 14, 2021 12:45:36 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4ba6ec50, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@642413d4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@fb2e3fd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43a09ce2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3f183caa, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b66322e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@63538bb4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@24534cb0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5a50d9fc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@106d77da, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6f9c5048, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5114b7c7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@dd71b20, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@767f6ee7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b6c6e70, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@f324455, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3a894088, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@370c1968, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2f0bfe17, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3206174f]
Nov 14, 2021 12:45:36 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Nov 14, 2021 12:45:36 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Nov 14, 2021 12:45:36 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Nov 14, 2021 12:45:36 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Nov 14, 2021 12:45:36 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Nov 14, 2021 12:45:36 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Nov 14, 2021 12:45:36 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Nov 14, 2021 12:45:36 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Nov 14, 2021 12:45:36 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Nov 14, 2021 12:45:36 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Nov 14, 2021 12:45:36 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Nov 14, 2021 12:45:36 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.35.0-SNAPSHOT
Nov 14, 2021 12:45:37 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-11-14_04_45_36-5683192734576139481?project=apache-beam-testing
Nov 14, 2021 12:45:37 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-11-14_04_45_36-5683192734576139481
Nov 14, 2021 12:45:37 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-11-14_04_45_36-5683192734576139481
Nov 14, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-11-14T12:45:47.524Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-11-o2oq. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Nov 14, 2021 12:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-14T12:45:51.821Z: Worker configuration: e2-standard-2 in us-central1-a.
Nov 14, 2021 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-14T12:45:52.768Z: Expanding SplittableParDo operations into optimizable parts.
Nov 14, 2021 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-14T12:45:52.874Z: Expanding CollectionToSingleton operations into optimizable parts.
Nov 14, 2021 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-14T12:45:52.947Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 14, 2021 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-14T12:45:53.018Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Nov 14, 2021 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-14T12:45:53.041Z: Expanding GroupByKey operations into streaming Read/Write steps
Nov 14, 2021 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-14T12:45:53.097Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 14, 2021 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-14T12:45:53.193Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 14, 2021 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-14T12:45:53.234Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Nov 14, 2021 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-14T12:45:53.262Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Nov 14, 2021 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-14T12:45:53.289Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Nov 14, 2021 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-14T12:45:53.311Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Nov 14, 2021 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-14T12:45:53.342Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Nov 14, 2021 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-14T12:45:53.365Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Nov 14, 2021 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-14T12:45:53.387Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Nov 14, 2021 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-14T12:45:53.420Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Nov 14, 2021 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-14T12:45:53.452Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Nov 14, 2021 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-14T12:45:53.492Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Nov 14, 2021 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-14T12:45:53.520Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Nov 14, 2021 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-14T12:45:53.578Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Nov 14, 2021 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-14T12:45:53.603Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Nov 14, 2021 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-14T12:45:53.627Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Nov 14, 2021 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-14T12:45:53.663Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Nov 14, 2021 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-14T12:45:53.688Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Nov 14, 2021 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-14T12:45:53.742Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Nov 14, 2021 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-14T12:45:53.777Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Nov 14, 2021 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-14T12:45:53.800Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Nov 14, 2021 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-14T12:45:53.836Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Nov 14, 2021 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-14T12:45:53.861Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Nov 14, 2021 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-14T12:45:53.896Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Nov 14, 2021 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-14T12:45:54.229Z: Starting 5 ****s in us-central1-a...
Nov 14, 2021 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-14T12:45:58.201Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Nov 14, 2021 12:46:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-14T12:46:34.488Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Nov 14, 2021 12:47:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-14T12:47:33.907Z: Workers have started successfully.
Nov 14, 2021 12:47:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-14T12:47:33.940Z: Workers have started successfully.
Nov 14, 2021 4:00:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-14T16:00:26.915Z: Cancel request is committed for workflow job: 2021-11-14_04_45_36-5683192734576139481.
Nov 14, 2021 4:00:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-14T16:00:27.033Z: Cleaning up.
Nov 14, 2021 4:00:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-14T16:00:27.101Z: Stopping **** pool...
Nov 14, 2021 4:00:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-14T16:00:27.181Z: Stopping **** pool...
Nov 14, 2021 4:02:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-14T16:02:45.651Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Nov 14, 2021 4:02:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-14T16:02:45.681Z: Worker pool stopped.
Nov 14, 2021 4:02:51 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-11-14_04_45_36-5683192734576139481 finished with status CANCELLED.
Load test results for test (ID): 5ffcbc17-ff6d-44f3-8604-517e0ea17320 and timestamp: 2021-11-14T12:45:30.910000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11489.554
dataflow_v2_java11_total_bytes_count             1.74862132E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211114124333
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:43762bcbecb5e049d54e9a08b7b25073a5144d6eef40c8aab0a235cff2c924de
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211114124333]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:43762bcbecb5e049d54e9a08b7b25073a5144d6eef40c8aab0a235cff2c924de]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211114124333] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:43762bcbecb5e049d54e9a08b7b25073a5144d6eef40c8aab0a235cff2c924de])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:43762bcbecb5e049d54e9a08b7b25073a5144d6eef40c8aab0a235cff2c924de
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:43762bcbecb5e049d54e9a08b7b25073a5144d6eef40c8aab0a235cff2c924de
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:43762bcbecb5e049d54e9a08b7b25073a5144d6eef40c8aab0a235cff2c924de].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 34s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/se24kacyytz6q

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #149

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/149/display/redirect?page=changes>

Changes:

[mmack] [BEAM-13009] Bugfix: Prevent dataloss in DynamoDBIO write if

[mmack] [BEAM-13009] Bugfix: Prevent dataloss in DynamoDBIO write if

[aydar.zaynutdinov] [BEAM-13224][Playground]

[Alexey Romanenko] [BEAM-4868] Bump com.amazonaws to 1.12.106

[noreply] [BEAM-13147] Fix nullability issues in aws (SDK v1 + v2) for AwsModule

[noreply] [BEAM-3293] Add encoding of map/multimap side inputs into proto (#15960)

[noreply] [BEAM-13052] Implement ProtoPlusCoder and add it to the default options

[noreply] [BEAM-13193] Add elements field in ProcessBundleRequest and ProcessBu…

[noreply] [BEAM-3304] Refactor trigger API (#15952)

[noreply] [BEAM-3293] Add NewKeyedIterable function to SideInputAdapter interface

[noreply] [BEAM-13223] Documents two new features of cross-language transforms


------------------------------------------
[...truncated 48.94 KB...]
36e0782f1159: Waiting
4efe358eb92d: Waiting
ba6e5ff31f23: Waiting
9f9f651e9303: Waiting
548ebbd63351: Waiting
5aa55cdcd3b7: Pushed
a18f9f86ccf0: Pushed
d51569dbd862: Pushed
337e6e031976: Pushed
5f0c26ac26b3: Pushed
548ebbd63351: Pushed
5ceb4016e1e8: Pushed
4efe358eb92d: Pushed
78700b6b35d0: Layer already exists
62a5b8741e83: Layer already exists
45a48cd22bd3: Pushed
36e0782f1159: Layer already exists
ba6e5ff31f23: Layer already exists
9f9f651e9303: Layer already exists
0b3c02b5d746: Layer already exists
62a747bf1719: Layer already exists
2b46f4037d09: Pushed
f93b95d11f1c: Pushed
d85eeaf7a18a: Pushed
20211113124333: digest: sha256:bcabd5c18e1479a4df46a226f2b252d16f9ed630894ade61aced83853f334467 size: 4311

> Task :sdks:java:testing:load-tests:run
Nov 13, 2021 12:45:21 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Nov 13, 2021 12:45:22 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 197 files. Enable logging at DEBUG level to see which files will be staged.
Nov 13, 2021 12:45:23 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Nov 13, 2021 12:45:23 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Nov 13, 2021 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 197 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Nov 13, 2021 12:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 197 files cached, 0 files newly uploaded in 0 seconds
Nov 13, 2021 12:45:25 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Nov 13, 2021 12:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <111704 bytes, hash 2967e31a4fd7e7a80b2376cdaad7e25d14e9ea9929b597ef3e988646365190bc> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-KWfjGk_X56gLI3bNqtfiXRTp6pkptZfvPpiGRjZRkLw.pb
Nov 13, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Nov 13, 2021 12:45:27 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b676112, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5578be42, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4e49ce2b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@136965e3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@53c6f96d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@435cc7f9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4364712f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1b7a52dd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7f93dd4e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5ad5be4a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ad85136, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@737d100a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12e5da86, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6535117e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1d1cbd0f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6fa13e6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3af7d855, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77049094, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@f88bfbe, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@59bbe88a]
Nov 13, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Nov 13, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Nov 13, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Nov 13, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Nov 13, 2021 12:45:27 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4ba6ec50, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@642413d4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@fb2e3fd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43a09ce2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3f183caa, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b66322e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@63538bb4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@24534cb0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5a50d9fc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@106d77da, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6f9c5048, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5114b7c7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@dd71b20, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@767f6ee7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b6c6e70, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@f324455, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3a894088, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@370c1968, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2f0bfe17, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3206174f]
Nov 13, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Nov 13, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Nov 13, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Nov 13, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Nov 13, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Nov 13, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Nov 13, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Nov 13, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Nov 13, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Nov 13, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Nov 13, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Nov 13, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.35.0-SNAPSHOT
Nov 13, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-11-13_04_45_27-7894454680376627106?project=apache-beam-testing
Nov 13, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-11-13_04_45_27-7894454680376627106
Nov 13, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-11-13_04_45_27-7894454680376627106
Nov 13, 2021 12:45:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-11-13T12:45:34.552Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-11-8en0. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Nov 13, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-13T12:45:38.522Z: Worker configuration: e2-standard-2 in us-central1-a.
Nov 13, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-13T12:45:39.066Z: Expanding SplittableParDo operations into optimizable parts.
Nov 13, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-13T12:45:39.127Z: Expanding CollectionToSingleton operations into optimizable parts.
Nov 13, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-13T12:45:39.193Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 13, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-13T12:45:39.265Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Nov 13, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-13T12:45:39.293Z: Expanding GroupByKey operations into streaming Read/Write steps
Nov 13, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-13T12:45:39.355Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 13, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-13T12:45:39.443Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 13, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-13T12:45:39.471Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Nov 13, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-13T12:45:39.499Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Nov 13, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-13T12:45:39.562Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Nov 13, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-13T12:45:39.585Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Nov 13, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-13T12:45:39.611Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Nov 13, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-13T12:45:39.644Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Nov 13, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-13T12:45:39.677Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Nov 13, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-13T12:45:39.707Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Nov 13, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-13T12:45:39.744Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Nov 13, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-13T12:45:39.776Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Nov 13, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-13T12:45:39.809Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Nov 13, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-13T12:45:39.840Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Nov 13, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-13T12:45:39.875Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Nov 13, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-13T12:45:39.909Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Nov 13, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-13T12:45:39.955Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Nov 13, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-13T12:45:39.982Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Nov 13, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-13T12:45:40.003Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Nov 13, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-13T12:45:40.038Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Nov 13, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-13T12:45:40.070Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Nov 13, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-13T12:45:40.129Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Nov 13, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-13T12:45:40.159Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Nov 13, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-13T12:45:40.185Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Nov 13, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-13T12:45:40.512Z: Starting 5 ****s in us-central1-a...
Nov 13, 2021 12:46:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-13T12:46:11.686Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Nov 13, 2021 12:46:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-13T12:46:25.510Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Nov 13, 2021 12:47:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-13T12:47:21.345Z: Workers have started successfully.
Nov 13, 2021 12:47:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-13T12:47:21.366Z: Workers have started successfully.
Nov 13, 2021 4:00:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-13T16:00:26.555Z: Cancel request is committed for workflow job: 2021-11-13_04_45_27-7894454680376627106.
Nov 13, 2021 4:00:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-13T16:00:26.610Z: Cleaning up.
Nov 13, 2021 4:00:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-13T16:00:26.706Z: Stopping **** pool...
Nov 13, 2021 4:00:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-13T16:00:26.745Z: Stopping **** pool...
Nov 13, 2021 4:02:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-13T16:02:45.063Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Nov 13, 2021 4:02:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-13T16:02:45.106Z: Worker pool stopped.
Nov 13, 2021 4:02:50 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-11-13_04_45_27-7894454680376627106 finished with status CANCELLED.
Load test results for test (ID): 19cd8509-1910-4f6d-832d-5f0f7276db5c and timestamp: 2021-11-13T12:45:22.673000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11552.236
dataflow_v2_java11_total_bytes_count             1.95279897E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211113124333
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:bcabd5c18e1479a4df46a226f2b252d16f9ed630894ade61aced83853f334467
Deleted: sha256:42201945660da287e915df98c55289f326627a04547468a0af76161ee7670678
Deleted: sha256:710a039a07511c2feb76251d921c58051ca7e21bf291247993fb7d4a1f393677
Deleted: sha256:b2858822efffb12059a9df456e0dec1f2441cdfb4a6ea1912042188e95d6751b
Deleted: sha256:34c907a17ab4b61b18cd76146136e389ba7aafafd590c068cc92c4622744e121
Deleted: sha256:694c40bd299d70813dbdfb275da9927e239be7150004dc1b6a79a9fd9d290b88
Deleted: sha256:1e74f10f4665b395dfccc6caeb5c1110faf7ea49d4f18a88d0903e151bf6899f
Deleted: sha256:6e2ee22f59e3e4f1bd3274328beb01792deb409292cb7c7b77da5892460fce6c
Deleted: sha256:9b907e93c92256207b20ad7e12b53bc3a105e3952538e6f041b25d81ed3886d0
Deleted: sha256:c0eac6df4091c4d900d1277418f0bfaacf0aab32b9b959f0abd56f49e8da1231
Deleted: sha256:ec839a739e511698749dfe1544b0ea5714dfe33f88e1cc3f673d56bf2f3fd6c9
Deleted: sha256:55dfe866f54c28378b04c355d28d201d8dca52e383f4ca6114af22a138c1652a
Deleted: sha256:0828d2e1fe4ec84bdaa0acefb61e879ca7a562277d20fadcdccebfb33af6bab6
Deleted: sha256:e6d178f5ba7b42ab94e0c6258bc340db5ce354df74682e0a72d6c06df23df137
Deleted: sha256:9eb135533d44d58391215a8b1875ba90d7d0b599f5a577b1ee7e4768b3b60a40
Deleted: sha256:58233ef39756774fd183667b75279f6ffc8266397607f016bbfad06d954076c2
Deleted: sha256:dd016ece6bb68c5091a47effbdeb94bfdac8d9816657d79532d22853252a1826
Deleted: sha256:e75b2ae5a24e4e3489969eee381558c028bce5f71f85a44eb5d58f0ebf133b72
Deleted: sha256:7797b5a8932a65bb997dc382eb1ae027e59a6698798a2f1eca0e6a84421cfc6e
Deleted: sha256:fd3e42c53163499cf6fa616bf04f45d1816b4caf3f79b8706ffc9a2cc683f094
Deleted: sha256:1a94b04dfaa07fb5cb681b08707c40f3281f791051686f7399db25f00ed91b78
Deleted: sha256:9a0cc664e37a3979c40cabb23f8d53a877f576503e7abe0c99346c32e3355d9c
Deleted: sha256:eb6095aad2e18da16b589c46e0e909c7ab8ed3b16442d6f7efd4079b877b628f
Deleted: sha256:eb7fcfc703c62e482882d32855789a0f69f7155c2f971a9060401e2e9bb02ed3
Deleted: sha256:50d6d305c87a3a4e2899f696a92980b41b68c43614eecbc2430db39d4c825f8a
Deleted: sha256:452df4e79eed2d3fbbbf153b8a741c47af2fd1d6f7ef7f3259160cd2e32def36
Deleted: sha256:8ad51c25810959682656f926083ca543794a8bac715c001818bf890ba369e389
Deleted: sha256:49c144cd09ca614ce300f151b5d81ba4eaab34eddebb736987224602fdbafc22
Deleted: sha256:06fc00fb81fa24768626a757697c07858452ab2ec94d35136c88b3fbd5208d46
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211113124333]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:bcabd5c18e1479a4df46a226f2b252d16f9ed630894ade61aced83853f334467]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211113124333] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:bcabd5c18e1479a4df46a226f2b252d16f9ed630894ade61aced83853f334467])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:bcabd5c18e1479a4df46a226f2b252d16f9ed630894ade61aced83853f334467
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:bcabd5c18e1479a4df46a226f2b252d16f9ed630894ade61aced83853f334467
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:bcabd5c18e1479a4df46a226f2b252d16f9ed630894ade61aced83853f334467].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 34s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/5275qyqmey4bo

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #148

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/148/display/redirect?page=changes>

Changes:

[Kyle Weaver] Update Beam website to release 2.34.0.

[Alexey Romanenko] [BEAM-8727] Bump software.amazon.awssdk to 2.17.77

[noreply] [BEAM-12550] Parallelizable kurtosis Implementation (#15909)

[noreply] [BEAM-13016] Remove avro-python3 dependency from Beam (#15900)

[Kyle Weaver] Prepare docs for 2.34.0 release.

[noreply] [BEAM-13133] Loosen partitioning requirement for sample (#15818)

[noreply] [BEAM-13228] fix data race in metrics.store (#15946)

[noreply] [BEAM-3293] Add binding cases for MultiMap side inputs (#15943)

[noreply] [BEAM-13222] Re-enable Spanner integration tests (#15948)

[noreply] Merge pull request #15926 from [BEAM-13110][Playground] Playground

[noreply] [BEAM-13025] Disable deduplicating messages, as Dedupe is broken on

[noreply] [BEAM-13001] collect DoFn metrics for Combine (#15911)


------------------------------------------
[...truncated 48.12 KB...]

See: https://cloud.google.com/container-registry/docs/support/deprecation-notices#gcloud-docker

The push refers to repository [us.gcr.io/apache-beam-testing/java-postcommit-it/java]
a869a04e1f77: Preparing
040ef4dc64d6: Preparing
dad6e3094b49: Preparing
89b0fc2d5155: Preparing
417a22b9dae1: Preparing
6958372bfa2d: Preparing
0ab501d5dd57: Preparing
cd236ee17a00: Preparing
ffbc5b28ded1: Preparing
1bf2d6c57d48: Preparing
6b769f6763e8: Preparing
7a69b1232949: Preparing
78700b6b35d0: Preparing
62a5b8741e83: Preparing
36e0782f1159: Preparing
cd236ee17a00: Waiting
ba6e5ff31f23: Preparing
6958372bfa2d: Waiting
9f9f651e9303: Preparing
0b3c02b5d746: Preparing
62a747bf1719: Preparing
1bf2d6c57d48: Waiting
62a5b8741e83: Waiting
6b769f6763e8: Waiting
0b3c02b5d746: Waiting
9f9f651e9303: Waiting
36e0782f1159: Waiting
ba6e5ff31f23: Waiting
78700b6b35d0: Waiting
dad6e3094b49: Pushed
040ef4dc64d6: Pushed
417a22b9dae1: Pushed
a869a04e1f77: Pushed
6958372bfa2d: Pushed
89b0fc2d5155: Pushed
cd236ee17a00: Pushed
ffbc5b28ded1: Pushed
78700b6b35d0: Layer already exists
62a5b8741e83: Layer already exists
36e0782f1159: Layer already exists
ba6e5ff31f23: Layer already exists
6b769f6763e8: Pushed
9f9f651e9303: Layer already exists
0b3c02b5d746: Layer already exists
62a747bf1719: Layer already exists
0ab501d5dd57: Pushed
7a69b1232949: Pushed
1bf2d6c57d48: Pushed
20211112124625: digest: sha256:30196a759ff24d3bfa0a6da33f36eca413fade78dc7715a230afe2a24603537d size: 4311

> Task :sdks:java:testing:load-tests:run
Nov 12, 2021 12:48:25 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Nov 12, 2021 12:48:25 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 197 files. Enable logging at DEBUG level to see which files will be staged.
Nov 12, 2021 12:48:26 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Nov 12, 2021 12:48:26 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Nov 12, 2021 12:48:27 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 197 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Nov 12, 2021 12:48:28 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 197 files cached, 0 files newly uploaded in 0 seconds
Nov 12, 2021 12:48:28 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Nov 12, 2021 12:48:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <111704 bytes, hash a94b1fadaa038bcbec4f0ef2dfe124c33780d89025b6bbeb3fd8034af93dd665> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-qUsfraoDi8vsTw7y3-EkwzeA2JAltrvrP9gDSvk91mU.pb
Nov 12, 2021 12:48:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Nov 12, 2021 12:48:30 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b676112, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5578be42, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4e49ce2b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@136965e3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@53c6f96d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@435cc7f9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4364712f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1b7a52dd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7f93dd4e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5ad5be4a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ad85136, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@737d100a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12e5da86, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6535117e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1d1cbd0f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6fa13e6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3af7d855, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77049094, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@f88bfbe, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@59bbe88a]
Nov 12, 2021 12:48:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Nov 12, 2021 12:48:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Nov 12, 2021 12:48:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Nov 12, 2021 12:48:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Nov 12, 2021 12:48:30 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4ba6ec50, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@642413d4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@fb2e3fd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43a09ce2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3f183caa, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b66322e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@63538bb4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@24534cb0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5a50d9fc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@106d77da, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6f9c5048, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5114b7c7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@dd71b20, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@767f6ee7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b6c6e70, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@f324455, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3a894088, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@370c1968, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2f0bfe17, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3206174f]
Nov 12, 2021 12:48:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Nov 12, 2021 12:48:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Nov 12, 2021 12:48:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Nov 12, 2021 12:48:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Nov 12, 2021 12:48:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Nov 12, 2021 12:48:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Nov 12, 2021 12:48:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Nov 12, 2021 12:48:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Nov 12, 2021 12:48:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Nov 12, 2021 12:48:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Nov 12, 2021 12:48:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Nov 12, 2021 12:48:30 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.35.0-SNAPSHOT
Nov 12, 2021 12:48:31 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-11-12_04_48_30-10808268834329089366?project=apache-beam-testing
Nov 12, 2021 12:48:31 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-11-12_04_48_30-10808268834329089366
Nov 12, 2021 12:48:31 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-11-12_04_48_30-10808268834329089366
Nov 12, 2021 12:48:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-11-12T12:48:36.449Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-11-v0gq. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Nov 12, 2021 12:48:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-12T12:48:40.580Z: Worker configuration: e2-standard-2 in us-central1-a.
Nov 12, 2021 12:48:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-12T12:48:41.243Z: Expanding SplittableParDo operations into optimizable parts.
Nov 12, 2021 12:48:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-12T12:48:41.294Z: Expanding CollectionToSingleton operations into optimizable parts.
Nov 12, 2021 12:48:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-12T12:48:41.370Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 12, 2021 12:48:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-12T12:48:41.446Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Nov 12, 2021 12:48:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-12T12:48:41.485Z: Expanding GroupByKey operations into streaming Read/Write steps
Nov 12, 2021 12:48:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-12T12:48:41.547Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 12, 2021 12:48:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-12T12:48:41.668Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 12, 2021 12:48:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-12T12:48:41.699Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Nov 12, 2021 12:48:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-12T12:48:41.733Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Nov 12, 2021 12:48:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-12T12:48:41.765Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Nov 12, 2021 12:48:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-12T12:48:41.799Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Nov 12, 2021 12:48:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-12T12:48:41.824Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Nov 12, 2021 12:48:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-12T12:48:41.846Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Nov 12, 2021 12:48:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-12T12:48:41.898Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Nov 12, 2021 12:48:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-12T12:48:41.927Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Nov 12, 2021 12:48:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-12T12:48:41.965Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Nov 12, 2021 12:48:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-12T12:48:41.988Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Nov 12, 2021 12:48:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-12T12:48:42.032Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Nov 12, 2021 12:48:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-12T12:48:42.076Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Nov 12, 2021 12:48:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-12T12:48:42.103Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Nov 12, 2021 12:48:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-12T12:48:42.147Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Nov 12, 2021 12:48:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-12T12:48:42.189Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Nov 12, 2021 12:48:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-12T12:48:42.219Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Nov 12, 2021 12:48:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-12T12:48:42.248Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Nov 12, 2021 12:48:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-12T12:48:42.291Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Nov 12, 2021 12:48:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-12T12:48:42.330Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Nov 12, 2021 12:48:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-12T12:48:42.364Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Nov 12, 2021 12:48:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-12T12:48:42.393Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Nov 12, 2021 12:48:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-12T12:48:42.426Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Nov 12, 2021 12:48:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-12T12:48:42.839Z: Starting 5 ****s in us-central1-a...
Nov 12, 2021 12:48:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-12T12:48:48.710Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Nov 12, 2021 12:49:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-12T12:49:28.086Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Nov 12, 2021 12:50:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-12T12:50:21.864Z: Workers have started successfully.
Nov 12, 2021 12:50:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-12T12:50:21.906Z: Workers have started successfully.
Nov 12, 2021 4:00:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-12T16:00:28.691Z: Cancel request is committed for workflow job: 2021-11-12_04_48_30-10808268834329089366.
Nov 12, 2021 4:00:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-12T16:00:28.783Z: Cleaning up.
Nov 12, 2021 4:00:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-12T16:00:28.871Z: Stopping **** pool...
Nov 12, 2021 4:00:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-12T16:00:28.942Z: Stopping **** pool...
Nov 12, 2021 4:02:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-12T16:02:52.005Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Nov 12, 2021 4:02:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-12T16:02:52.044Z: Worker pool stopped.
Nov 12, 2021 4:02:58 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-11-12_04_48_30-10808268834329089366 finished with status CANCELLED.
Load test results for test (ID): b51f9d9a-f860-44ad-aed2-1ccc8ada050b and timestamp: 2021-11-12T12:48:25.667000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11372.685
dataflow_v2_java11_total_bytes_count             2.04816701E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211112124625
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:30196a759ff24d3bfa0a6da33f36eca413fade78dc7715a230afe2a24603537d
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211112124625]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:30196a759ff24d3bfa0a6da33f36eca413fade78dc7715a230afe2a24603537d]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211112124625] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:30196a759ff24d3bfa0a6da33f36eca413fade78dc7715a230afe2a24603537d])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:30196a759ff24d3bfa0a6da33f36eca413fade78dc7715a230afe2a24603537d
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:30196a759ff24d3bfa0a6da33f36eca413fade78dc7715a230afe2a24603537d
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:30196a759ff24d3bfa0a6da33f36eca413fade78dc7715a230afe2a24603537d].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 16m 51s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/dkdbg5vkihawa

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #147

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/147/display/redirect?page=changes>

Changes:

[sergey.kalinin] Initial add for dockerfiles and build scripts

[sergey.kalinin] Initial add for terraform scripts for infra provision

[sergey.kalinin] Add licenses

[sergey.kalinin] Add fixes to docker plugin and also build arg for beam sdk image

[sergey.kalinin] Remove wrongly added comments

[sergey.kalinin] Add beam playground applications deploy

[sergey.kalinin] Remove wrongly added log file

[sergey.kalinin] Fix syntax

[sergey.kalinin] Added default values

[sergey.kalinin] Add default value to base image

[sergey.kalinin] Added Beam Version as build arg, also changed way of getting tags for

[sergey.kalinin] Added readme file

[sergey.kalinin] Add workflow to build backend application

[sergey.kalinin] Add license

[asottile] remove unused future / futures dependencies

[sergey.kalinin] Add frontend workflow

[sergey.kalinin] Add license to README and added reader access to bucket

[sergey.kalinin] Remove trailing whitespace

[relax] for existing tables, no need to set a schema

[avilovpavel6] Fix environment_service_test.go

[sergey.kalinin] Remove comment from dockerfile

[noreply] [BEAM-13190] Elide trailing nils (#15920)

[noreply] [BEAM-13052] Increment protobuf version to 3.19.0 (#15883)

[noreply] [BEAM-3304] refactoring trigger package in GoSDK (#15933)

[noreply] [BEAM-13001] Hook to configure sampling period for DoFn metric

[noreply] [BEAM-3293] Add MultiMap side input decomposition (#15934)

[noreply] [BEAM-12651] Exclude packages from jacoco report (#15792)

[noreply] [BEAM-13222] Skip spanner integration tests (#15942)

[noreply] BEAM-13189 Python TextIO: add escapechar feature. (#15901)


------------------------------------------
[...truncated 48.78 KB...]
The push refers to repository [us.gcr.io/apache-beam-testing/java-postcommit-it/java]
1e048c164368: Preparing
997e117e5e87: Preparing
e2e032eff0a3: Preparing
17d52d9852e0: Preparing
9d26d2bba9b6: Preparing
62eb36d48d33: Preparing
e4ba9ced659c: Preparing
98df2d3515ab: Preparing
c63ca5a1ddaa: Preparing
1920673022c2: Preparing
406725ccd9a0: Preparing
d8281c757a99: Preparing
78700b6b35d0: Preparing
62a5b8741e83: Preparing
36e0782f1159: Preparing
ba6e5ff31f23: Preparing
c63ca5a1ddaa: Waiting
9f9f651e9303: Preparing
0b3c02b5d746: Preparing
62a747bf1719: Preparing
406725ccd9a0: Waiting
ba6e5ff31f23: Waiting
d8281c757a99: Waiting
62a5b8741e83: Waiting
e4ba9ced659c: Waiting
9f9f651e9303: Waiting
62eb36d48d33: Waiting
98df2d3515ab: Waiting
0b3c02b5d746: Waiting
62a747bf1719: Waiting
78700b6b35d0: Waiting
36e0782f1159: Waiting
e2e032eff0a3: Pushed
9d26d2bba9b6: Pushed
997e117e5e87: Pushed
17d52d9852e0: Pushed
1e048c164368: Pushed
62eb36d48d33: Pushed
98df2d3515ab: Pushed
c63ca5a1ddaa: Pushed
78700b6b35d0: Layer already exists
62a5b8741e83: Layer already exists
36e0782f1159: Layer already exists
406725ccd9a0: Pushed
ba6e5ff31f23: Layer already exists
9f9f651e9303: Layer already exists
0b3c02b5d746: Layer already exists
62a747bf1719: Layer already exists
e4ba9ced659c: Pushed
d8281c757a99: Pushed
1920673022c2: Pushed
20211111124331: digest: sha256:7ef99c842bfc7cfa17ba0b0d1c3dbc603b4b7775aa909cd9b3eca5c9217e8a5d size: 4311

> Task :sdks:java:testing:load-tests:run
Nov 11, 2021 12:45:25 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Nov 11, 2021 12:45:25 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 197 files. Enable logging at DEBUG level to see which files will be staged.
Nov 11, 2021 12:45:26 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Nov 11, 2021 12:45:26 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Nov 11, 2021 12:45:29 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 197 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Nov 11, 2021 12:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 197 files cached, 0 files newly uploaded in 0 seconds
Nov 11, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Nov 11, 2021 12:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <111704 bytes, hash 02e10b42c35c5dc81f90680922e37f28d6dd213d4a4a76d9258efa49c1132095> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-AuELQsNcXcgfkGgJIuN_KNbdIT1KSnbZJY76ScETIJU.pb
Nov 11, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Nov 11, 2021 12:45:31 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4e49ce2b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@136965e3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@53c6f96d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@435cc7f9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4364712f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1b7a52dd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7f93dd4e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5ad5be4a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ad85136, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@737d100a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12e5da86, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6535117e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1d1cbd0f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6fa13e6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3af7d855, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77049094, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@f88bfbe, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@59bbe88a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5d8ab698, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@ed91d8d]
Nov 11, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Nov 11, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Nov 11, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Nov 11, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Nov 11, 2021 12:45:31 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@fb2e3fd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43a09ce2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3f183caa, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b66322e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@63538bb4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@24534cb0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5a50d9fc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@106d77da, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6f9c5048, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5114b7c7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@dd71b20, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@767f6ee7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b6c6e70, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@f324455, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3a894088, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@370c1968, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2f0bfe17, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3206174f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@42fb8c87, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@15eb0ae9]
Nov 11, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Nov 11, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Nov 11, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Nov 11, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Nov 11, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Nov 11, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Nov 11, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Nov 11, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Nov 11, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Nov 11, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Nov 11, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Nov 11, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.35.0-SNAPSHOT
Nov 11, 2021 12:45:32 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-11-11_04_45_32-6754862074261732272?project=apache-beam-testing
Nov 11, 2021 12:45:32 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-11-11_04_45_32-6754862074261732272
Nov 11, 2021 12:45:32 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-11-11_04_45_32-6754862074261732272
Nov 11, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-11-11T12:45:39.463Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-11-fgce. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Nov 11, 2021 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-11T12:45:45.658Z: Worker configuration: e2-standard-2 in us-central1-a.
Nov 11, 2021 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-11T12:45:46.285Z: Expanding SplittableParDo operations into optimizable parts.
Nov 11, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-11T12:45:46.319Z: Expanding CollectionToSingleton operations into optimizable parts.
Nov 11, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-11T12:45:46.403Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 11, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-11T12:45:46.509Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Nov 11, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-11T12:45:46.546Z: Expanding GroupByKey operations into streaming Read/Write steps
Nov 11, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-11T12:45:46.683Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 11, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-11T12:45:47.052Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 11, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-11T12:45:47.094Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Nov 11, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-11T12:45:47.161Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Nov 11, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-11T12:45:47.194Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Nov 11, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-11T12:45:47.239Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Nov 11, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-11T12:45:47.268Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Nov 11, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-11T12:45:47.304Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Nov 11, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-11T12:45:47.337Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Nov 11, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-11T12:45:47.373Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Nov 11, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-11T12:45:47.406Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Nov 11, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-11T12:45:47.457Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Nov 11, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-11T12:45:47.510Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Nov 11, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-11T12:45:47.549Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Nov 11, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-11T12:45:47.578Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Nov 11, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-11T12:45:47.619Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Nov 11, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-11T12:45:47.644Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Nov 11, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-11T12:45:47.672Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Nov 11, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-11T12:45:47.710Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Nov 11, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-11T12:45:47.750Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Nov 11, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-11T12:45:47.788Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Nov 11, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-11T12:45:47.846Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Nov 11, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-11T12:45:47.895Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Nov 11, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-11T12:45:47.956Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Nov 11, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-11T12:45:48.403Z: Starting 5 ****s in us-central1-a...
Nov 11, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-11T12:45:50.569Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Nov 11, 2021 12:46:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-11T12:46:33.244Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Nov 11, 2021 12:47:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-11T12:47:26.542Z: Workers have started successfully.
Nov 11, 2021 12:47:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-11T12:47:26.642Z: Workers have started successfully.
Nov 11, 2021 4:00:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-11T16:00:26.692Z: Cancel request is committed for workflow job: 2021-11-11_04_45_32-6754862074261732272.
Nov 11, 2021 4:00:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-11T16:00:27.022Z: Cleaning up.
Nov 11, 2021 4:00:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-11T16:00:27.148Z: Stopping **** pool...
Nov 11, 2021 4:00:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-11T16:00:27.247Z: Stopping **** pool...
Nov 11, 2021 4:02:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-11T16:02:49.657Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Nov 11, 2021 4:02:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-11T16:02:49.765Z: Worker pool stopped.
Nov 11, 2021 4:02:55 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-11-11_04_45_32-6754862074261732272 finished with status CANCELLED.
Load test results for test (ID): fec54057-83e8-4360-b104-5d53221f6452 and timestamp: 2021-11-11T12:45:26.196000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11545.394
dataflow_v2_java11_total_bytes_count             3.08687739E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211111124331
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:7ef99c842bfc7cfa17ba0b0d1c3dbc603b4b7775aa909cd9b3eca5c9217e8a5d
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211111124331]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:7ef99c842bfc7cfa17ba0b0d1c3dbc603b4b7775aa909cd9b3eca5c9217e8a5d]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211111124331] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:7ef99c842bfc7cfa17ba0b0d1c3dbc603b4b7775aa909cd9b3eca5c9217e8a5d])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:7ef99c842bfc7cfa17ba0b0d1c3dbc603b4b7775aa909cd9b3eca5c9217e8a5d
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:7ef99c842bfc7cfa17ba0b0d1c3dbc603b4b7775aa909cd9b3eca5c9217e8a5d
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:7ef99c842bfc7cfa17ba0b0d1c3dbc603b4b7775aa909cd9b3eca5c9217e8a5d].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 39s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/4ep3tcom6r23g

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #146

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/146/display/redirect?page=changes>

Changes:

[Kyle Weaver] [BEAM-12976] Add Java interfaces for projection pushdown.

[Kyle Weaver] fix cast

[Kyle Weaver] [BEAM-12976] Address review comments.

[Kyle Weaver] Hard-code output tag instead of making it public.

[Kyle Weaver] revert MAIN_OUTPUT_TAG to private

[Kyle Weaver] Remove WITHOUT_FIELD_ORDERING. Field reordering will be a prerequisite

[msbukal] Minor HL7v2IO Fixes

[dpcollins] Add detection of BundleFinalizers to DoFnSignatures

[dpcollins] Add enable_prime to the unified worker experiment list.

[dpcollins] Raise an exception before trying to launch a pipeline using

[dpcollins] Fix detection to properly null check.

[Andrew Pilloud] [BEAM-13056] Expose FieldAccess in DoFnSchemaInformation

[noreply] [BEAM-11097] Create hook to enable cross-bundle side input caching

[noreply] Merge pull request #15804 from [BEAM-13109][Playground] Add processing

[noreply] Improve FhirIO LRO Counters + minor fixes (#15921)

[noreply] Merge pull request #15771 from Enable BQ Standard SQL dialect style

[noreply] [BEAM-11217] Metrics querying for Pcol metrics (#15923)


------------------------------------------
[...truncated 46.67 KB...]
494d77973dc2: Preparing
b0ad93889d25: Preparing
05beb89e30dd: Preparing
862dcfb29807: Preparing
0be7a0def9f1: Preparing
11b44f891c78: Preparing
c7df4a35b5d2: Preparing
14723a6d7dbf: Preparing
78700b6b35d0: Preparing
62a5b8741e83: Preparing
36e0782f1159: Preparing
ba6e5ff31f23: Preparing
9f9f651e9303: Preparing
0b3c02b5d746: Preparing
62a747bf1719: Preparing
c7df4a35b5d2: Waiting
14723a6d7dbf: Waiting
78700b6b35d0: Waiting
62a5b8741e83: Waiting
b0ad93889d25: Waiting
36e0782f1159: Waiting
05beb89e30dd: Waiting
ba6e5ff31f23: Waiting
862dcfb29807: Waiting
0be7a0def9f1: Waiting
9f9f651e9303: Waiting
11b44f891c78: Waiting
62a747bf1719: Waiting
0b3c02b5d746: Waiting
cb1589008ab5: Pushed
5fa432681ce0: Pushed
494d77973dc2: Pushed
3c78bb10b769: Pushed
b0ad93889d25: Pushed
862dcfb29807: Pushed
a77d4a74a163: Pushed
0be7a0def9f1: Pushed
78700b6b35d0: Layer already exists
62a5b8741e83: Layer already exists
36e0782f1159: Layer already exists
05beb89e30dd: Pushed
ba6e5ff31f23: Layer already exists
c7df4a35b5d2: Pushed
9f9f651e9303: Layer already exists
0b3c02b5d746: Layer already exists
62a747bf1719: Layer already exists
14723a6d7dbf: Pushed
11b44f891c78: Pushed
20211110124335: digest: sha256:c2d20cf75dc21d34c486f24a4c653116372b40d0bd74ddc7dfc361f0dccbd568 size: 4311

> Task :sdks:java:testing:load-tests:run
Nov 10, 2021 12:45:27 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Nov 10, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 197 files. Enable logging at DEBUG level to see which files will be staged.
Nov 10, 2021 12:45:28 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Nov 10, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Nov 10, 2021 12:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 197 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Nov 10, 2021 12:45:31 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 197 files cached, 0 files newly uploaded in 0 seconds
Nov 10, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Nov 10, 2021 12:45:31 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <111704 bytes, hash 095dc21e99473dc0c56fb5da5312918aa98f9b34be7a287e1d2ee79091c48eab> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-CV3CHplHPcDFb7XaUxKRiqmPmzS-eih-HS7nkJHEjqs.pb
Nov 10, 2021 12:45:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Nov 10, 2021 12:45:32 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b676112, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5578be42, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4e49ce2b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@136965e3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@53c6f96d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@435cc7f9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4364712f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1b7a52dd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7f93dd4e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5ad5be4a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ad85136, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@737d100a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12e5da86, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6535117e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1d1cbd0f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6fa13e6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3af7d855, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77049094, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@f88bfbe, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@59bbe88a]
Nov 10, 2021 12:45:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Nov 10, 2021 12:45:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Nov 10, 2021 12:45:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Nov 10, 2021 12:45:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Nov 10, 2021 12:45:32 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4ba6ec50, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@642413d4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@fb2e3fd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43a09ce2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3f183caa, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b66322e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@63538bb4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@24534cb0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5a50d9fc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@106d77da, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6f9c5048, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5114b7c7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@dd71b20, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@767f6ee7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b6c6e70, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@f324455, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3a894088, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@370c1968, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2f0bfe17, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3206174f]
Nov 10, 2021 12:45:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Nov 10, 2021 12:45:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Nov 10, 2021 12:45:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Nov 10, 2021 12:45:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Nov 10, 2021 12:45:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Nov 10, 2021 12:45:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Nov 10, 2021 12:45:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Nov 10, 2021 12:45:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Nov 10, 2021 12:45:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Nov 10, 2021 12:45:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Nov 10, 2021 12:45:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Nov 10, 2021 12:45:33 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.35.0-SNAPSHOT
Nov 10, 2021 12:45:41 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-11-10_04_45_33-10542089246108965070?project=apache-beam-testing
Nov 10, 2021 12:45:41 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-11-10_04_45_33-10542089246108965070
Nov 10, 2021 12:45:41 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-11-10_04_45_33-10542089246108965070
Nov 10, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-11-10T12:45:49.335Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-11-e3id. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Nov 10, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-10T12:45:54.136Z: Worker configuration: e2-standard-2 in us-central1-a.
Nov 10, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-10T12:45:54.854Z: Expanding SplittableParDo operations into optimizable parts.
Nov 10, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-10T12:45:54.891Z: Expanding CollectionToSingleton operations into optimizable parts.
Nov 10, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-10T12:45:54.971Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 10, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-10T12:45:55.037Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Nov 10, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-10T12:45:55.074Z: Expanding GroupByKey operations into streaming Read/Write steps
Nov 10, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-10T12:45:55.150Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 10, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-10T12:45:55.260Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 10, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-10T12:45:55.299Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Nov 10, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-10T12:45:55.336Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Nov 10, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-10T12:45:55.360Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Nov 10, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-10T12:45:55.402Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Nov 10, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-10T12:45:55.449Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Nov 10, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-10T12:45:55.474Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Nov 10, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-10T12:45:55.509Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Nov 10, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-10T12:45:55.549Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Nov 10, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-10T12:45:55.594Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Nov 10, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-10T12:45:55.634Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Nov 10, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-10T12:45:55.680Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Nov 10, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-10T12:45:55.706Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Nov 10, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-10T12:45:55.742Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Nov 10, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-10T12:45:55.779Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Nov 10, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-10T12:45:55.816Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Nov 10, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-10T12:45:55.850Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Nov 10, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-10T12:45:55.874Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Nov 10, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-10T12:45:55.913Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Nov 10, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-10T12:45:55.944Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Nov 10, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-10T12:45:55.981Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Nov 10, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-10T12:45:56.001Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Nov 10, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-10T12:45:56.032Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Nov 10, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-10T12:45:56.401Z: Starting 5 ****s in us-central1-a...
Nov 10, 2021 12:46:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-10T12:46:17.735Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Nov 10, 2021 12:46:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-10T12:46:40.991Z: Autoscaling: Raised the number of ****s to 1 so that the pipeline can catch up with its backlog and keep up with its input rate.
Nov 10, 2021 12:46:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-10T12:46:41.022Z: Resized **** pool to 1, though goal was 5.  This could be a quota issue.
Nov 10, 2021 12:46:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-10T12:46:51.326Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Nov 10, 2021 12:47:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-10T12:47:43.725Z: Workers have started successfully.
Nov 10, 2021 12:47:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-10T12:47:43.761Z: Workers have started successfully.
Nov 10, 2021 4:00:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-10T16:00:31.323Z: Cancel request is committed for workflow job: 2021-11-10_04_45_33-10542089246108965070.
Nov 10, 2021 4:00:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-10T16:00:31.394Z: Cleaning up.
Nov 10, 2021 4:00:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-10T16:00:31.510Z: Stopping **** pool...
Nov 10, 2021 4:00:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-10T16:00:31.575Z: Stopping **** pool...
Nov 10, 2021 4:02:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-10T16:02:44.046Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Nov 10, 2021 4:02:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-10T16:02:44.101Z: Worker pool stopped.
Nov 10, 2021 4:02:50 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-11-10_04_45_33-10542089246108965070 finished with status CANCELLED.
Load test results for test (ID): dba4d411-65aa-4337-98d4-e96e40b93013 and timestamp: 2021-11-10T12:45:27.705000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                  11484.77
dataflow_v2_java11_total_bytes_count              1.7584729E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211110124335
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:c2d20cf75dc21d34c486f24a4c653116372b40d0bd74ddc7dfc361f0dccbd568
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211110124335]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:c2d20cf75dc21d34c486f24a4c653116372b40d0bd74ddc7dfc361f0dccbd568]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211110124335] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:c2d20cf75dc21d34c486f24a4c653116372b40d0bd74ddc7dfc361f0dccbd568])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:c2d20cf75dc21d34c486f24a4c653116372b40d0bd74ddc7dfc361f0dccbd568
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:c2d20cf75dc21d34c486f24a4c653116372b40d0bd74ddc7dfc361f0dccbd568
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:c2d20cf75dc21d34c486f24a4c653116372b40d0bd74ddc7dfc361f0dccbd568].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 33s
101 actionable tasks: 72 executed, 27 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/oahi6vq4jfj32

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #145

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/145/display/redirect?page=changes>

Changes:

[aydar.zaynutdinov] [BEAM-13155][Playground]

[25622840+adude3141] [BEAM-13157] add regression test for hadoop configuration on

[noreply] Merge pull request #15873 from [BEAM-13181] Remove Sharding from

[noreply] Merge pull request #15813 from [BEAM-13071] [Playground] Update run code

[Kyle Weaver] [BEAM-5172] Temporary ignore testSplit and testSizes tests waiting for a


------------------------------------------
[...truncated 49.42 KB...]
ba6e5ff31f23: Layer already exists
9f9f651e9303: Layer already exists
0b3c02b5d746: Layer already exists
62a747bf1719: Layer already exists
0ef7ed11257c: Pushed
06eaf5132039: Pushed
20211109124335: digest: sha256:b1a4ac3e92f33de968d7af6b534ba5552d6a40e93ed30cf44a04c9897b1a8edf size: 4311

> Task :sdks:java:testing:load-tests:run
Nov 09, 2021 12:45:27 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Nov 09, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 197 files. Enable logging at DEBUG level to see which files will be staged.
Nov 09, 2021 12:45:28 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Nov 09, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Nov 09, 2021 12:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 197 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Nov 09, 2021 12:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 197 files cached, 0 files newly uploaded in 0 seconds
Nov 09, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Nov 09, 2021 12:45:31 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <109683 bytes, hash f10bdd28cc35a8d6e1baad6d541d81ffc608693d335b68493f7826586a84afc4> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-8QvdKMw1qNbhuq1tVB2B_8YIaT0zW2hJP3gmWGqEr8Q.pb
Nov 09, 2021 12:45:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Nov 09, 2021 12:45:32 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6ae62c7e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2e869098, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@37c36608, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5d497a91, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@617389a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1c8f6a90, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3050ac2f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@265bd546, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1937eaff, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7e0bc8a3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4b0f2299, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@33063f5b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@15405bd6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@352ed70d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@70730db, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5793b87, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12704e15, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@512575e9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6f1a16fe, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2373ad99]
Nov 09, 2021 12:45:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Nov 09, 2021 12:45:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Nov 09, 2021 12:45:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Nov 09, 2021 12:45:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Nov 09, 2021 12:45:32 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@285c6918, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@78a0ff63, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7c601d50, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@79b2852b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@326d27ac, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4d499d65, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@313f8301, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5cc9d3d0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7c2dfa2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@661d88a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4b0b64cc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@59ce792e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4860827a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@404db674, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@50f097b5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7add838c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3662bdff, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1bb15351, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4fa822ad, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@597f0937]
Nov 09, 2021 12:45:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Nov 09, 2021 12:45:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Nov 09, 2021 12:45:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Nov 09, 2021 12:45:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Nov 09, 2021 12:45:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Nov 09, 2021 12:45:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Nov 09, 2021 12:45:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Nov 09, 2021 12:45:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Nov 09, 2021 12:45:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Nov 09, 2021 12:45:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Nov 09, 2021 12:45:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Nov 09, 2021 12:45:32 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.35.0-SNAPSHOT
Nov 09, 2021 12:45:33 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-11-09_04_45_32-15915895828950689049?project=apache-beam-testing
Nov 09, 2021 12:45:33 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-11-09_04_45_32-15915895828950689049
Nov 09, 2021 12:45:33 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-11-09_04_45_32-15915895828950689049
Nov 09, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-11-09T12:45:40.746Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-11-wf64. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Nov 09, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-09T12:45:47.233Z: Worker configuration: e2-standard-2 in us-central1-a.
Nov 09, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-09T12:45:47.884Z: Expanding SplittableParDo operations into optimizable parts.
Nov 09, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-09T12:45:47.917Z: Expanding CollectionToSingleton operations into optimizable parts.
Nov 09, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-09T12:45:48.019Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 09, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-09T12:45:48.112Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Nov 09, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-09T12:45:48.156Z: Expanding GroupByKey operations into streaming Read/Write steps
Nov 09, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-09T12:45:48.239Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 09, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-09T12:45:48.337Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 09, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-09T12:45:48.394Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Nov 09, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-09T12:45:48.435Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Nov 09, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-09T12:45:48.465Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Nov 09, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-09T12:45:48.499Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Nov 09, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-09T12:45:48.531Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Nov 09, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-09T12:45:48.567Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Nov 09, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-09T12:45:48.619Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Nov 09, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-09T12:45:48.669Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Nov 09, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-09T12:45:48.717Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Nov 09, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-09T12:45:48.752Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Nov 09, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-09T12:45:48.795Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Nov 09, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-09T12:45:48.837Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Nov 09, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-09T12:45:48.887Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Nov 09, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-09T12:45:48.925Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Nov 09, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-09T12:45:48.959Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Nov 09, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-09T12:45:48.998Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Nov 09, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-09T12:45:49.039Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Nov 09, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-09T12:45:49.085Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Nov 09, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-09T12:45:49.121Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Nov 09, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-09T12:45:49.150Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Nov 09, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-09T12:45:49.182Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Nov 09, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-09T12:45:49.221Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Nov 09, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-09T12:45:49.674Z: Starting 5 ****s in us-central1-a...
Nov 09, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-09T12:45:53.163Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Nov 09, 2021 12:46:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-09T12:46:34.900Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Nov 09, 2021 12:47:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-09T12:47:29.985Z: Workers have started successfully.
Nov 09, 2021 12:47:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-09T12:47:30.019Z: Workers have started successfully.
Nov 09, 2021 1:36:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-09T13:36:53.262Z: Staged package kafka-clients-2.4.1-or4Vrm01S3aXE64MuzGE8iWqFvyYIFseskr9n8KjYNk.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/kafka-clients-2.4.1-or4Vrm01S3aXE64MuzGE8iWqFvyYIFseskr9n8KjYNk.jar' is inaccessible.
Nov 09, 2021 1:36:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-09T13:36:53.465Z: Staged package lz4-java-1.6.0-0ilUWqKx1SA8h2YUvbz_yswwNpf0-PJvdk4dbB7S5BY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/lz4-java-1.6.0-0ilUWqKx1SA8h2YUvbz_yswwNpf0-PJvdk4dbB7S5BY.jar' is inaccessible.
Nov 09, 2021 1:36:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-11-09T13:36:54.453Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Nov 09, 2021 1:39:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-11-09T13:39:54.439Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Nov 09, 2021 1:42:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-09T13:42:53.039Z: Staged package kafka-clients-2.4.1-or4Vrm01S3aXE64MuzGE8iWqFvyYIFseskr9n8KjYNk.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/kafka-clients-2.4.1-or4Vrm01S3aXE64MuzGE8iWqFvyYIFseskr9n8KjYNk.jar' is inaccessible.
Nov 09, 2021 1:42:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-09T13:42:53.135Z: Staged package lz4-java-1.6.0-0ilUWqKx1SA8h2YUvbz_yswwNpf0-PJvdk4dbB7S5BY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/lz4-java-1.6.0-0ilUWqKx1SA8h2YUvbz_yswwNpf0-PJvdk4dbB7S5BY.jar' is inaccessible.
Nov 09, 2021 1:42:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-11-09T13:42:54.062Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Nov 09, 2021 1:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-11-09T13:45:54.270Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Nov 09, 2021 1:48:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-09T13:48:53.183Z: Staged package kafka-clients-2.4.1-or4Vrm01S3aXE64MuzGE8iWqFvyYIFseskr9n8KjYNk.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/kafka-clients-2.4.1-or4Vrm01S3aXE64MuzGE8iWqFvyYIFseskr9n8KjYNk.jar' is inaccessible.
Nov 09, 2021 1:48:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-09T13:48:53.313Z: Staged package lz4-java-1.6.0-0ilUWqKx1SA8h2YUvbz_yswwNpf0-PJvdk4dbB7S5BY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/lz4-java-1.6.0-0ilUWqKx1SA8h2YUvbz_yswwNpf0-PJvdk4dbB7S5BY.jar' is inaccessible.
Nov 09, 2021 1:48:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-11-09T13:48:54.273Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Nov 09, 2021 1:51:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-11-09T13:51:54.500Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Nov 09, 2021 1:54:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-09T13:54:53.252Z: Staged package kafka-clients-2.4.1-or4Vrm01S3aXE64MuzGE8iWqFvyYIFseskr9n8KjYNk.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/kafka-clients-2.4.1-or4Vrm01S3aXE64MuzGE8iWqFvyYIFseskr9n8KjYNk.jar' is inaccessible.
Nov 09, 2021 1:54:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-09T13:54:53.349Z: Staged package lz4-java-1.6.0-0ilUWqKx1SA8h2YUvbz_yswwNpf0-PJvdk4dbB7S5BY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/lz4-java-1.6.0-0ilUWqKx1SA8h2YUvbz_yswwNpf0-PJvdk4dbB7S5BY.jar' is inaccessible.
Nov 09, 2021 1:54:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-11-09T13:54:54.299Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Nov 09, 2021 1:57:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-11-09T13:57:54.482Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Nov 09, 2021 2:00:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-09T14:00:53.714Z: Staged package kafka-clients-2.4.1-or4Vrm01S3aXE64MuzGE8iWqFvyYIFseskr9n8KjYNk.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/kafka-clients-2.4.1-or4Vrm01S3aXE64MuzGE8iWqFvyYIFseskr9n8KjYNk.jar' is inaccessible.
Nov 09, 2021 2:00:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-09T14:00:53.832Z: Staged package lz4-java-1.6.0-0ilUWqKx1SA8h2YUvbz_yswwNpf0-PJvdk4dbB7S5BY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/lz4-java-1.6.0-0ilUWqKx1SA8h2YUvbz_yswwNpf0-PJvdk4dbB7S5BY.jar' is inaccessible.
Nov 09, 2021 2:00:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-11-09T14:00:54.936Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Nov 09, 2021 2:03:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-11-09T14:03:54.555Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Nov 09, 2021 3:38:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-09T15:38:03.225Z: Workers have started successfully.
Nov 09, 2021 3:38:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-09T15:38:09.841Z: Worker configuration: e2-standard-2 in us-central1-a.
Nov 09, 2021 3:38:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-09T15:38:27.569Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Nov 09, 2021 4:00:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-09T16:00:34.550Z: Cancel request is committed for workflow job: 2021-11-09_04_45_32-15915895828950689049.
Nov 09, 2021 4:00:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-09T16:00:34.586Z: Cleaning up.
Nov 09, 2021 4:00:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-09T16:00:34.682Z: Stopping **** pool...
Nov 09, 2021 4:00:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-09T16:00:34.745Z: Stopping **** pool...
Nov 09, 2021 4:02:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-09T16:02:53.255Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Nov 09, 2021 4:02:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-09T16:02:53.296Z: Worker pool stopped.
Nov 09, 2021 4:02:59 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-11-09_04_45_32-15915895828950689049 finished with status CANCELLED.
Load test results for test (ID): 7ed0c298-cb29-4d36-87fc-df67b0b3d9a3 and timestamp: 2021-11-09T12:45:27.964000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11553.589
dataflow_v2_java11_total_bytes_count             1.92578252E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211109124335
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:b1a4ac3e92f33de968d7af6b534ba5552d6a40e93ed30cf44a04c9897b1a8edf
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211109124335]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:b1a4ac3e92f33de968d7af6b534ba5552d6a40e93ed30cf44a04c9897b1a8edf]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211109124335] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:b1a4ac3e92f33de968d7af6b534ba5552d6a40e93ed30cf44a04c9897b1a8edf])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:b1a4ac3e92f33de968d7af6b534ba5552d6a40e93ed30cf44a04c9897b1a8edf
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:b1a4ac3e92f33de968d7af6b534ba5552d6a40e93ed30cf44a04c9897b1a8edf
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:b1a4ac3e92f33de968d7af6b534ba5552d6a40e93ed30cf44a04c9897b1a8edf].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 41s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/qrlww2fljv6nk

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #144

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/144/display/redirect>

Changes:


------------------------------------------
[...truncated 48.33 KB...]
c04a1b798a26: Preparing
b71b7c5f32ad: Preparing
183f35830e0e: Preparing
fb344681763e: Preparing
e988ae52c7d1: Preparing
a26246fcdaab: Preparing
5f1e6364ea7f: Preparing
352b42c5ec40: Preparing
09d6d0a70316: Preparing
1874387716a0: Preparing
7a66e17b3e82: Preparing
89ce2f4c7d1a: Preparing
78700b6b35d0: Preparing
62a5b8741e83: Preparing
36e0782f1159: Preparing
ba6e5ff31f23: Preparing
9f9f651e9303: Preparing
0b3c02b5d746: Preparing
5f1e6364ea7f: Waiting
62a747bf1719: Preparing
352b42c5ec40: Waiting
7a66e17b3e82: Waiting
62a5b8741e83: Waiting
09d6d0a70316: Waiting
36e0782f1159: Waiting
1874387716a0: Waiting
89ce2f4c7d1a: Waiting
ba6e5ff31f23: Waiting
78700b6b35d0: Waiting
0b3c02b5d746: Waiting
9f9f651e9303: Waiting
a26246fcdaab: Waiting
62a747bf1719: Waiting
183f35830e0e: Pushed
e988ae52c7d1: Pushed
b71b7c5f32ad: Pushed
fb344681763e: Pushed
c04a1b798a26: Pushed
a26246fcdaab: Pushed
352b42c5ec40: Pushed
09d6d0a70316: Pushed
78700b6b35d0: Layer already exists
62a5b8741e83: Layer already exists
36e0782f1159: Layer already exists
ba6e5ff31f23: Layer already exists
9f9f651e9303: Layer already exists
7a66e17b3e82: Pushed
0b3c02b5d746: Layer already exists
62a747bf1719: Layer already exists
5f1e6364ea7f: Pushed
89ce2f4c7d1a: Pushed
1874387716a0: Pushed
20211108124334: digest: sha256:f5d957cb4f8eab9dfad32286fe173af315dd66545b7e0e7591f374b7a6d4ca3d size: 4311

> Task :sdks:java:testing:load-tests:run
Nov 08, 2021 12:45:27 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Nov 08, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 197 files. Enable logging at DEBUG level to see which files will be staged.
Nov 08, 2021 12:45:28 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Nov 08, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Nov 08, 2021 12:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 197 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Nov 08, 2021 12:45:31 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 197 files cached, 0 files newly uploaded in 0 seconds
Nov 08, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Nov 08, 2021 12:45:31 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <109683 bytes, hash f045e696f2a6625bc71736ec88f7c9aea5b3d9aef59cbb7ce272d76931fada9f> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-8EXmlvKmYlvHFzbsiPfJrqWz2a71nLt84nLXaTH62p8.pb
Nov 08, 2021 12:45:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Nov 08, 2021 12:45:33 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6ae62c7e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2e869098, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@37c36608, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5d497a91, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@617389a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1c8f6a90, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3050ac2f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@265bd546, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1937eaff, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7e0bc8a3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4b0f2299, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@33063f5b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@15405bd6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@352ed70d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@70730db, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5793b87, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12704e15, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@512575e9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6f1a16fe, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2373ad99]
Nov 08, 2021 12:45:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Nov 08, 2021 12:45:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Nov 08, 2021 12:45:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Nov 08, 2021 12:45:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Nov 08, 2021 12:45:33 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@285c6918, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@78a0ff63, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7c601d50, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@79b2852b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@326d27ac, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4d499d65, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@313f8301, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5cc9d3d0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7c2dfa2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@661d88a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4b0b64cc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@59ce792e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4860827a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@404db674, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@50f097b5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7add838c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3662bdff, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1bb15351, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4fa822ad, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@597f0937]
Nov 08, 2021 12:45:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Nov 08, 2021 12:45:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Nov 08, 2021 12:45:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Nov 08, 2021 12:45:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Nov 08, 2021 12:45:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Nov 08, 2021 12:45:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Nov 08, 2021 12:45:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Nov 08, 2021 12:45:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Nov 08, 2021 12:45:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Nov 08, 2021 12:45:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Nov 08, 2021 12:45:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Nov 08, 2021 12:45:33 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.35.0-SNAPSHOT
Nov 08, 2021 12:45:34 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-11-08_04_45_33-7389419280968439881?project=apache-beam-testing
Nov 08, 2021 12:45:34 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-11-08_04_45_33-7389419280968439881
Nov 08, 2021 12:45:34 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-11-08_04_45_33-7389419280968439881
Nov 08, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-11-08T12:45:40.883Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-11-x9nq. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Nov 08, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-08T12:45:45.194Z: Worker configuration: e2-standard-2 in us-central1-a.
Nov 08, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-08T12:45:45.926Z: Expanding SplittableParDo operations into optimizable parts.
Nov 08, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-08T12:45:45.956Z: Expanding CollectionToSingleton operations into optimizable parts.
Nov 08, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-08T12:45:46.025Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 08, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-08T12:45:46.088Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Nov 08, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-08T12:45:46.123Z: Expanding GroupByKey operations into streaming Read/Write steps
Nov 08, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-08T12:45:46.167Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 08, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-08T12:45:46.277Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 08, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-08T12:45:46.314Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Nov 08, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-08T12:45:46.349Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Nov 08, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-08T12:45:46.371Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Nov 08, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-08T12:45:46.404Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Nov 08, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-08T12:45:46.436Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Nov 08, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-08T12:45:46.480Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Nov 08, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-08T12:45:46.513Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Nov 08, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-08T12:45:46.546Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Nov 08, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-08T12:45:46.571Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Nov 08, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-08T12:45:46.607Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Nov 08, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-08T12:45:46.627Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Nov 08, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-08T12:45:46.652Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Nov 08, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-08T12:45:46.676Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Nov 08, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-08T12:45:46.702Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Nov 08, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-08T12:45:46.725Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Nov 08, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-08T12:45:46.757Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Nov 08, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-08T12:45:46.791Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Nov 08, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-08T12:45:46.824Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Nov 08, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-08T12:45:46.857Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Nov 08, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-08T12:45:46.892Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Nov 08, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-08T12:45:46.927Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Nov 08, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-08T12:45:46.960Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Nov 08, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-08T12:45:47.314Z: Starting 5 ****s in us-central1-a...
Nov 08, 2021 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-08T12:46:18.977Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Nov 08, 2021 12:46:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-08T12:46:32.171Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Nov 08, 2021 12:47:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-08T12:47:28.920Z: Workers have started successfully.
Nov 08, 2021 12:47:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-08T12:47:28.967Z: Workers have started successfully.
Nov 08, 2021 4:00:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-08T16:00:32.838Z: Cancel request is committed for workflow job: 2021-11-08_04_45_33-7389419280968439881.
Nov 08, 2021 4:00:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-08T16:00:32.921Z: Cleaning up.
Nov 08, 2021 4:00:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-08T16:00:33.000Z: Stopping **** pool...
Nov 08, 2021 4:00:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-08T16:00:33.114Z: Stopping **** pool...
Nov 08, 2021 4:02:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-08T16:02:55.783Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Nov 08, 2021 4:02:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-08T16:02:55.836Z: Worker pool stopped.
Nov 08, 2021 4:03:02 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-11-08_04_45_33-7389419280968439881 finished with status CANCELLED.
Load test results for test (ID): 4a6b3be3-5b65-48e9-b0c3-3a2ca45dbbdd and timestamp: 2021-11-08T12:45:27.995000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11553.556
dataflow_v2_java11_total_bytes_count             1.80117038E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211108124334
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:f5d957cb4f8eab9dfad32286fe173af315dd66545b7e0e7591f374b7a6d4ca3d
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211108124334]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:f5d957cb4f8eab9dfad32286fe173af315dd66545b7e0e7591f374b7a6d4ca3d]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211108124334] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:f5d957cb4f8eab9dfad32286fe173af315dd66545b7e0e7591f374b7a6d4ca3d])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:f5d957cb4f8eab9dfad32286fe173af315dd66545b7e0e7591f374b7a6d4ca3d
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:f5d957cb4f8eab9dfad32286fe173af315dd66545b7e0e7591f374b7a6d4ca3d
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:f5d957cb4f8eab9dfad32286fe173af315dd66545b7e0e7591f374b7a6d4ca3d].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 46s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/5ndm6qzatdbxm

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 - Build # 143 - Aborted!

Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 - Build # 143 - Aborted:

Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/143/ to view the results.

Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #142

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/142/display/redirect?page=changes>

Changes:

[noreply] [adhoc] Speedup slow tests for AWS IO modules (#15899)

[noreply] [BEAM-12566] Implement set_axis for DataFrame and Series (#15773)

[noreply] [BEAM-13192] Fix buggy retry tests for AWS SnsIO (#15910)

[noreply] [BEAM-11217] Metrics Query filtering for DoFn metrics. (#15887)

[noreply] [Go SDK] Go SDK Exits Experimental (#15894)

[joseinigo] [BEAM-13080] Fix number of default keys

[joseinigo] [BEAM-13080] Fix number of default keys


------------------------------------------
[...truncated 48.60 KB...]
b54e21fc98b6: Preparing
80513f585ca8: Preparing
b306fc598105: Preparing
e1ea39e25414: Preparing
2c253a1ecb3d: Preparing
e823c04abb62: Preparing
f8506eb6d448: Preparing
66dbdac24a88: Preparing
78700b6b35d0: Preparing
62a5b8741e83: Preparing
36e0782f1159: Preparing
ba6e5ff31f23: Preparing
9f9f651e9303: Preparing
0b3c02b5d746: Preparing
62a747bf1719: Preparing
e823c04abb62: Waiting
2c253a1ecb3d: Waiting
e1ea39e25414: Waiting
f8506eb6d448: Waiting
80513f585ca8: Waiting
66dbdac24a88: Waiting
b306fc598105: Waiting
9f9f651e9303: Waiting
36e0782f1159: Waiting
78700b6b35d0: Waiting
0b3c02b5d746: Waiting
62a5b8741e83: Waiting
ba6e5ff31f23: Waiting
62a747bf1719: Waiting
b54e21fc98b6: Pushed
effff550acfb: Pushed
999d8bb81cd1: Pushed
79f8b53e4f15: Pushed
c966e6346529: Pushed
80513f585ca8: Pushed
e1ea39e25414: Pushed
2c253a1ecb3d: Pushed
78700b6b35d0: Layer already exists
62a5b8741e83: Layer already exists
36e0782f1159: Layer already exists
ba6e5ff31f23: Layer already exists
b306fc598105: Pushed
9f9f651e9303: Layer already exists
62a747bf1719: Layer already exists
0b3c02b5d746: Layer already exists
66dbdac24a88: Pushed
f8506eb6d448: Pushed
e823c04abb62: Pushed
20211106124333: digest: sha256:3767c712d2b32b7250bfe8e82e722d8cb59e058c2022992b6e73288bb2a27063 size: 4311

> Task :sdks:java:testing:load-tests:run
Nov 06, 2021 12:45:32 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Nov 06, 2021 12:45:32 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 197 files. Enable logging at DEBUG level to see which files will be staged.
Nov 06, 2021 12:45:33 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Nov 06, 2021 12:45:33 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Nov 06, 2021 12:45:36 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 197 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Nov 06, 2021 12:45:36 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 197 files cached, 0 files newly uploaded in 0 seconds
Nov 06, 2021 12:45:36 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Nov 06, 2021 12:45:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <109683 bytes, hash e0f99d2369055f65369016f95c646e02ac6e57f4db3c0df472216533e8cdd873> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-4PmdI2kFX2U2kBb5XGRuAqxuV_TbPA30ciFlM-jN2HM.pb
Nov 06, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Nov 06, 2021 12:45:38 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6ae62c7e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2e869098, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@37c36608, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5d497a91, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@617389a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1c8f6a90, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3050ac2f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@265bd546, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1937eaff, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7e0bc8a3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4b0f2299, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@33063f5b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@15405bd6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@352ed70d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@70730db, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5793b87, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12704e15, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@512575e9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6f1a16fe, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2373ad99]
Nov 06, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Nov 06, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Nov 06, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Nov 06, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Nov 06, 2021 12:45:38 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@285c6918, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@78a0ff63, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7c601d50, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@79b2852b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@326d27ac, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4d499d65, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@313f8301, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5cc9d3d0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7c2dfa2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@661d88a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4b0b64cc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@59ce792e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4860827a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@404db674, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@50f097b5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7add838c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3662bdff, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1bb15351, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4fa822ad, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@597f0937]
Nov 06, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Nov 06, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Nov 06, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Nov 06, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Nov 06, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Nov 06, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Nov 06, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Nov 06, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Nov 06, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Nov 06, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Nov 06, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Nov 06, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.35.0-SNAPSHOT
Nov 06, 2021 12:45:39 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-11-06_05_45_38-12108461136062105938?project=apache-beam-testing
Nov 06, 2021 12:45:39 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-11-06_05_45_38-12108461136062105938
Nov 06, 2021 12:45:39 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-11-06_05_45_38-12108461136062105938
Nov 06, 2021 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-11-06T12:45:45.142Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-11-omxk. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Nov 06, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-06T12:45:50.029Z: Worker configuration: e2-standard-2 in us-central1-a.
Nov 06, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-06T12:45:50.555Z: Expanding SplittableParDo operations into optimizable parts.
Nov 06, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-06T12:45:50.586Z: Expanding CollectionToSingleton operations into optimizable parts.
Nov 06, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-06T12:45:50.653Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 06, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-06T12:45:50.728Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Nov 06, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-06T12:45:50.753Z: Expanding GroupByKey operations into streaming Read/Write steps
Nov 06, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-06T12:45:50.821Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 06, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-06T12:45:50.914Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 06, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-06T12:45:50.951Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Nov 06, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-06T12:45:50.983Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Nov 06, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-06T12:45:51.006Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Nov 06, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-06T12:45:51.041Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Nov 06, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-06T12:45:51.067Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Nov 06, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-06T12:45:51.098Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Nov 06, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-06T12:45:51.120Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Nov 06, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-06T12:45:51.157Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Nov 06, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-06T12:45:51.192Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Nov 06, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-06T12:45:51.225Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Nov 06, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-06T12:45:51.251Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Nov 06, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-06T12:45:51.285Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Nov 06, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-06T12:45:51.314Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Nov 06, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-06T12:45:51.348Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Nov 06, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-06T12:45:51.378Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Nov 06, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-06T12:45:51.399Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Nov 06, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-06T12:45:51.435Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Nov 06, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-06T12:45:51.494Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Nov 06, 2021 12:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-06T12:45:51.521Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Nov 06, 2021 12:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-06T12:45:51.550Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Nov 06, 2021 12:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-06T12:45:51.574Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Nov 06, 2021 12:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-06T12:45:51.608Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Nov 06, 2021 12:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-06T12:45:52.072Z: Starting 5 ****s in us-central1-a...
Nov 06, 2021 12:46:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-06T12:46:09.614Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Nov 06, 2021 12:46:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-06T12:46:37.836Z: Autoscaling: Raised the number of ****s to 3 so that the pipeline can catch up with its backlog and keep up with its input rate.
Nov 06, 2021 12:46:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-06T12:46:37.861Z: Resized **** pool to 3, though goal was 5.  This could be a quota issue.
Nov 06, 2021 12:46:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-06T12:46:48.163Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Nov 06, 2021 12:47:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-06T12:47:36.437Z: Workers have started successfully.
Nov 06, 2021 12:47:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-06T12:47:36.463Z: Workers have started successfully.
Nov 06, 2021 4:00:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-06T16:00:36.044Z: Cancel request is committed for workflow job: 2021-11-06_05_45_38-12108461136062105938.
Nov 06, 2021 4:00:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-06T16:00:36.108Z: Cleaning up.
Nov 06, 2021 4:00:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-06T16:00:36.176Z: Stopping **** pool...
Nov 06, 2021 4:00:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-06T16:00:36.229Z: Stopping **** pool...
Nov 06, 2021 4:03:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-06T16:03:26.712Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Nov 06, 2021 4:03:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-06T16:03:26.749Z: Worker pool stopped.
Nov 06, 2021 4:03:34 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-11-06_05_45_38-12108461136062105938 finished with status CANCELLED.
Load test results for test (ID): fe22fbdc-a19f-469e-adaa-aac694f6e264 and timestamp: 2021-11-06T12:45:33.244000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11523.914
dataflow_v2_java11_total_bytes_count             2.67386971E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211106124333
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:3767c712d2b32b7250bfe8e82e722d8cb59e058c2022992b6e73288bb2a27063
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211106124333]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:3767c712d2b32b7250bfe8e82e722d8cb59e058c2022992b6e73288bb2a27063]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211106124333] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:3767c712d2b32b7250bfe8e82e722d8cb59e058c2022992b6e73288bb2a27063])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:3767c712d2b32b7250bfe8e82e722d8cb59e058c2022992b6e73288bb2a27063
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:3767c712d2b32b7250bfe8e82e722d8cb59e058c2022992b6e73288bb2a27063
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:3767c712d2b32b7250bfe8e82e722d8cb59e058c2022992b6e73288bb2a27063].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 20m 18s
101 actionable tasks: 71 executed, 28 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/pryongyp37moa

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #141

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/141/display/redirect?page=changes>

Changes:

[heejong] [BEAM-13021] Deduplicate Python artifact not only by hash but also by

[heejong] add test

[heejong] rename variables

[Luke Cwik] [BEAM-3811] Code clean-up of the CancelleableQueue to not throw

[heejong] rearrange comparison order

[Luke Cwik] [BEAM-13164] Address most of a race condition between instantiation and

[Robert Bradshaw] Defer filesToStage construction until after full jar resolution.

[heejong] s/pathes/paths/

[noreply] Merge pull request #15835: [BEAM-11205] Google Libraries BOM 24.0.0 and

[noreply] Merge pull request #15810: [BEAM-2791] Support low-latency StorageApi

[noreply] [BEAM-13081] Fixes a compatible issue of decoding null-value bitmap


------------------------------------------
[...truncated 49.66 KB...]
ba6e5ff31f23: Waiting
0b3c02b5d746: Waiting
379f31d19858: Waiting
eaf5029db1fb: Waiting
36e0782f1159: Waiting
fabf3b6d80fc: Pushed
a7c2e9d18f2a: Pushed
be23f8775916: Pushed
fe9147d6e839: Pushed
2eae6dabe400: Pushed
ad122fac4193: Pushed
eaf5029db1fb: Pushed
379f31d19858: Pushed
78700b6b35d0: Layer already exists
e967acfd455f: Pushed
62a5b8741e83: Layer already exists
36e0782f1159: Layer already exists
ba6e5ff31f23: Layer already exists
9f9f651e9303: Layer already exists
6ab784edc93e: Pushed
0b3c02b5d746: Layer already exists
62a747bf1719: Layer already exists
89278d4876d4: Pushed
e12820d25c5b: Pushed
20211105124339: digest: sha256:7c124b4b6a5f6433e10fb4a36068ef12a9045b1acb9378583a7864d4128fb619 size: 4311

> Task :sdks:java:testing:load-tests:run
Nov 05, 2021 12:45:44 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Nov 05, 2021 12:45:45 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 197 files. Enable logging at DEBUG level to see which files will be staged.
Nov 05, 2021 12:45:46 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Nov 05, 2021 12:45:46 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Nov 05, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 197 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Nov 05, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 197 files cached, 0 files newly uploaded in 0 seconds
Nov 05, 2021 12:45:50 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Nov 05, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <109683 bytes, hash 53e97eec66dbdbeaed8338582d934c1ee4c30162f04d98769a3bade2b65c5a3f> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-U-l-7Gbb2-rtgzhYLZNMHuTDAWLwTZh2mjut4rZcWj8.pb
Nov 05, 2021 12:45:51 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Nov 05, 2021 12:45:51 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@37c36608, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5d497a91, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@617389a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1c8f6a90, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3050ac2f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@265bd546, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1937eaff, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7e0bc8a3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4b0f2299, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@33063f5b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@15405bd6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@352ed70d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@70730db, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5793b87, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12704e15, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@512575e9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6f1a16fe, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2373ad99, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@33634f04, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4993febc]
Nov 05, 2021 12:45:51 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Nov 05, 2021 12:45:51 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Nov 05, 2021 12:45:51 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Nov 05, 2021 12:45:51 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Nov 05, 2021 12:45:51 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7c601d50, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@79b2852b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@326d27ac, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4d499d65, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@313f8301, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5cc9d3d0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7c2dfa2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@661d88a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4b0b64cc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@59ce792e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4860827a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@404db674, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@50f097b5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7add838c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3662bdff, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1bb15351, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4fa822ad, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@597f0937, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7ad1caa2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6b6b3572]
Nov 05, 2021 12:45:51 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Nov 05, 2021 12:45:51 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Nov 05, 2021 12:45:51 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Nov 05, 2021 12:45:52 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Nov 05, 2021 12:45:52 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Nov 05, 2021 12:45:52 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Nov 05, 2021 12:45:52 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Nov 05, 2021 12:45:52 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Nov 05, 2021 12:45:52 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Nov 05, 2021 12:45:52 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Nov 05, 2021 12:45:52 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Nov 05, 2021 12:45:52 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.35.0-SNAPSHOT
Nov 05, 2021 12:45:52 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-11-05_05_45_52-15779773588238686739?project=apache-beam-testing
Nov 05, 2021 12:45:52 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-11-05_05_45_52-15779773588238686739
Nov 05, 2021 12:45:52 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-11-05_05_45_52-15779773588238686739
Nov 05, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-11-05T12:46:00.636Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-11-f0m2. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Nov 05, 2021 12:46:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-05T12:46:05.064Z: Worker configuration: e2-standard-2 in us-central1-a.
Nov 05, 2021 12:46:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-05T12:46:05.705Z: Expanding SplittableParDo operations into optimizable parts.
Nov 05, 2021 12:46:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-05T12:46:05.738Z: Expanding CollectionToSingleton operations into optimizable parts.
Nov 05, 2021 12:46:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-05T12:46:05.800Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 05, 2021 12:46:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-05T12:46:05.869Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Nov 05, 2021 12:46:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-05T12:46:05.896Z: Expanding GroupByKey operations into streaming Read/Write steps
Nov 05, 2021 12:46:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-05T12:46:05.966Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 05, 2021 12:46:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-05T12:46:06.080Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 05, 2021 12:46:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-05T12:46:06.111Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Nov 05, 2021 12:46:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-05T12:46:06.135Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Nov 05, 2021 12:46:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-05T12:46:06.174Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Nov 05, 2021 12:46:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-05T12:46:06.201Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Nov 05, 2021 12:46:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-05T12:46:06.222Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Nov 05, 2021 12:46:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-05T12:46:06.246Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Nov 05, 2021 12:46:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-05T12:46:06.282Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Nov 05, 2021 12:46:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-05T12:46:06.311Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Nov 05, 2021 12:46:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-05T12:46:06.344Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Nov 05, 2021 12:46:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-05T12:46:06.365Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Nov 05, 2021 12:46:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-05T12:46:06.390Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Nov 05, 2021 12:46:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-05T12:46:06.412Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Nov 05, 2021 12:46:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-05T12:46:06.432Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Nov 05, 2021 12:46:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-05T12:46:06.464Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Nov 05, 2021 12:46:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-05T12:46:06.497Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Nov 05, 2021 12:46:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-05T12:46:06.530Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Nov 05, 2021 12:46:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-05T12:46:06.563Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Nov 05, 2021 12:46:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-05T12:46:06.591Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Nov 05, 2021 12:46:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-05T12:46:06.620Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Nov 05, 2021 12:46:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-05T12:46:06.687Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Nov 05, 2021 12:46:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-05T12:46:06.745Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Nov 05, 2021 12:46:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-05T12:46:06.825Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Nov 05, 2021 12:46:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-05T12:46:07.363Z: Starting 5 ****s in us-central1-a...
Nov 05, 2021 12:46:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-05T12:46:26.364Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Nov 05, 2021 12:46:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-05T12:46:58.100Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Nov 05, 2021 12:47:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-05T12:47:52.174Z: Workers have started successfully.
Nov 05, 2021 12:47:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-05T12:47:52.239Z: Workers have started successfully.
Nov 05, 2021 4:00:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-05T16:00:40.085Z: Cancel request is committed for workflow job: 2021-11-05_05_45_52-15779773588238686739.
Nov 05, 2021 4:00:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-05T16:00:40.154Z: Cleaning up.
Nov 05, 2021 4:00:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-05T16:00:40.263Z: Stopping **** pool...
Nov 05, 2021 4:00:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-05T16:00:40.310Z: Stopping **** pool...
Nov 05, 2021 4:02:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-05T16:02:57.894Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Nov 05, 2021 4:02:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-05T16:02:57.928Z: Worker pool stopped.
Nov 05, 2021 4:03:06 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-11-05_05_45_52-15779773588238686739 finished with status CANCELLED.
Load test results for test (ID): fdcfcbf3-53b2-4585-abe6-ae67c4eaec3b and timestamp: 2021-11-05T12:45:45.675000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11545.383
dataflow_v2_java11_total_bytes_count             2.06464136E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211105124339
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:7c124b4b6a5f6433e10fb4a36068ef12a9045b1acb9378583a7864d4128fb619
Deleted: sha256:eb95e0312ce127d3b9e8f8485f70e5368fb345db908e57c7d46ea92d3d5c8091
Deleted: sha256:db1ccc2876864aea6b16892a501b24c7326915438d014040506157ad8b5c56de
Deleted: sha256:f410e6253d7aa3b307b1e3bd2dc67794e5817bf4f0618d6b07aaad87eefa56c0
Deleted: sha256:7aec492d546a60b380484e7dbd800744e71f3f8477317a7619b24eb1a84da9d5
Deleted: sha256:6dcc2cbe661ad50072415c7cec4a6e263ae43572b8b27554e8e9ab453556a77d
Deleted: sha256:65da22f6d1b7d948392bf88292c4f90c7147dfc6a29850ff9706d5fb6fda9fb6
Deleted: sha256:67d81025792b008c535cbf03f13b3477ab7a7b0c88496d9810ed1d89f2d4e040
Deleted: sha256:539fd19ece85bd38f0923f31e8c6fbb1809d3a17dcb8dabaddac242bb8577d5f
Deleted: sha256:591038e15cda4123dc7518edfc8367161c86d75a160410b2cf0f5b7ae3d23a49
Deleted: sha256:bd4d7b484734e4db207d5b844e79900da4867f2f8974e44a0bbe2e31b9a11ee2
Deleted: sha256:db57a098f64e865ef93269b97848973156e108fe69938f254404bdd9b705b5fc
Deleted: sha256:21738747e6f0793337ff0d241e85f3ebf58e8fc9ab43fc9bef93919f76e46582
Deleted: sha256:91c2d19acdd1e1b10a7d7ef38b85c16deac8d8e8b2a19324534eb376662a897a
Deleted: sha256:4d5b0cc523b2100f02312d099d6ab08ac2928c1fd6f996799d42212767e9ba07
Deleted: sha256:948f43fde6337fb52f36de81e8df7fe0acbf849e71d030233d2bf15b8c2ac2d1
Deleted: sha256:51edd63245e6a97eba5e8dc4705d08e7d4e7a7945a6d99c56269144aa31ed41f
Deleted: sha256:d3a83747995e35445c4e0be18dc442dfaa3f528cee9319f0a27ad4ad40aa7adc
Deleted: sha256:c97c3fecea151198a661ed096e616dc129ce034bfe1c94b1cc808f1539fa5773
Deleted: sha256:4cda81f2c6642e6eee1dc682ce57c945b2f8a4dfa499efcb37d6454b008fcc56
Deleted: sha256:5ea5b897c81f256fd999bfed9d6fba8c3821b58da93a92cee3af61aff1541582
Deleted: sha256:57855ec99b059f068754a683fb257e90eceee4d90d27a3ac71f814fde051f411
Deleted: sha256:e361b4f486f4d8dafc7f03401a9aaf862894cd8539f211363ef234465ed0ab83
Deleted: sha256:e70a67bfa95bf22ea7e0dc62e79e03645645e0ccbcc9e8adddf22cfb7d44a0f9
Deleted: sha256:d8454c7162a6893e31c95b19f828e27e60c7e73e0a240086c2ae022e05b6e797
Deleted: sha256:d09c1f1dabf3e70f651607b87b74ffd45634cee733f2878d92bcd25d4d88cca5
Deleted: sha256:fb3fd05eff157eb2b14ba630a1afbea81dae6600afba98dc18642b82101dbc74
Deleted: sha256:188c8bb34ba2be4e4a372caf7278cea5787996678973d369ba4c5b71c0b5e7c4
Deleted: sha256:ad1ab5724299fb8f9e7190385ca7d78d2fe0080969089571b4c49bfeac0fedb1
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211105124339]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:7c124b4b6a5f6433e10fb4a36068ef12a9045b1acb9378583a7864d4128fb619]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211105124339] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:7c124b4b6a5f6433e10fb4a36068ef12a9045b1acb9378583a7864d4128fb619])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:7c124b4b6a5f6433e10fb4a36068ef12a9045b1acb9378583a7864d4128fb619
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:7c124b4b6a5f6433e10fb4a36068ef12a9045b1acb9378583a7864d4128fb619
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:7c124b4b6a5f6433e10fb4a36068ef12a9045b1acb9378583a7864d4128fb619].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 48s
101 actionable tasks: 72 executed, 27 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/2jwgvbmclkgme

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #140

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/140/display/redirect?page=changes>

Changes:

[noreply] Add missing parentheses for Python test example

[noreply] Update test input and imports

[stranniknm] [BEAM-13105] playground - add shorcut hint tooltip

[mmack] [BEAM-11440] Add integration test for KinesisIO using Localstack on

[mmack] Bump localstack container version for more robust version with respect

[mmack] Bump localstack container version for DynamoDBIOTest to version used in

[mmack] Apply format

[mmack] Apply format

[mmack] Use vendored guava deps

[25622840+adude3141] [BEAM-13157] support hadoop configuration on ParquetIO.Parse

[Pablo Estrada] JdbcIO has a single WriteFn underlying all implementations

[dpcollins] Performance improvement to PubSubLiteIO to not use a streaming committer

[noreply] Merge pull request #15854 from [BEAM-13046][Playground] protobuf

[noreply] [BEAM-12550] Parallelizable skew Implementation  (#15809)

[noreply] Merge pull request #15782 from [BEAM-13034] [Playground] add semantics

[hlagosperez] Use BigQuieryIO.loadProjectId  in WriteRename class to create

[noreply] [BEAM-13099] Use vendored Calcite 1.28.0 in SQL extensions (#15836)

[Luke Cwik] [BEAM-13015] Use a network based channel instead of an inmemory one

[Pablo Estrada] Addressing comments

[noreply] Merge pull request #15839 from [BEAM-13041][Playground] Prepare files

[noreply] [BEAM-13001] updated CHANGES.md to include msec counter for Go (#15872)

[noreply] Add window mapping to CHANGES.md (#15871)

[noreply] Merge pull request #15784 from [BEAM-8135]  - Removing

[noreply] [BEAM-13119] Subdirectory prefix tag for Go SDK (#15881)

[noreply] Merge pull request #15852 from [BEAM-13102] [Playground] update

[Kyle Weaver] Fix typo: s/spark/twister2


------------------------------------------
[...truncated 232.95 KB...]
Nov 04, 2021 3:49:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-04T15:49:13.732Z: Staged package metrics-core-3.1.2-JFuipmqbxxDOTbFHERJud7y05tlu9-YiZZKA88kMu1w.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/metrics-core-3.1.2-JFuipmqbxxDOTbFHERJud7y05tlu9-YiZZKA88kMu1w.jar' is inaccessible.
Nov 04, 2021 3:49:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-04T15:49:14.075Z: Staged package proto-google-cloud-bigquerystorage-v1-2.1.0--M8Hmi82bEAs_rlZfGOmOJeniDwYV9GXd4uYuCIzxIg.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/proto-google-cloud-bigquerystorage-v1-2.1.0--M8Hmi82bEAs_rlZfGOmOJeniDwYV9GXd4uYuCIzxIg.jar' is inaccessible.
Nov 04, 2021 3:49:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-04T15:49:14.112Z: Staged package proto-google-cloud-bigquerystorage-v1beta1-0.125.0-qKRRBewW8WdLWXjsY9uGABgBUBdfwqmhSwqFkWRQV1Q.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/proto-google-cloud-bigquerystorage-v1beta1-0.125.0-qKRRBewW8WdLWXjsY9uGABgBUBdfwqmhSwqFkWRQV1Q.jar' is inaccessible.
Nov 04, 2021 3:49:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-04T15:49:14.173Z: Staged package proto-google-cloud-bigquerystorage-v1beta2-0.125.0-JSL7XosXdUS5ZGraL_j5Dnbf6dY04ZrPTdxB_T_NsNc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/proto-google-cloud-bigquerystorage-v1beta2-0.125.0-JSL7XosXdUS5ZGraL_j5Dnbf6dY04ZrPTdxB_T_NsNc.jar' is inaccessible.
Nov 04, 2021 3:49:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-04T15:49:14.209Z: Staged package proto-google-cloud-bigtable-admin-v2-2.1.0-Oha95NdNmvlfAh6KTLXm9FygpotYhLsS7Cb837ZbIJo.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/proto-google-cloud-bigtable-admin-v2-2.1.0-Oha95NdNmvlfAh6KTLXm9FygpotYhLsS7Cb837ZbIJo.jar' is inaccessible.
Nov 04, 2021 3:49:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-04T15:49:14.246Z: Staged package proto-google-cloud-bigtable-v2-2.1.0-BA4Xw2RDAfoRzQCX0ac5VopZzaDym4fSBY0Msbm8txM.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/proto-google-cloud-bigtable-v2-2.1.0-BA4Xw2RDAfoRzQCX0ac5VopZzaDym4fSBY0Msbm8txM.jar' is inaccessible.
Nov 04, 2021 3:49:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-04T15:49:14.285Z: Staged package proto-google-cloud-datastore-v1-0.91.3-vS4xd5n4PaDd-Kyt6_V3G8YGqO_QMcuShghC9MLpSns.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/proto-google-cloud-datastore-v1-0.91.3-vS4xd5n4PaDd-Kyt6_V3G8YGqO_QMcuShghC9MLpSns.jar' is inaccessible.
Nov 04, 2021 3:49:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-04T15:49:14.332Z: Staged package proto-google-cloud-firestore-bundle-v1-3.0.2-vr6gpK2KOX9hYv8tzTr4-RsKC2t_wRcUKeLYKqLm2oQ.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/proto-google-cloud-firestore-bundle-v1-3.0.2-vr6gpK2KOX9hYv8tzTr4-RsKC2t_wRcUKeLYKqLm2oQ.jar' is inaccessible.
Nov 04, 2021 3:49:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-04T15:49:14.386Z: Staged package proto-google-cloud-firestore-v1-3.0.2-e7pBKXoawrVbDlBanNo5GpdjMFfUfMropSFYGm-RA6I.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/proto-google-cloud-firestore-v1-3.0.2-e7pBKXoawrVbDlBanNo5GpdjMFfUfMropSFYGm-RA6I.jar' is inaccessible.
Nov 04, 2021 3:49:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-04T15:49:14.438Z: Staged package proto-google-cloud-pubsub-v1-1.96.2-Hd5TFaDg9u0PWkcRBAM7lW0P0qLQO4y5u-JvPoVBYsQ.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/proto-google-cloud-pubsub-v1-1.96.2-Hd5TFaDg9u0PWkcRBAM7lW0P0qLQO4y5u-JvPoVBYsQ.jar' is inaccessible.
Nov 04, 2021 3:49:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-04T15:49:14.668Z: Staged package proto-google-cloud-spanner-admin-database-v1-6.12.1-i6wlp_PYFwGObAmINb8JcCzH8AbDpzYSVJyQTC4R8EA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/proto-google-cloud-spanner-admin-database-v1-6.12.1-i6wlp_PYFwGObAmINb8JcCzH8AbDpzYSVJyQTC4R8EA.jar' is inaccessible.
Nov 04, 2021 3:49:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-04T15:49:14.713Z: Staged package proto-google-cloud-spanner-admin-instance-v1-6.12.1-BFVz7rtnZfKL0xslKUB_uKkNZhJX-EMm-G8-R_A1hQA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/proto-google-cloud-spanner-admin-instance-v1-6.12.1-BFVz7rtnZfKL0xslKUB_uKkNZhJX-EMm-G8-R_A1hQA.jar' is inaccessible.
Nov 04, 2021 3:49:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-04T15:49:14.763Z: Staged package proto-google-cloud-spanner-v1-6.12.1-s4xvKx28n2TpL6buW2o5wxJ_2nxIGjAN4WHs_WXhCu0.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/proto-google-cloud-spanner-v1-6.12.1-s4xvKx28n2TpL6buW2o5wxJ_2nxIGjAN4WHs_WXhCu0.jar' is inaccessible.
Nov 04, 2021 3:49:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-04T15:49:14.830Z: Staged package proto-google-common-protos-2.3.2-maqGpeUt_1i-QQtcMJBbLqmiyZWofiN_FgWQlTxmG20.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/proto-google-common-protos-2.3.2-maqGpeUt_1i-QQtcMJBbLqmiyZWofiN_FgWQlTxmG20.jar' is inaccessible.
Nov 04, 2021 3:49:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-04T15:49:14.869Z: Staged package proto-google-iam-v1-1.0.14-5yLo1nFyqHrRtb07oCxnhISVgZDEnAWUlEU7B-fqXco.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/proto-google-iam-v1-1.0.14-5yLo1nFyqHrRtb07oCxnhISVgZDEnAWUlEU7B-fqXco.jar' is inaccessible.
Nov 04, 2021 3:49:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-04T15:49:14.908Z: Staged package protobuf-java-3.17.3-SsVJsZJpQUGVgEnwYKHIJqMzQvYZ4QjO2MF9mHf14-0.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/protobuf-java-3.17.3-SsVJsZJpQUGVgEnwYKHIJqMzQvYZ4QjO2MF9mHf14-0.jar' is inaccessible.
Nov 04, 2021 3:49:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-04T15:49:14.954Z: Staged package protobuf-java-util-3.17.3-vzIO0HYADh2MfL92AbBWrK7KuA91uaZZufY5jQ1-P3k.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/protobuf-java-util-3.17.3-vzIO0HYADh2MfL92AbBWrK7KuA91uaZZufY5jQ1-P3k.jar' is inaccessible.
Nov 04, 2021 3:49:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-11-04T15:49:15.218Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Nov 04, 2021 3:52:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-11-04T15:52:15.286Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Nov 04, 2021 3:55:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-04T15:55:08.198Z: Staged package api-common-2.0.1-g8yqFlAvDwxYbosBUrWoK3ZxW93EQzdvb03u7M2bUvE.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/api-common-2.0.1-g8yqFlAvDwxYbosBUrWoK3ZxW93EQzdvb03u7M2bUvE.jar' is inaccessible.
Nov 04, 2021 3:55:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-04T15:55:08.991Z: Staged package bigtable-client-core-1.23.1-xSDVAE9Xpy4qeW4--vFAYxs7AZ8EspP5_1fdu3yIbio.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/bigtable-client-core-1.23.1-xSDVAE9Xpy4qeW4--vFAYxs7AZ8EspP5_1fdu3yIbio.jar' is inaccessible.
Nov 04, 2021 3:55:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-04T15:55:09.038Z: Staged package bigtable-metrics-api-1.23.1-TdnTs_hKYdupIMUGd2wTnqTzdMVhrfwI5ciftUxIsTI.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/bigtable-metrics-api-1.23.1-TdnTs_hKYdupIMUGd2wTnqTzdMVhrfwI5ciftUxIsTI.jar' is inaccessible.
Nov 04, 2021 3:55:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-04T15:55:09.220Z: Staged package commons-codec-1.15-s-n21jp5AQm_DQVmEfvtHPaQVYJt7-uYlKcTadJG7WM.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/commons-codec-1.15-s-n21jp5AQm_DQVmEfvtHPaQVYJt7-uYlKcTadJG7WM.jar' is inaccessible.
Nov 04, 2021 3:55:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-04T15:55:09.602Z: Staged package gax-2.3.0-eMOD-8BQkNf6hb3BUVVu0XlCJ625EV96u1iWoYhJDkE.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/gax-2.3.0-eMOD-8BQkNf6hb3BUVVu0XlCJ625EV96u1iWoYhJDkE.jar' is inaccessible.
Nov 04, 2021 3:55:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-04T15:55:09.653Z: Staged package gax-grpc-2.3.0-go68qqPEG0mDW_ITj2MCF6xg1flj0dbLagP6N-meEcE.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/gax-grpc-2.3.0-go68qqPEG0mDW_ITj2MCF6xg1flj0dbLagP6N-meEcE.jar' is inaccessible.
Nov 04, 2021 3:55:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-04T15:55:09.698Z: Staged package gax-httpjson-0.88.0-lHShTFD8TLYZ6Og3AlxdxDNaz-i-zOzHxXH621Tfr7I.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/gax-httpjson-0.88.0-lHShTFD8TLYZ6Og3AlxdxDNaz-i-zOzHxXH621Tfr7I.jar' is inaccessible.
Nov 04, 2021 3:55:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-04T15:55:09.774Z: Staged package google-api-client-1.32.1-d63CrvrOT8kqaYuvoPi6txarBRuyHLQQYA9d4qfmsw4.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-api-client-1.32.1-d63CrvrOT8kqaYuvoPi6txarBRuyHLQQYA9d4qfmsw4.jar' is inaccessible.
Nov 04, 2021 3:55:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-04T15:55:09.828Z: Staged package google-api-client-jackson2-1.32.1-IHUWP1-fG6iVCC3WHq3xXUQivJT6Do-E9uGIMkVniWI.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-api-client-jackson2-1.32.1-IHUWP1-fG6iVCC3WHq3xXUQivJT6Do-E9uGIMkVniWI.jar' is inaccessible.
Nov 04, 2021 3:55:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-04T15:55:09.901Z: Staged package google-api-services-bigquery-v2-rev20210813-1.32.1-7nw9MuSxBly9ryv9a52-E2PAUKahjZ3uDSELjX39KWY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-api-services-bigquery-v2-rev20210813-1.32.1-7nw9MuSxBly9ryv9a52-E2PAUKahjZ3uDSELjX39KWY.jar' is inaccessible.
Nov 04, 2021 3:55:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-04T15:55:09.950Z: Staged package google-api-services-clouddebugger-v2-rev20210813-1.32.1-tA91lCc58HMAryr4ZImcEVZcXxRVeUcBPdhsp2hGRLg.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-api-services-clouddebugger-v2-rev20210813-1.32.1-tA91lCc58HMAryr4ZImcEVZcXxRVeUcBPdhsp2hGRLg.jar' is inaccessible.
Nov 04, 2021 3:55:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-04T15:55:10.005Z: Staged package google-api-services-cloudresourcemanager-v1-rev20210815-1.32.1-jjiHPuwSpPBmqZ5Jn8DVFaSBpRRY1kfbzRQj37XWFYo.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-api-services-cloudresourcemanager-v1-rev20210815-1.32.1-jjiHPuwSpPBmqZ5Jn8DVFaSBpRRY1kfbzRQj37XWFYo.jar' is inaccessible.
Nov 04, 2021 3:55:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-04T15:55:10.083Z: Staged package google-api-services-dataflow-v1b3-rev20210818-1.32.1-BnHDq1zo3n0xyJIGrH3BrFg_puCQ1gikxyFLKnc01BA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-api-services-dataflow-v1b3-rev20210818-1.32.1-BnHDq1zo3n0xyJIGrH3BrFg_puCQ1gikxyFLKnc01BA.jar' is inaccessible.
Nov 04, 2021 3:55:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-04T15:55:10.127Z: Staged package google-api-services-healthcare-v1-rev20210806-1.32.1-SnySyLXRzcgtws4uL-LdxBk9pXHKSkqALaUpVX24lCk.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-api-services-healthcare-v1-rev20210806-1.32.1-SnySyLXRzcgtws4uL-LdxBk9pXHKSkqALaUpVX24lCk.jar' is inaccessible.
Nov 04, 2021 3:55:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-04T15:55:10.207Z: Staged package google-api-services-pubsub-v1-rev20210809-1.32.1-Tr6gARITJUuN7ntkiL7S7o0i4J7WSYy-wzGrzZKhk2Y.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-api-services-pubsub-v1-rev20210809-1.32.1-Tr6gARITJUuN7ntkiL7S7o0i4J7WSYy-wzGrzZKhk2Y.jar' is inaccessible.
Nov 04, 2021 3:55:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-04T15:55:10.248Z: Staged package google-api-services-storage-v1-rev20210127-1.32.1-wjvrBbuEKr7RTI_XWj5rei9RsNPs4ZA6nF_Tq3cK2T4.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-api-services-storage-v1-rev20210127-1.32.1-wjvrBbuEKr7RTI_XWj5rei9RsNPs4ZA6nF_Tq3cK2T4.jar' is inaccessible.
Nov 04, 2021 3:55:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-04T15:55:10.370Z: Staged package google-auth-library-credentials-1.1.0-bpL3c79EMdarUbVBnhFrDdMakBZ35P-W3M5I9PK3Z3g.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-auth-library-credentials-1.1.0-bpL3c79EMdarUbVBnhFrDdMakBZ35P-W3M5I9PK3Z3g.jar' is inaccessible.
Nov 04, 2021 3:55:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-04T15:55:10.524Z: Staged package google-auth-library-oauth2-http-1.1.0-wj6_4-5nU0FD-eJQlSbU7vu4BtTOUYB1SmxyH6lBwYY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-auth-library-oauth2-http-1.1.0-wj6_4-5nU0FD-eJQlSbU7vu4BtTOUYB1SmxyH6lBwYY.jar' is inaccessible.
Nov 04, 2021 3:55:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-04T15:55:10.684Z: Staged package google-cloud-bigquery-2.1.4-acRfQOzlj8IvHdpxBQzjvS62zx7fJX4QTffgCZNrmY0.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-cloud-bigquery-2.1.4-acRfQOzlj8IvHdpxBQzjvS62zx7fJX4QTffgCZNrmY0.jar' is inaccessible.
Nov 04, 2021 3:55:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-04T15:55:10.779Z: Staged package google-cloud-bigquerystorage-2.1.0-rTRf2sdrbajSOilVmucmxLU9KIb3JYthfFcGKp7Tlfw.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-cloud-bigquerystorage-2.1.0-rTRf2sdrbajSOilVmucmxLU9KIb3JYthfFcGKp7Tlfw.jar' is inaccessible.
Nov 04, 2021 3:55:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-04T15:55:10.889Z: Staged package google-cloud-bigtable-2.1.0-CK5V53ksW3WZ9J4Bld4e1RwmvP3YnfFt5ftg3_Li5Z0.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-cloud-bigtable-2.1.0-CK5V53ksW3WZ9J4Bld4e1RwmvP3YnfFt5ftg3_Li5Z0.jar' is inaccessible.
Nov 04, 2021 3:55:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-04T15:55:10.989Z: Staged package google-cloud-core-2.1.0-V-_lolxckO-O1jfRQnLh7h3dFdZzvH61DyWVuUtsobc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-cloud-core-2.1.0-V-_lolxckO-O1jfRQnLh7h3dFdZzvH61DyWVuUtsobc.jar' is inaccessible.
Nov 04, 2021 3:55:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-04T15:55:11.042Z: Staged package google-cloud-core-grpc-2.1.0-5vUOMd-3N4ETe4Usmj7x_SySNTbkMqFNyVKZBe2YEVc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-cloud-core-grpc-2.1.0-5vUOMd-3N4ETe4Usmj7x_SySNTbkMqFNyVKZBe2YEVc.jar' is inaccessible.
Nov 04, 2021 3:55:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-04T15:55:11.107Z: Staged package google-cloud-core-http-2.1.0-RGVkQiv87L4ZbuLar_XW_aYIoGpacuBbYwLg1QLedYo.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-cloud-core-http-2.1.0-RGVkQiv87L4ZbuLar_XW_aYIoGpacuBbYwLg1QLedYo.jar' is inaccessible.
Nov 04, 2021 3:55:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-04T15:55:11.169Z: Staged package google-cloud-firestore-3.0.2-tKeDy091f9U7vlOalQx0yqjzXhBiwT4GkrjzBW2BCws.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-cloud-firestore-3.0.2-tKeDy091f9U7vlOalQx0yqjzXhBiwT4GkrjzBW2BCws.jar' is inaccessible.
Nov 04, 2021 3:55:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-04T15:55:11.216Z: Staged package google-cloud-pubsub-1.114.2-4kM0U8-6GadXz75swlcehVyseIl4rSzBPW0XvlAlEB8.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-cloud-pubsub-1.114.2-4kM0U8-6GadXz75swlcehVyseIl4rSzBPW0XvlAlEB8.jar' is inaccessible.
Nov 04, 2021 3:55:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-04T15:55:11.298Z: Staged package google-cloud-spanner-6.12.1-9r3EtK4WZZ485rU7NAfKI7xhEyFZ1ApnZ7gAUElnMe4.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-cloud-spanner-6.12.1-9r3EtK4WZZ485rU7NAfKI7xhEyFZ1ApnZ7gAUElnMe4.jar' is inaccessible.
Nov 04, 2021 3:55:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-04T15:55:11.497Z: Staged package grpc-alts-1.40.0-YwW5drbCb3_Wyf9z2n6bFZ5X1T7_FigR7di02444Sk0.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/grpc-alts-1.40.0-YwW5drbCb3_Wyf9z2n6bFZ5X1T7_FigR7di02444Sk0.jar' is inaccessible.
Nov 04, 2021 3:55:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-04T15:55:11.555Z: Staged package grpc-api-1.40.0-6JlsF6D_ZmXDRj9oACWaN1WqPUhjxdUXN7k7EegYoL0.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/grpc-api-1.40.0-6JlsF6D_ZmXDRj9oACWaN1WqPUhjxdUXN7k7EegYoL0.jar' is inaccessible.
Nov 04, 2021 3:55:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-04T15:55:11.605Z: Staged package grpc-auth-1.40.0-PoW7NsnBJ2_XWeC17cLgxQx16n0N7jZaneUMJPIz9UM.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/grpc-auth-1.40.0-PoW7NsnBJ2_XWeC17cLgxQx16n0N7jZaneUMJPIz9UM.jar' is inaccessible.
Nov 04, 2021 3:55:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-04T15:55:11.660Z: Staged package grpc-context-1.40.0-MYgqv87MjQnKh6T1FEFMOr4NjNKmKzeSSetW1j7bmXQ.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/grpc-context-1.40.0-MYgqv87MjQnKh6T1FEFMOr4NjNKmKzeSSetW1j7bmXQ.jar' is inaccessible.
Nov 04, 2021 3:55:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-04T15:55:11.711Z: Staged package grpc-core-1.40.0-jXEll3JqBHjtCl4FzFZi4aa3ue--LVhdQ8lH7JQnW4s.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/grpc-core-1.40.0-jXEll3JqBHjtCl4FzFZi4aa3ue--LVhdQ8lH7JQnW4s.jar' is inaccessible.
Nov 04, 2021 3:55:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-04T15:55:11.766Z: Staged package grpc-gcp-1.1.0-zKq--BAvLo2Cks8hjNm7iNASsOWFeFMqwc55hcc1YEY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/grpc-gcp-1.1.0-zKq--BAvLo2Cks8hjNm7iNASsOWFeFMqwc55hcc1YEY.jar' is inaccessible.
Nov 04, 2021 3:55:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-04T15:55:11.836Z: Staged package grpc-google-cloud-bigquerystorage-v1-2.1.0-pmqLBn9jJX6cMBQNRZqEOdSbrNClk757IV6iPV2sc-E.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/grpc-google-cloud-bigquerystorage-v1-2.1.0-pmqLBn9jJX6cMBQNRZqEOdSbrNClk757IV6iPV2sc-E.jar' is inaccessible.
Nov 04, 2021 3:55:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-04T15:55:11.880Z: Staged package grpc-google-cloud-bigquerystorage-v1beta1-0.125.0-lie7DrgZCGZoIdynaS5DtII9wwrPhUmrclc0mC1pRJ0.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/grpc-google-cloud-bigquerystorage-v1beta1-0.125.0-lie7DrgZCGZoIdynaS5DtII9wwrPhUmrclc0mC1pRJ0.jar' is inaccessible.
Nov 04, 2021 3:55:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-04T15:55:11.935Z: Staged package grpc-google-cloud-bigquerystorage-v1beta2-0.125.0-zOXapRF30ZRCTchndCTgYjr297vPZNoLFuJiRrg4rqE.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/grpc-google-cloud-bigquerystorage-v1beta2-0.125.0-zOXapRF30ZRCTchndCTgYjr297vPZNoLFuJiRrg4rqE.jar' is inaccessible.
Nov 04, 2021 3:55:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-04T15:55:11.973Z: Staged package grpc-google-cloud-bigtable-admin-v2-2.1.0-O9M5vtkkhXZ1hJGrv61OGCaET60iZucVgfx6SVbGMo8.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/grpc-google-cloud-bigtable-admin-v2-2.1.0-O9M5vtkkhXZ1hJGrv61OGCaET60iZucVgfx6SVbGMo8.jar' is inaccessible.
Nov 04, 2021 3:55:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-04T15:55:12.026Z: Staged package grpc-google-cloud-bigtable-v2-2.1.0-xRpHCbcaKPpU2xjJ4uKK1sDd2cpQMum0QacDJEpb2wg.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/grpc-google-cloud-bigtable-v2-2.1.0-xRpHCbcaKPpU2xjJ4uKK1sDd2cpQMum0QacDJEpb2wg.jar' is inaccessible.
Nov 04, 2021 3:55:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-04T15:55:12.082Z: Staged package grpc-google-cloud-pubsub-v1-1.96.2-LSfgPfeyJMuoujfLiVJGuAcqKQh1TjDhBNFwA-L3s1I.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/grpc-google-cloud-pubsub-v1-1.96.2-LSfgPfeyJMuoujfLiVJGuAcqKQh1TjDhBNFwA-L3s1I.jar' is inaccessible.
Nov 04, 2021 3:55:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-04T15:55:12.147Z: Staged package grpc-google-cloud-spanner-admin-database-v1-6.12.1-unPYqI58YhKv_6mc6T6YQgqBcvtRPSdGpo1hXi2mQQc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/grpc-google-cloud-spanner-admin-database-v1-6.12.1-unPYqI58YhKv_6mc6T6YQgqBcvtRPSdGpo1hXi2mQQc.jar' is inaccessible.
Nov 04, 2021 3:55:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-04T15:55:12.199Z: Staged package grpc-google-cloud-spanner-admin-instance-v1-6.12.1-vVVDW3Mzs5VrMu-ONFpIupQowCosX0kgctveASt1T_M.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/grpc-google-cloud-spanner-admin-instance-v1-6.12.1-vVVDW3Mzs5VrMu-ONFpIupQowCosX0kgctveASt1T_M.jar' is inaccessible.
Nov 04, 2021 3:55:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-04T15:55:12.244Z: Staged package grpc-google-cloud-spanner-v1-6.12.1-sTZfm0NLp3Dt4c16Ot_ONtv-qQgCVvddmp8P56UjCgs.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/grpc-google-cloud-spanner-v1-6.12.1-sTZfm0NLp3Dt4c16Ot_ONtv-qQgCVvddmp8P56UjCgs.jar' is inaccessible.
Nov 04, 2021 3:55:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-04T15:55:12.315Z: Staged package grpc-google-common-protos-2.3.2-027cVEqmfFtfw0r7Y8evdsYc_NET1FNnWGqtGva-nn8.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/grpc-google-common-protos-2.3.2-027cVEqmfFtfw0r7Y8evdsYc_NET1FNnWGqtGva-nn8.jar' is inaccessible.
Nov 04, 2021 3:55:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-04T15:55:12.358Z: Staged package grpc-grpclb-1.40.0-3x_pdrKP1JFP5mkjhs5ctpBStgFD0Av1ls3HyVryo0M.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/grpc-grpclb-1.40.0-3x_pdrKP1JFP5mkjhs5ctpBStgFD0Av1ls3HyVryo0M.jar' is inaccessible.
Nov 04, 2021 3:55:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-04T15:55:12.411Z: Staged package grpc-netty-1.40.0-KI8g3du3gxngQ-zUfmn47sHGrI-vrl_7QxMVy8tV0R0.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/grpc-netty-1.40.0-KI8g3du3gxngQ-zUfmn47sHGrI-vrl_7QxMVy8tV0R0.jar' is inaccessible.
Nov 04, 2021 3:55:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-04T15:55:12.474Z: Staged package grpc-netty-shaded-1.40.0-bfX9Qgk5ZMZ_L1OTR0Izgy70u2hNSvIXKHItGr9w9Z4.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/grpc-netty-shaded-1.40.0-bfX9Qgk5ZMZ_L1OTR0Izgy70u2hNSvIXKHItGr9w9Z4.jar' is inaccessible.
Nov 04, 2021 3:55:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-04T15:55:12.524Z: Staged package grpc-protobuf-1.40.0-9lmDVCdqFREyDkUqGEg3MmMsmnOiNyuewKZsmoJI8pg.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/grpc-protobuf-1.40.0-9lmDVCdqFREyDkUqGEg3MmMsmnOiNyuewKZsmoJI8pg.jar' is inaccessible.
Nov 04, 2021 3:55:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-04T15:55:12.562Z: Staged package grpc-protobuf-lite-1.40.1-wX_o9Paj_2Dy3BxRGYFcW8vMOWfmCrHlOCckWeT9aac.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/grpc-protobuf-lite-1.40.1-wX_o9Paj_2Dy3BxRGYFcW8vMOWfmCrHlOCckWeT9aac.jar' is inaccessible.
Nov 04, 2021 3:55:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-04T15:55:12.605Z: Staged package grpc-stub-1.40.0--7XO3mWD78nDt0upNPSfu4LJ8OX52rRbz7Lxg1wFRcs.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/grpc-stub-1.40.0--7XO3mWD78nDt0upNPSfu4LJ8OX52rRbz7Lxg1wFRcs.jar' is inaccessible.
Nov 04, 2021 3:55:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-04T15:55:12.669Z: Staged package guava-30.1.1-jre-RM4inOJtiAvzr8Niu_zsNNfmkD0ZW7sdufO24NmDTwY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/guava-30.1.1-jre-RM4inOJtiAvzr8Niu_zsNNfmkD0ZW7sdufO24NmDTwY.jar' is inaccessible.
Nov 04, 2021 3:55:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-04T15:55:12.766Z: Staged package httpclient-4.5.13-b-kCalZsalABYIzz_DIZZkH2weXhmG0QN8zb1fMe90M.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/httpclient-4.5.13-b-kCalZsalABYIzz_DIZZkH2weXhmG0QN8zb1fMe90M.jar' is inaccessible.
Nov 04, 2021 3:55:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-04T15:55:12.846Z: Staged package httpcore-4.4.14--VYgnkUMsdDFF3bfvSPlPp3Y25oSmO1itwvwlEumOyg.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/httpcore-4.4.14--VYgnkUMsdDFF3bfvSPlPp3Y25oSmO1itwvwlEumOyg.jar' is inaccessible.
Nov 04, 2021 3:55:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-04T15:55:13.006Z: Staged package jackson-annotations-2.12.4-9qo3Bqh1aJtmzawzNPZd_beVzPrUEXvwcok7GW7R7I4.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/jackson-annotations-2.12.4-9qo3Bqh1aJtmzawzNPZd_beVzPrUEXvwcok7GW7R7I4.jar' is inaccessible.
Nov 04, 2021 3:55:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-04T15:55:13.052Z: Staged package jackson-core-2.12.4-NQbOR-wmBK4tgNeVBffLN09xgGBjlBXAfRRK2t0taKM.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/jackson-core-2.12.4-NQbOR-wmBK4tgNeVBffLN09xgGBjlBXAfRRK2t0taKM.jar' is inaccessible.
Nov 04, 2021 3:55:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-04T15:55:13.155Z: Staged package jackson-databind-2.12.4-6Zp7S4kHS8aJqrzZ6x8sExi2jMXDSXna8-NO3FWMegE.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/jackson-databind-2.12.4-6Zp7S4kHS8aJqrzZ6x8sExi2jMXDSXna8-NO3FWMegE.jar' is inaccessible.
Nov 04, 2021 3:55:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-04T15:55:13.221Z: Staged package jackson-dataformat-yaml-2.12.4--CQtUTyPlCxyPdbUU6o76-9mn6Y4PKEIIDPvr5FbTdQ.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/jackson-dataformat-yaml-2.12.4--CQtUTyPlCxyPdbUU6o76-9mn6Y4PKEIIDPvr5FbTdQ.jar' is inaccessible.
Nov 04, 2021 3:55:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-04T15:55:13.684Z: Staged package metrics-core-3.1.2-JFuipmqbxxDOTbFHERJud7y05tlu9-YiZZKA88kMu1w.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/metrics-core-3.1.2-JFuipmqbxxDOTbFHERJud7y05tlu9-YiZZKA88kMu1w.jar' is inaccessible.
Nov 04, 2021 3:55:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-04T15:55:14.219Z: Staged package proto-google-cloud-bigquerystorage-v1-2.1.0--M8Hmi82bEAs_rlZfGOmOJeniDwYV9GXd4uYuCIzxIg.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/proto-google-cloud-bigquerystorage-v1-2.1.0--M8Hmi82bEAs_rlZfGOmOJeniDwYV9GXd4uYuCIzxIg.jar' is inaccessible.
Nov 04, 2021 3:55:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-04T15:55:14.259Z: Staged package proto-google-cloud-bigquerystorage-v1beta1-0.125.0-qKRRBewW8WdLWXjsY9uGABgBUBdfwqmhSwqFkWRQV1Q.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/proto-google-cloud-bigquerystorage-v1beta1-0.125.0-qKRRBewW8WdLWXjsY9uGABgBUBdfwqmhSwqFkWRQV1Q.jar' is inaccessible.
Nov 04, 2021 3:55:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-04T15:55:14.330Z: Staged package proto-google-cloud-bigquerystorage-v1beta2-0.125.0-JSL7XosXdUS5ZGraL_j5Dnbf6dY04ZrPTdxB_T_NsNc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/proto-google-cloud-bigquerystorage-v1beta2-0.125.0-JSL7XosXdUS5ZGraL_j5Dnbf6dY04ZrPTdxB_T_NsNc.jar' is inaccessible.
Nov 04, 2021 3:55:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-04T15:55:14.368Z: Staged package proto-google-cloud-bigtable-admin-v2-2.1.0-Oha95NdNmvlfAh6KTLXm9FygpotYhLsS7Cb837ZbIJo.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/proto-google-cloud-bigtable-admin-v2-2.1.0-Oha95NdNmvlfAh6KTLXm9FygpotYhLsS7Cb837ZbIJo.jar' is inaccessible.
Nov 04, 2021 3:55:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-04T15:55:14.418Z: Staged package proto-google-cloud-bigtable-v2-2.1.0-BA4Xw2RDAfoRzQCX0ac5VopZzaDym4fSBY0Msbm8txM.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/proto-google-cloud-bigtable-v2-2.1.0-BA4Xw2RDAfoRzQCX0ac5VopZzaDym4fSBY0Msbm8txM.jar' is inaccessible.
Nov 04, 2021 3:55:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-04T15:55:14.466Z: Staged package proto-google-cloud-datastore-v1-0.91.3-vS4xd5n4PaDd-Kyt6_V3G8YGqO_QMcuShghC9MLpSns.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/proto-google-cloud-datastore-v1-0.91.3-vS4xd5n4PaDd-Kyt6_V3G8YGqO_QMcuShghC9MLpSns.jar' is inaccessible.
Nov 04, 2021 3:55:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-04T15:55:14.514Z: Staged package proto-google-cloud-firestore-bundle-v1-3.0.2-vr6gpK2KOX9hYv8tzTr4-RsKC2t_wRcUKeLYKqLm2oQ.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/proto-google-cloud-firestore-bundle-v1-3.0.2-vr6gpK2KOX9hYv8tzTr4-RsKC2t_wRcUKeLYKqLm2oQ.jar' is inaccessible.
Nov 04, 2021 3:55:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-04T15:55:14.554Z: Staged package proto-google-cloud-firestore-v1-3.0.2-e7pBKXoawrVbDlBanNo5GpdjMFfUfMropSFYGm-RA6I.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/proto-google-cloud-firestore-v1-3.0.2-e7pBKXoawrVbDlBanNo5GpdjMFfUfMropSFYGm-RA6I.jar' is inaccessible.
Nov 04, 2021 3:55:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-04T15:55:14.602Z: Staged package proto-google-cloud-pubsub-v1-1.96.2-Hd5TFaDg9u0PWkcRBAM7lW0P0qLQO4y5u-JvPoVBYsQ.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/proto-google-cloud-pubsub-v1-1.96.2-Hd5TFaDg9u0PWkcRBAM7lW0P0qLQO4y5u-JvPoVBYsQ.jar' is inaccessible.
Nov 04, 2021 3:55:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-04T15:55:14.665Z: Staged package proto-google-cloud-spanner-admin-database-v1-6.12.1-i6wlp_PYFwGObAmINb8JcCzH8AbDpzYSVJyQTC4R8EA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/proto-google-cloud-spanner-admin-database-v1-6.12.1-i6wlp_PYFwGObAmINb8JcCzH8AbDpzYSVJyQTC4R8EA.jar' is inaccessible.
Nov 04, 2021 3:55:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-04T15:55:14.714Z: Staged package proto-google-cloud-spanner-admin-instance-v1-6.12.1-BFVz7rtnZfKL0xslKUB_uKkNZhJX-EMm-G8-R_A1hQA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/proto-google-cloud-spanner-admin-instance-v1-6.12.1-BFVz7rtnZfKL0xslKUB_uKkNZhJX-EMm-G8-R_A1hQA.jar' is inaccessible.
Nov 04, 2021 3:55:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-04T15:55:14.757Z: Staged package proto-google-cloud-spanner-v1-6.12.1-s4xvKx28n2TpL6buW2o5wxJ_2nxIGjAN4WHs_WXhCu0.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/proto-google-cloud-spanner-v1-6.12.1-s4xvKx28n2TpL6buW2o5wxJ_2nxIGjAN4WHs_WXhCu0.jar' is inaccessible.
Nov 04, 2021 3:55:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-04T15:55:14.810Z: Staged package proto-google-common-protos-2.3.2-maqGpeUt_1i-QQtcMJBbLqmiyZWofiN_FgWQlTxmG20.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/proto-google-common-protos-2.3.2-maqGpeUt_1i-QQtcMJBbLqmiyZWofiN_FgWQlTxmG20.jar' is inaccessible.
Nov 04, 2021 3:55:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-04T15:55:14.846Z: Staged package proto-google-iam-v1-1.0.14-5yLo1nFyqHrRtb07oCxnhISVgZDEnAWUlEU7B-fqXco.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/proto-google-iam-v1-1.0.14-5yLo1nFyqHrRtb07oCxnhISVgZDEnAWUlEU7B-fqXco.jar' is inaccessible.
Nov 04, 2021 3:55:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-04T15:55:14.902Z: Staged package protobuf-java-3.17.3-SsVJsZJpQUGVgEnwYKHIJqMzQvYZ4QjO2MF9mHf14-0.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/protobuf-java-3.17.3-SsVJsZJpQUGVgEnwYKHIJqMzQvYZ4QjO2MF9mHf14-0.jar' is inaccessible.
Nov 04, 2021 3:55:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-04T15:55:14.949Z: Staged package protobuf-java-util-3.17.3-vzIO0HYADh2MfL92AbBWrK7KuA91uaZZufY5jQ1-P3k.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/protobuf-java-util-3.17.3-vzIO0HYADh2MfL92AbBWrK7KuA91uaZZufY5jQ1-P3k.jar' is inaccessible.
Nov 04, 2021 3:55:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-11-04T15:55:15.229Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Nov 04, 2021 3:58:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-11-04T15:58:12.604Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Nov 04, 2021 4:00:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-04T16:00:37.494Z: Cancel request is committed for workflow job: 2021-11-04_05_45_34-7043171245915014881.
Nov 04, 2021 4:00:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-04T16:00:37.556Z: Cleaning up.
Nov 04, 2021 4:00:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-04T16:00:37.637Z: Stopping **** pool...
Nov 04, 2021 4:00:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-04T16:00:37.681Z: Stopping **** pool...
Nov 04, 2021 4:02:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-04T16:02:54.634Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Nov 04, 2021 4:02:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-04T16:02:54.684Z: Worker pool stopped.
Nov 04, 2021 4:03:02 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-11-04_05_45_34-7043171245915014881 finished with status CANCELLED.
Load test results for test (ID): 1bde0e43-cd5b-4b7f-adcd-ae88e876fc0a and timestamp: 2021-11-04T12:45:29.078000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11477.704
dataflow_v2_java11_total_bytes_count             1.52906431E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211104124333
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:6c5da4ee64ea8d299cc29d8f441c82f5069a9c3d4a58e4c56e2947aa39c43766
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211104124333]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:6c5da4ee64ea8d299cc29d8f441c82f5069a9c3d4a58e4c56e2947aa39c43766]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211104124333] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:6c5da4ee64ea8d299cc29d8f441c82f5069a9c3d4a58e4c56e2947aa39c43766])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:6c5da4ee64ea8d299cc29d8f441c82f5069a9c3d4a58e4c56e2947aa39c43766
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:6c5da4ee64ea8d299cc29d8f441c82f5069a9c3d4a58e4c56e2947aa39c43766
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:6c5da4ee64ea8d299cc29d8f441c82f5069a9c3d4a58e4c56e2947aa39c43766].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 47s
101 actionable tasks: 71 executed, 28 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/5maektvgqyias

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure


---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #139

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/139/display/redirect?page=changes>

Changes:

[Andrew Pilloud] [BEAM-13118] Use branch for nightly "tag", tags are immutable

[dmitrii_kuzin] [BEAM-12730] Python. Custom delimiter add corner case

[Alexey Romanenko] [BEAM-12070] ParquetIO: use splittable reading by default

[noreply] [BEAM-13001] fixes nil reference error in Extractor.ExtractFrom (#15865)

[noreply] Merge pull request #15842 from [BEAM-13125][Playground] Update

[noreply] Merge pull request #15838 from [BEAM-13127] [Playground] Implement TCP

[noreply] Merge pull request #15832 from [BEAM-13149] Changing readWithPartitions

[noreply] Minor: add resource key to presentation materials link (#15867)


------------------------------------------
[...truncated 48.69 KB...]
2dd1db7da1dd: Preparing
53c9017b1be0: Preparing
55df7bbac955: Preparing
4235ea42d291: Preparing
a15f236935d4: Preparing
78700b6b35d0: Preparing
2dd1db7da1dd: Waiting
62a5b8741e83: Preparing
53c9017b1be0: Waiting
36e0782f1159: Preparing
55df7bbac955: Waiting
ba6e5ff31f23: Preparing
4235ea42d291: Waiting
9f9f651e9303: Preparing
0b3c02b5d746: Preparing
60268ebf88f8: Waiting
62a747bf1719: Preparing
62a5b8741e83: Waiting
ba6e5ff31f23: Waiting
36e0782f1159: Waiting
9f9f651e9303: Waiting
62a747bf1719: Waiting
0b3c02b5d746: Waiting
1ff23cddd2eb: Waiting
a15f236935d4: Waiting
97d68374f77d: Pushed
36bf3ba895ba: Pushed
922e1df58ed7: Pushed
71cfa21f6028: Pushed
20ebdb385def: Pushed
60268ebf88f8: Pushed
2dd1db7da1dd: Pushed
53c9017b1be0: Pushed
78700b6b35d0: Layer already exists
62a5b8741e83: Layer already exists
4235ea42d291: Pushed
36e0782f1159: Layer already exists
1ff23cddd2eb: Pushed
ba6e5ff31f23: Layer already exists
9f9f651e9303: Layer already exists
0b3c02b5d746: Layer already exists
a15f236935d4: Pushed
62a747bf1719: Layer already exists
55df7bbac955: Pushed
20211103124333: digest: sha256:df3ff75fba7ea1b2b418e87b8486098e715472f701f3e15685db6a4557f26568 size: 4311

> Task :sdks:java:testing:load-tests:run
Nov 03, 2021 12:45:38 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Nov 03, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 195 files. Enable logging at DEBUG level to see which files will be staged.
Nov 03, 2021 12:45:39 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Nov 03, 2021 12:45:39 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Nov 03, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 195 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Nov 03, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 195 files cached, 0 files newly uploaded in 0 seconds
Nov 03, 2021 12:45:41 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Nov 03, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <108949 bytes, hash 860a91ac9ed28a235533986fd6cfa0e4f6cbefd47ea851fe7dad903d12c82c90> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-hgqRrJ7SiiNVM5hv1s-g5PbL79R-qFH-fa2QPRLILJA.pb
Nov 03, 2021 12:45:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Nov 03, 2021 12:45:43 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@33425811, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4b74b35, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@e4e1ef5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6d11ceef, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4cb2918c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@72e295cc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@c2584d3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6fa0450e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@37468787, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@51ec2856, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@714f3da4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1caa9eb6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1f53481b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2fcd7d3f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@27e7c77f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6f70a21b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6ae62c7e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2e869098, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@37c36608, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5d497a91]
Nov 03, 2021 12:45:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Nov 03, 2021 12:45:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Nov 03, 2021 12:45:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Nov 03, 2021 12:45:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Nov 03, 2021 12:45:43 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@28bdbe88, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6a87026, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@ef60710, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@600f5704, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2503ec73, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@606f81b5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5e1fc42f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@44b21f9f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7ee8130e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6296474f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4288d98e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@169268a7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@285c6918, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@78a0ff63, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7c601d50, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@79b2852b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@326d27ac, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4d499d65, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@313f8301, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5cc9d3d0]
Nov 03, 2021 12:45:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Nov 03, 2021 12:45:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Nov 03, 2021 12:45:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Nov 03, 2021 12:45:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Nov 03, 2021 12:45:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Nov 03, 2021 12:45:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Nov 03, 2021 12:45:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Nov 03, 2021 12:45:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Nov 03, 2021 12:45:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Nov 03, 2021 12:45:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Nov 03, 2021 12:45:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Nov 03, 2021 12:45:43 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.35.0-SNAPSHOT
Nov 03, 2021 12:45:44 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-11-03_05_45_43-6875680358526166536?project=apache-beam-testing
Nov 03, 2021 12:45:44 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-11-03_05_45_43-6875680358526166536
Nov 03, 2021 12:45:44 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-11-03_05_45_43-6875680358526166536
Nov 03, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-11-03T12:45:50.887Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-11-glln. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Nov 03, 2021 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-03T12:45:57.153Z: Worker configuration: e2-standard-2 in us-central1-a.
Nov 03, 2021 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-03T12:45:57.808Z: Expanding SplittableParDo operations into optimizable parts.
Nov 03, 2021 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-03T12:45:57.839Z: Expanding CollectionToSingleton operations into optimizable parts.
Nov 03, 2021 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-03T12:45:57.904Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 03, 2021 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-03T12:45:57.965Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Nov 03, 2021 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-03T12:45:57.987Z: Expanding GroupByKey operations into streaming Read/Write steps
Nov 03, 2021 12:46:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-03T12:45:58.071Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 03, 2021 12:46:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-03T12:45:58.171Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 03, 2021 12:46:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-03T12:45:58.210Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Nov 03, 2021 12:46:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-03T12:45:58.235Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Nov 03, 2021 12:46:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-03T12:45:58.259Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Nov 03, 2021 12:46:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-03T12:45:58.294Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Nov 03, 2021 12:46:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-03T12:45:58.328Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Nov 03, 2021 12:46:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-03T12:45:58.373Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Nov 03, 2021 12:46:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-03T12:45:58.396Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Nov 03, 2021 12:46:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-03T12:45:58.428Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Nov 03, 2021 12:46:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-03T12:45:58.461Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Nov 03, 2021 12:46:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-03T12:45:58.496Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Nov 03, 2021 12:46:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-03T12:45:58.539Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Nov 03, 2021 12:46:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-03T12:45:58.571Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Nov 03, 2021 12:46:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-03T12:45:58.615Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Nov 03, 2021 12:46:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-03T12:45:58.649Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Nov 03, 2021 12:46:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-03T12:45:58.684Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Nov 03, 2021 12:46:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-03T12:45:58.718Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Nov 03, 2021 12:46:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-03T12:45:58.754Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Nov 03, 2021 12:46:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-03T12:45:58.784Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Nov 03, 2021 12:46:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-03T12:45:58.812Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Nov 03, 2021 12:46:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-03T12:45:58.837Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Nov 03, 2021 12:46:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-03T12:45:58.860Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Nov 03, 2021 12:46:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-03T12:45:58.892Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Nov 03, 2021 12:46:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-03T12:45:59.292Z: Starting 5 ****s in us-central1-a...
Nov 03, 2021 12:46:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-03T12:46:26.473Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Nov 03, 2021 12:46:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-03T12:46:51.302Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Nov 03, 2021 12:47:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-03T12:47:45.489Z: Workers have started successfully.
Nov 03, 2021 12:47:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-03T12:47:45.591Z: Workers have started successfully.
Nov 03, 2021 2:16:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-03T14:15:59.432Z: Staged package animal-sniffer-annotations-1.20-vt1E38otwrj1wIzR1vDgznQJTsZ3gSYJaOA_wOd1Iqw.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/animal-sniffer-annotations-1.20-vt1E38otwrj1wIzR1vDgznQJTsZ3gSYJaOA_wOd1Iqw.jar' is inaccessible.
Nov 03, 2021 2:16:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-11-03T14:16:02.958Z: Staged package opencensus-contrib-http-util-0.28.0-ScPbKinx_bL3OSjL6pab0dQPq3zFu2JzAiur2W96eJs.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/opencensus-contrib-http-util-0.28.0-ScPbKinx_bL3OSjL6pab0dQPq3zFu2JzAiur2W96eJs.jar' is inaccessible.
Nov 03, 2021 2:16:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-11-03T14:16:03.509Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Nov 03, 2021 2:19:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-11-03T14:19:03.274Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Nov 03, 2021 4:00:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-03T16:00:32.098Z: Cancel request is committed for workflow job: 2021-11-03_05_45_43-6875680358526166536.
Nov 03, 2021 4:00:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-03T16:00:32.146Z: Cleaning up.
Nov 03, 2021 4:00:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-03T16:00:32.233Z: Stopping **** pool...
Nov 03, 2021 4:00:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-03T16:00:32.290Z: Stopping **** pool...
Nov 03, 2021 4:02:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-03T16:02:52.793Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Nov 03, 2021 4:02:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-03T16:02:52.860Z: Worker pool stopped.
Nov 03, 2021 4:02:59 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-11-03_05_45_43-6875680358526166536 finished with status CANCELLED.
Load test results for test (ID): e5970ce7-384f-4aa3-ae5e-7a85df0b2952 and timestamp: 2021-11-03T12:45:38.884000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11535.531
dataflow_v2_java11_total_bytes_count             2.25421996E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211103124333
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:df3ff75fba7ea1b2b418e87b8486098e715472f701f3e15685db6a4557f26568
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211103124333]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:df3ff75fba7ea1b2b418e87b8486098e715472f701f3e15685db6a4557f26568]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211103124333] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:df3ff75fba7ea1b2b418e87b8486098e715472f701f3e15685db6a4557f26568])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:df3ff75fba7ea1b2b418e87b8486098e715472f701f3e15685db6a4557f26568
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:df3ff75fba7ea1b2b418e87b8486098e715472f701f3e15685db6a4557f26568
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:df3ff75fba7ea1b2b418e87b8486098e715472f701f3e15685db6a4557f26568].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 43s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/5ysaqeql23r7q

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #138

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/138/display/redirect?page=changes>

Changes:

[aydar.zaynutdinov] [BEAM-13145][Playground]

[avilovpavel6] Added cleanup os env before and after tests

[noreply] [BEAM-11936] Fix errorprone warnings (#15821)

[noreply] [BEAM-13001] fixes Golint issues (#15859)

[noreply] Merge pull request #15840 from [Playground][BEAM-13146][Bugfix]


------------------------------------------
[...truncated 48.50 KB...]
6f2d0f609fa2: Preparing
fbfe7e609ece: Preparing
2b9a9942b6c1: Preparing
5a652505ed5e: Preparing
acf4563f911e: Preparing
a8b9b2257dd2: Preparing
1efe0a188f85: Preparing
4f20cc1160e5: Preparing
49de65239a80: Preparing
f1ecb3cf56ea: Preparing
8065d226bf37: Preparing
1a2a9a996bfa: Preparing
78700b6b35d0: Preparing
62a5b8741e83: Preparing
36e0782f1159: Preparing
ba6e5ff31f23: Preparing
9f9f651e9303: Preparing
0b3c02b5d746: Preparing
62a747bf1719: Preparing
78700b6b35d0: Waiting
62a5b8741e83: Waiting
a8b9b2257dd2: Waiting
36e0782f1159: Waiting
1efe0a188f85: Waiting
ba6e5ff31f23: Waiting
9f9f651e9303: Waiting
4f20cc1160e5: Waiting
0b3c02b5d746: Waiting
49de65239a80: Waiting
62a747bf1719: Waiting
f1ecb3cf56ea: Waiting
8065d226bf37: Waiting
1a2a9a996bfa: Waiting
2b9a9942b6c1: Pushed
fbfe7e609ece: Pushed
acf4563f911e: Pushed
a8b9b2257dd2: Pushed
6f2d0f609fa2: Pushed
5a652505ed5e: Pushed
4f20cc1160e5: Pushed
49de65239a80: Pushed
78700b6b35d0: Layer already exists
62a5b8741e83: Layer already exists
36e0782f1159: Layer already exists
ba6e5ff31f23: Layer already exists
1a2a9a996bfa: Pushed
9f9f651e9303: Layer already exists
8065d226bf37: Pushed
0b3c02b5d746: Layer already exists
62a747bf1719: Layer already exists
1efe0a188f85: Pushed
f1ecb3cf56ea: Pushed
20211102124331: digest: sha256:30ab3ece2033b5c69b96f0b184ad618b152dfd79c2c671732c3750af8b2476ea size: 4311

> Task :sdks:java:testing:load-tests:run
Nov 02, 2021 12:45:35 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Nov 02, 2021 12:45:35 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 195 files. Enable logging at DEBUG level to see which files will be staged.
Nov 02, 2021 12:45:36 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Nov 02, 2021 12:45:36 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Nov 02, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 195 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Nov 02, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 195 files cached, 0 files newly uploaded in 0 seconds
Nov 02, 2021 12:45:39 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Nov 02, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <108949 bytes, hash f0f335e69457a4717c3a309eebb6f69db118f7d853ca4b045fe8aa506627b9bc> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-8PM15pRXpHF8OjCe67b2nbEY99hTyksEX-iqUGYnubw.pb
Nov 02, 2021 12:45:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Nov 02, 2021 12:45:41 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4bd51d3e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@33425811, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4b74b35, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@e4e1ef5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6d11ceef, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4cb2918c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@72e295cc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@c2584d3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6fa0450e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@37468787, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@51ec2856, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@714f3da4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1caa9eb6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1f53481b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2fcd7d3f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@27e7c77f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6f70a21b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6ae62c7e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2e869098, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@37c36608]
Nov 02, 2021 12:45:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Nov 02, 2021 12:45:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Nov 02, 2021 12:45:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Nov 02, 2021 12:45:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Nov 02, 2021 12:45:41 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@53a7a60c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@28bdbe88, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6a87026, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@ef60710, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@600f5704, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2503ec73, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@606f81b5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5e1fc42f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@44b21f9f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7ee8130e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6296474f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4288d98e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@169268a7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@285c6918, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@78a0ff63, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7c601d50, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@79b2852b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@326d27ac, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4d499d65, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@313f8301]
Nov 02, 2021 12:45:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Nov 02, 2021 12:45:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Nov 02, 2021 12:45:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Nov 02, 2021 12:45:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Nov 02, 2021 12:45:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Nov 02, 2021 12:45:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Nov 02, 2021 12:45:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Nov 02, 2021 12:45:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Nov 02, 2021 12:45:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Nov 02, 2021 12:45:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Nov 02, 2021 12:45:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Nov 02, 2021 12:45:41 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.35.0-SNAPSHOT
Nov 02, 2021 12:45:42 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-11-02_05_45_41-11128727296524182537?project=apache-beam-testing
Nov 02, 2021 12:45:42 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-11-02_05_45_41-11128727296524182537
Nov 02, 2021 12:45:42 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-11-02_05_45_41-11128727296524182537
Nov 02, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-11-02T12:45:47.977Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-11-mihj. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Nov 02, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-02T12:45:53.037Z: Worker configuration: e2-standard-2 in us-central1-a.
Nov 02, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-02T12:45:53.863Z: Expanding SplittableParDo operations into optimizable parts.
Nov 02, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-02T12:45:53.897Z: Expanding CollectionToSingleton operations into optimizable parts.
Nov 02, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-02T12:45:53.998Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 02, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-02T12:45:54.080Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Nov 02, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-02T12:45:54.100Z: Expanding GroupByKey operations into streaming Read/Write steps
Nov 02, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-02T12:45:54.179Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 02, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-02T12:45:54.286Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 02, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-02T12:45:54.325Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Nov 02, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-02T12:45:54.346Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Nov 02, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-02T12:45:54.368Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Nov 02, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-02T12:45:54.424Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Nov 02, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-02T12:45:54.455Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Nov 02, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-02T12:45:54.502Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Nov 02, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-02T12:45:54.555Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Nov 02, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-02T12:45:54.594Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Nov 02, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-02T12:45:54.630Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Nov 02, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-02T12:45:54.669Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Nov 02, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-02T12:45:54.702Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Nov 02, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-02T12:45:54.736Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Nov 02, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-02T12:45:54.760Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Nov 02, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-02T12:45:54.783Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Nov 02, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-02T12:45:54.813Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Nov 02, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-02T12:45:54.837Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Nov 02, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-02T12:45:54.861Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Nov 02, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-02T12:45:54.884Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Nov 02, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-02T12:45:54.910Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Nov 02, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-02T12:45:54.953Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Nov 02, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-02T12:45:54.985Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Nov 02, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-02T12:45:55.012Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Nov 02, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-02T12:45:55.373Z: Starting 5 ****s in us-central1-a...
Nov 02, 2021 12:46:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-02T12:46:27.695Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Nov 02, 2021 12:46:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-02T12:46:35.270Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Nov 02, 2021 12:47:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-02T12:47:31.354Z: Workers have started successfully.
Nov 02, 2021 12:47:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-02T12:47:31.387Z: Workers have started successfully.
Nov 02, 2021 4:00:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-02T16:00:23.175Z: Cancel request is committed for workflow job: 2021-11-02_05_45_41-11128727296524182537.
Nov 02, 2021 4:00:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-02T16:00:23.241Z: Cleaning up.
Nov 02, 2021 4:00:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-02T16:00:23.311Z: Stopping **** pool...
Nov 02, 2021 4:00:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-02T16:00:23.372Z: Stopping **** pool...
Nov 02, 2021 4:02:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-02T16:02:47.559Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Nov 02, 2021 4:02:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-02T16:02:47.598Z: Worker pool stopped.
Nov 02, 2021 4:02:53 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-11-02_05_45_41-11128727296524182537 finished with status CANCELLED.
Load test results for test (ID): 160e81c4-91cd-4804-9359-1de69e181b0f and timestamp: 2021-11-02T12:45:36.005000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11532.046
dataflow_v2_java11_total_bytes_count             2.71981656E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211102124331
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:30ab3ece2033b5c69b96f0b184ad618b152dfd79c2c671732c3750af8b2476ea
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211102124331]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:30ab3ece2033b5c69b96f0b184ad618b152dfd79c2c671732c3750af8b2476ea]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211102124331] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:30ab3ece2033b5c69b96f0b184ad618b152dfd79c2c671732c3750af8b2476ea])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:30ab3ece2033b5c69b96f0b184ad618b152dfd79c2c671732c3750af8b2476ea
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:30ab3ece2033b5c69b96f0b184ad618b152dfd79c2c671732c3750af8b2476ea
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:30ab3ece2033b5c69b96f0b184ad618b152dfd79c2c671732c3750af8b2476ea].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 38s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/4riyv33ietvow

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #137

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/137/display/redirect>

Changes:


------------------------------------------
[...truncated 49.30 KB...]
d79efce45f86: Preparing
fb963ca4c4dc: Preparing
9e7a4c8d4431: Preparing
964c39a7cd1c: Preparing
83596ea38e00: Preparing
3cdf2cfc0e4e: Preparing
13743e8f9b7b: Preparing
20d67df6a282: Preparing
3c6b195f73b0: Preparing
2e913ba9e385: Preparing
7cd745a02111: Preparing
314c40b596c3: Preparing
78700b6b35d0: Preparing
62a5b8741e83: Preparing
36e0782f1159: Preparing
ba6e5ff31f23: Preparing
9f9f651e9303: Preparing
0b3c02b5d746: Preparing
62a747bf1719: Preparing
2e913ba9e385: Waiting
7cd745a02111: Waiting
314c40b596c3: Waiting
78700b6b35d0: Waiting
3cdf2cfc0e4e: Waiting
9f9f651e9303: Waiting
13743e8f9b7b: Waiting
0b3c02b5d746: Waiting
ba6e5ff31f23: Waiting
62a5b8741e83: Waiting
62a747bf1719: Waiting
36e0782f1159: Waiting
20d67df6a282: Waiting
3c6b195f73b0: Waiting
83596ea38e00: Pushed
9e7a4c8d4431: Pushed
fb963ca4c4dc: Pushed
3cdf2cfc0e4e: Pushed
964c39a7cd1c: Pushed
d79efce45f86: Pushed
20d67df6a282: Pushed
3c6b195f73b0: Pushed
78700b6b35d0: Layer already exists
62a5b8741e83: Layer already exists
7cd745a02111: Pushed
13743e8f9b7b: Pushed
36e0782f1159: Layer already exists
314c40b596c3: Pushed
9f9f651e9303: Layer already exists
ba6e5ff31f23: Layer already exists
0b3c02b5d746: Layer already exists
62a747bf1719: Layer already exists
2e913ba9e385: Pushed
20211101124332: digest: sha256:1ff0762820c320f2491e17b8f7fb411d8ba5bc60567b11dacdeefe67d5d22793 size: 4311

> Task :sdks:java:testing:load-tests:run
Nov 01, 2021 12:45:18 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Nov 01, 2021 12:45:19 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 195 files. Enable logging at DEBUG level to see which files will be staged.
Nov 01, 2021 12:45:20 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Nov 01, 2021 12:45:20 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Nov 01, 2021 12:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 195 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Nov 01, 2021 12:45:23 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 195 files cached, 0 files newly uploaded in 0 seconds
Nov 01, 2021 12:45:23 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Nov 01, 2021 12:45:23 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <108949 bytes, hash 2406fab948b04e175ce73b64bd32f7adc99b0f30c9e0bf834ad4d9577a1ca55c> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-JAb6uUiwThdc5ztkvTL3rcmbDzDJ4L-DStTZV3ocpVw.pb
Nov 01, 2021 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Nov 01, 2021 12:45:24 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4b74b35, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@e4e1ef5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6d11ceef, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4cb2918c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@72e295cc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@c2584d3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6fa0450e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@37468787, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@51ec2856, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@714f3da4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1caa9eb6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1f53481b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2fcd7d3f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@27e7c77f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6f70a21b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6ae62c7e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2e869098, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@37c36608, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5d497a91, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@617389a]
Nov 01, 2021 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Nov 01, 2021 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Nov 01, 2021 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Nov 01, 2021 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Nov 01, 2021 12:45:24 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6a87026, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@ef60710, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@600f5704, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2503ec73, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@606f81b5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5e1fc42f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@44b21f9f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7ee8130e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6296474f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4288d98e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@169268a7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@285c6918, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@78a0ff63, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7c601d50, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@79b2852b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@326d27ac, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4d499d65, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@313f8301, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5cc9d3d0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7c2dfa2]
Nov 01, 2021 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Nov 01, 2021 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Nov 01, 2021 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Nov 01, 2021 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Nov 01, 2021 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Nov 01, 2021 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Nov 01, 2021 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Nov 01, 2021 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Nov 01, 2021 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Nov 01, 2021 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Nov 01, 2021 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Nov 01, 2021 12:45:24 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.35.0-SNAPSHOT
Nov 01, 2021 12:45:25 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-11-01_05_45_24-11615703837044486865?project=apache-beam-testing
Nov 01, 2021 12:45:25 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-11-01_05_45_24-11615703837044486865
Nov 01, 2021 12:45:25 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-11-01_05_45_24-11615703837044486865
Nov 01, 2021 12:45:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-11-01T12:45:31.049Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-11-6llt. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Nov 01, 2021 12:45:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-01T12:45:36.138Z: Worker configuration: e2-standard-2 in us-central1-a.
Nov 01, 2021 12:45:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-01T12:45:36.758Z: Expanding SplittableParDo operations into optimizable parts.
Nov 01, 2021 12:45:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-01T12:45:36.794Z: Expanding CollectionToSingleton operations into optimizable parts.
Nov 01, 2021 12:45:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-01T12:45:36.870Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 01, 2021 12:45:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-01T12:45:36.936Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Nov 01, 2021 12:45:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-01T12:45:36.968Z: Expanding GroupByKey operations into streaming Read/Write steps
Nov 01, 2021 12:45:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-01T12:45:37.034Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 01, 2021 12:45:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-01T12:45:37.132Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 01, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-01T12:45:37.166Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Nov 01, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-01T12:45:37.198Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Nov 01, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-01T12:45:37.231Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Nov 01, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-01T12:45:37.264Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Nov 01, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-01T12:45:37.297Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Nov 01, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-01T12:45:37.331Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Nov 01, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-01T12:45:37.363Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Nov 01, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-01T12:45:37.394Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Nov 01, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-01T12:45:37.429Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Nov 01, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-01T12:45:37.464Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Nov 01, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-01T12:45:37.485Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Nov 01, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-01T12:45:37.518Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Nov 01, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-01T12:45:37.540Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Nov 01, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-01T12:45:37.570Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Nov 01, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-01T12:45:37.606Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Nov 01, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-01T12:45:37.637Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Nov 01, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-01T12:45:37.659Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Nov 01, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-01T12:45:37.693Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Nov 01, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-01T12:45:37.726Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Nov 01, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-01T12:45:37.752Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Nov 01, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-01T12:45:37.780Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Nov 01, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-01T12:45:37.837Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Nov 01, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-01T12:45:38.179Z: Starting 5 ****s in us-central1-a...
Nov 01, 2021 12:45:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-01T12:45:55.693Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Nov 01, 2021 12:46:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-01T12:46:20.209Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Nov 01, 2021 12:47:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-01T12:47:12.957Z: Workers have started successfully.
Nov 01, 2021 12:47:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-01T12:47:12.983Z: Workers have started successfully.
Nov 01, 2021 4:00:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-01T16:00:35.421Z: Cancel request is committed for workflow job: 2021-11-01_05_45_24-11615703837044486865.
Nov 01, 2021 4:00:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-01T16:00:35.523Z: Cleaning up.
Nov 01, 2021 4:00:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-01T16:00:35.603Z: Stopping **** pool...
Nov 01, 2021 4:00:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-01T16:00:35.648Z: Stopping **** pool...
Nov 01, 2021 4:02:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-01T16:02:55.578Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Nov 01, 2021 4:02:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-11-01T16:02:55.616Z: Worker pool stopped.
Nov 01, 2021 4:03:04 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-11-01_05_45_24-11615703837044486865 finished with status CANCELLED.
Load test results for test (ID): 685ba138-1ce8-4559-9c77-2e5de61ed990 and timestamp: 2021-11-01T12:45:19.655000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11561.385
dataflow_v2_java11_total_bytes_count             2.56330316E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211101124332
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:1ff0762820c320f2491e17b8f7fb411d8ba5bc60567b11dacdeefe67d5d22793
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211101124332]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:1ff0762820c320f2491e17b8f7fb411d8ba5bc60567b11dacdeefe67d5d22793]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211101124332] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:1ff0762820c320f2491e17b8f7fb411d8ba5bc60567b11dacdeefe67d5d22793])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:1ff0762820c320f2491e17b8f7fb411d8ba5bc60567b11dacdeefe67d5d22793
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:1ff0762820c320f2491e17b8f7fb411d8ba5bc60567b11dacdeefe67d5d22793
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:1ff0762820c320f2491e17b8f7fb411d8ba5bc60567b11dacdeefe67d5d22793].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 51s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/zdx37fofkis34

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #136

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/136/display/redirect?page=changes>

Changes:

[noreply] Merge pull request #15753 from [BEAM-13080] Add option in Reshuffle to


------------------------------------------
[...truncated 48.49 KB...]
0fe811529e21: Preparing
c635961765b5: Preparing
0d2fce186cf1: Preparing
afc1c5da1be2: Preparing
7287cdf1103c: Preparing
59209505e674: Preparing
d2b2c114fca5: Preparing
59b99cfa4691: Preparing
7e1f83da258e: Preparing
54742e584f37: Preparing
c61eec161bba: Preparing
71ab1f18d33f: Preparing
78700b6b35d0: Preparing
62a5b8741e83: Preparing
36e0782f1159: Preparing
ba6e5ff31f23: Preparing
9f9f651e9303: Preparing
0b3c02b5d746: Preparing
62a747bf1719: Preparing
54742e584f37: Waiting
c61eec161bba: Waiting
59b99cfa4691: Waiting
71ab1f18d33f: Waiting
ba6e5ff31f23: Waiting
78700b6b35d0: Waiting
7e1f83da258e: Waiting
9f9f651e9303: Waiting
0b3c02b5d746: Waiting
62a747bf1719: Waiting
36e0782f1159: Waiting
62a5b8741e83: Waiting
d2b2c114fca5: Waiting
59209505e674: Waiting
0d2fce186cf1: Pushed
7287cdf1103c: Pushed
c635961765b5: Pushed
59209505e674: Pushed
0fe811529e21: Pushed
afc1c5da1be2: Pushed
59b99cfa4691: Pushed
7e1f83da258e: Pushed
78700b6b35d0: Layer already exists
62a5b8741e83: Layer already exists
36e0782f1159: Layer already exists
ba6e5ff31f23: Layer already exists
9f9f651e9303: Layer already exists
0b3c02b5d746: Layer already exists
c61eec161bba: Pushed
62a747bf1719: Layer already exists
d2b2c114fca5: Pushed
71ab1f18d33f: Pushed
54742e584f37: Pushed
20211031124332: digest: sha256:0f47d3d3b4a01ef3041f36347f38df837db674e18059b5f6b4e635d281a81b80 size: 4311

> Task :sdks:java:testing:load-tests:run
Oct 31, 2021 12:45:19 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Oct 31, 2021 12:45:20 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 195 files. Enable logging at DEBUG level to see which files will be staged.
Oct 31, 2021 12:45:20 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Oct 31, 2021 12:45:20 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Oct 31, 2021 12:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 195 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Oct 31, 2021 12:45:23 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 195 files cached, 0 files newly uploaded in 0 seconds
Oct 31, 2021 12:45:23 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Oct 31, 2021 12:45:23 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <108949 bytes, hash 34bce12ab4fe2cf85115ed12731879f77613ddfaaac5dae463260061de728f33> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-NLzhKrT-LPhRFe0Scxh593YT3fqqxdrkYyYAYd5yjzM.pb
Oct 31, 2021 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Oct 31, 2021 12:45:24 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4bd51d3e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@33425811, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4b74b35, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@e4e1ef5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6d11ceef, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4cb2918c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@72e295cc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@c2584d3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6fa0450e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@37468787, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@51ec2856, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@714f3da4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1caa9eb6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1f53481b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2fcd7d3f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@27e7c77f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6f70a21b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6ae62c7e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2e869098, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@37c36608]
Oct 31, 2021 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Oct 31, 2021 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Oct 31, 2021 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Oct 31, 2021 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Oct 31, 2021 12:45:24 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@53a7a60c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@28bdbe88, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6a87026, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@ef60710, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@600f5704, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2503ec73, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@606f81b5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5e1fc42f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@44b21f9f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7ee8130e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6296474f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4288d98e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@169268a7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@285c6918, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@78a0ff63, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7c601d50, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@79b2852b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@326d27ac, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4d499d65, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@313f8301]
Oct 31, 2021 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Oct 31, 2021 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Oct 31, 2021 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Oct 31, 2021 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Oct 31, 2021 12:45:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Oct 31, 2021 12:45:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Oct 31, 2021 12:45:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Oct 31, 2021 12:45:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Oct 31, 2021 12:45:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Oct 31, 2021 12:45:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Oct 31, 2021 12:45:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Oct 31, 2021 12:45:25 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.35.0-SNAPSHOT
Oct 31, 2021 12:45:25 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-10-31_05_45_25-909215253647654451?project=apache-beam-testing
Oct 31, 2021 12:45:25 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-10-31_05_45_25-909215253647654451
Oct 31, 2021 12:45:25 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-10-31_05_45_25-909215253647654451
Oct 31, 2021 12:45:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-10-31T12:45:34.897Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-10-h9fm. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Oct 31, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-31T12:45:38.961Z: Worker configuration: e2-standard-2 in us-central1-a.
Oct 31, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-31T12:45:39.561Z: Expanding SplittableParDo operations into optimizable parts.
Oct 31, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-31T12:45:39.607Z: Expanding CollectionToSingleton operations into optimizable parts.
Oct 31, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-31T12:45:39.676Z: Expanding CoGroupByKey operations into optimizable parts.
Oct 31, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-31T12:45:39.725Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Oct 31, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-31T12:45:39.745Z: Expanding GroupByKey operations into streaming Read/Write steps
Oct 31, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-31T12:45:39.800Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Oct 31, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-31T12:45:39.910Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Oct 31, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-31T12:45:39.937Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Oct 31, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-31T12:45:39.966Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Oct 31, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-31T12:45:39.993Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Oct 31, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-31T12:45:40.016Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Oct 31, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-31T12:45:40.045Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Oct 31, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-31T12:45:40.066Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Oct 31, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-31T12:45:40.097Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Oct 31, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-31T12:45:40.131Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Oct 31, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-31T12:45:40.161Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Oct 31, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-31T12:45:40.200Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Oct 31, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-31T12:45:40.235Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Oct 31, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-31T12:45:40.267Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Oct 31, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-31T12:45:40.291Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Oct 31, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-31T12:45:40.316Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Oct 31, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-31T12:45:40.345Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Oct 31, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-31T12:45:40.371Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Oct 31, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-31T12:45:40.413Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Oct 31, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-31T12:45:40.437Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Oct 31, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-31T12:45:40.471Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Oct 31, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-31T12:45:40.494Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Oct 31, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-31T12:45:40.527Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Oct 31, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-31T12:45:40.562Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Oct 31, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-31T12:45:40.884Z: Starting 5 ****s in us-central1-a...
Oct 31, 2021 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-31T12:45:46.342Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Oct 31, 2021 12:46:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-31T12:46:25.432Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Oct 31, 2021 12:47:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-31T12:47:23.242Z: Workers have started successfully.
Oct 31, 2021 12:47:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-31T12:47:23.271Z: Workers have started successfully.
Oct 31, 2021 4:00:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-31T16:00:23.805Z: Cancel request is committed for workflow job: 2021-10-31_05_45_25-909215253647654451.
Oct 31, 2021 4:00:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-31T16:00:23.860Z: Cleaning up.
Oct 31, 2021 4:00:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-31T16:00:23.911Z: Stopping **** pool...
Oct 31, 2021 4:00:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-31T16:00:23.954Z: Stopping **** pool...
Oct 31, 2021 4:02:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-31T16:02:38.765Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Oct 31, 2021 4:02:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-31T16:02:38.799Z: Worker pool stopped.
Oct 31, 2021 4:02:45 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-10-31_05_45_25-909215253647654451 finished with status CANCELLED.
Load test results for test (ID): 7e6a4f9b-ba60-4c25-a4ed-7a68ff45ce29 and timestamp: 2021-10-31T12:45:20.418000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11536.735
dataflow_v2_java11_total_bytes_count             2.32191087E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211031124332
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:0f47d3d3b4a01ef3041f36347f38df837db674e18059b5f6b4e635d281a81b80
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211031124332]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:0f47d3d3b4a01ef3041f36347f38df837db674e18059b5f6b4e635d281a81b80]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211031124332] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:0f47d3d3b4a01ef3041f36347f38df837db674e18059b5f6b4e635d281a81b80])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:0f47d3d3b4a01ef3041f36347f38df837db674e18059b5f6b4e635d281a81b80
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:0f47d3d3b4a01ef3041f36347f38df837db674e18059b5f6b4e635d281a81b80
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:0f47d3d3b4a01ef3041f36347f38df837db674e18059b5f6b4e635d281a81b80].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 28s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/d2svplfodkiw6

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #135

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/135/display/redirect?page=changes>

Changes:

[Luke Cwik] [BEAM-13015] Migrate bundle processing in the SDK harness to using

[noreply] [BEAM-8958] Use AWS credentials provider with BasicKinesisProvider

[Brian Hulette] [BEAM-13099] Copy vendored calcite build for 1.28.0

[Brian Hulette] [BEAM-13099] Modifications for vendor/calcite-1_28_0 build

[noreply] [BEAM-13099] Use BlockBuilder.add(..) rather than

[noreply] Minor: Remove broken python compatibility checks (#15828)

[noreply] [BEAM-12047] Updates CHANGES.md to mention the URN convention (#15845)

[noreply] [BEAM-13001] DoFn metrics for Go SDK (#15657)


------------------------------------------
[...truncated 48.51 KB...]
62ffb163308a: Preparing
a3e8466288ca: Preparing
af59f67fb9eb: Preparing
aa0b48ac2a87: Preparing
72ac93015a3a: Preparing
f66cbc051eb8: Preparing
bc30c76e1ea6: Preparing
f365b800c5c7: Preparing
81ceaa9f8013: Preparing
46d790545cf1: Preparing
cafa2d3a4a5a: Preparing
d0a858d06060: Preparing
78700b6b35d0: Preparing
62a5b8741e83: Preparing
36e0782f1159: Preparing
ba6e5ff31f23: Preparing
9f9f651e9303: Preparing
0b3c02b5d746: Preparing
62a747bf1719: Preparing
f66cbc051eb8: Waiting
ba6e5ff31f23: Waiting
d0a858d06060: Waiting
9f9f651e9303: Waiting
bc30c76e1ea6: Waiting
62a747bf1719: Waiting
0b3c02b5d746: Waiting
f365b800c5c7: Waiting
36e0782f1159: Waiting
cafa2d3a4a5a: Waiting
81ceaa9f8013: Waiting
46d790545cf1: Waiting
78700b6b35d0: Waiting
62a5b8741e83: Waiting
72ac93015a3a: Pushed
af59f67fb9eb: Pushed
a3e8466288ca: Pushed
f66cbc051eb8: Pushed
aa0b48ac2a87: Pushed
f365b800c5c7: Pushed
62ffb163308a: Pushed
81ceaa9f8013: Pushed
78700b6b35d0: Layer already exists
62a5b8741e83: Layer already exists
cafa2d3a4a5a: Pushed
36e0782f1159: Layer already exists
ba6e5ff31f23: Layer already exists
0b3c02b5d746: Layer already exists
9f9f651e9303: Layer already exists
62a747bf1719: Layer already exists
bc30c76e1ea6: Pushed
d0a858d06060: Pushed
46d790545cf1: Pushed
20211030124336: digest: sha256:ca577b4f870e432816af8a2bc181a81bdc1fed3857c3e38c57a5bc5cc66dc5ef size: 4311

> Task :sdks:java:testing:load-tests:run
Oct 30, 2021 12:45:31 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Oct 30, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 195 files. Enable logging at DEBUG level to see which files will be staged.
Oct 30, 2021 12:45:32 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Oct 30, 2021 12:45:32 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Oct 30, 2021 12:45:34 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 195 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Oct 30, 2021 12:45:34 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 195 files cached, 0 files newly uploaded in 0 seconds
Oct 30, 2021 12:45:34 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Oct 30, 2021 12:45:34 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <108949 bytes, hash 0a61e45cf248142a8466894ba36c31242228cfac6e4c35bc6eb64e4aa62646a3> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-CmHkXPJIFCqEZolLo2wxJCIoz6xuTDW8brZOSqYmRqM.pb
Oct 30, 2021 12:45:36 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Oct 30, 2021 12:45:36 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@33425811, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4b74b35, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@e4e1ef5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6d11ceef, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4cb2918c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@72e295cc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@c2584d3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6fa0450e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@37468787, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@51ec2856, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@714f3da4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1caa9eb6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1f53481b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2fcd7d3f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@27e7c77f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6f70a21b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6ae62c7e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2e869098, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@37c36608, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5d497a91]
Oct 30, 2021 12:45:36 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Oct 30, 2021 12:45:36 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Oct 30, 2021 12:45:36 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Oct 30, 2021 12:45:36 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Oct 30, 2021 12:45:36 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@28bdbe88, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6a87026, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@ef60710, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@600f5704, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2503ec73, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@606f81b5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5e1fc42f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@44b21f9f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7ee8130e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6296474f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4288d98e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@169268a7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@285c6918, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@78a0ff63, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7c601d50, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@79b2852b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@326d27ac, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4d499d65, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@313f8301, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5cc9d3d0]
Oct 30, 2021 12:45:36 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Oct 30, 2021 12:45:36 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Oct 30, 2021 12:45:36 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Oct 30, 2021 12:45:36 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Oct 30, 2021 12:45:36 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Oct 30, 2021 12:45:36 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Oct 30, 2021 12:45:36 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Oct 30, 2021 12:45:36 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Oct 30, 2021 12:45:36 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Oct 30, 2021 12:45:36 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Oct 30, 2021 12:45:36 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Oct 30, 2021 12:45:36 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.35.0-SNAPSHOT
Oct 30, 2021 12:45:37 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-10-30_05_45_36-1707718346160858959?project=apache-beam-testing
Oct 30, 2021 12:45:37 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-10-30_05_45_36-1707718346160858959
Oct 30, 2021 12:45:37 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-10-30_05_45_36-1707718346160858959
Oct 30, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-10-30T12:45:43.393Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-10-too8. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Oct 30, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-30T12:45:47.528Z: Worker configuration: e2-standard-2 in us-central1-a.
Oct 30, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-30T12:45:48.328Z: Expanding SplittableParDo operations into optimizable parts.
Oct 30, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-30T12:45:48.359Z: Expanding CollectionToSingleton operations into optimizable parts.
Oct 30, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-30T12:45:48.462Z: Expanding CoGroupByKey operations into optimizable parts.
Oct 30, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-30T12:45:48.534Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Oct 30, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-30T12:45:48.562Z: Expanding GroupByKey operations into streaming Read/Write steps
Oct 30, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-30T12:45:48.611Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Oct 30, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-30T12:45:48.673Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Oct 30, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-30T12:45:48.689Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Oct 30, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-30T12:45:48.716Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Oct 30, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-30T12:45:48.735Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Oct 30, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-30T12:45:48.758Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Oct 30, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-30T12:45:48.781Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Oct 30, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-30T12:45:48.803Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Oct 30, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-30T12:45:48.827Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Oct 30, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-30T12:45:48.848Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Oct 30, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-30T12:45:48.872Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Oct 30, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-30T12:45:48.885Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Oct 30, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-30T12:45:48.921Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Oct 30, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-30T12:45:48.944Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Oct 30, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-30T12:45:48.967Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Oct 30, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-30T12:45:49.000Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Oct 30, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-30T12:45:49.035Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Oct 30, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-30T12:45:49.064Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Oct 30, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-30T12:45:49.090Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Oct 30, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-30T12:45:49.122Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Oct 30, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-30T12:45:49.151Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Oct 30, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-30T12:45:49.174Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Oct 30, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-30T12:45:49.240Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Oct 30, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-30T12:45:49.266Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Oct 30, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-30T12:45:49.556Z: Starting 5 ****s in us-central1-a...
Oct 30, 2021 12:46:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-30T12:46:20.123Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Oct 30, 2021 12:46:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-30T12:46:31.368Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Oct 30, 2021 12:47:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-30T12:47:27.023Z: Workers have started successfully.
Oct 30, 2021 12:47:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-30T12:47:27.068Z: Workers have started successfully.
Oct 30, 2021 4:00:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-30T16:00:24.234Z: Cancel request is committed for workflow job: 2021-10-30_05_45_36-1707718346160858959.
Oct 30, 2021 4:00:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-30T16:00:24.303Z: Cleaning up.
Oct 30, 2021 4:00:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-30T16:00:24.365Z: Stopping **** pool...
Oct 30, 2021 4:00:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-30T16:00:24.410Z: Stopping **** pool...
Oct 30, 2021 4:02:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-30T16:02:46.104Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Oct 30, 2021 4:02:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-30T16:02:46.142Z: Worker pool stopped.
Oct 30, 2021 4:02:53 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-10-30_05_45_36-1707718346160858959 finished with status CANCELLED.
Load test results for test (ID): 429ea241-71e6-40cd-ab5f-05ebe6f1df4f and timestamp: 2021-10-30T12:45:32.021000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11536.936
dataflow_v2_java11_total_bytes_count             1.95846834E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211030124336
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ca577b4f870e432816af8a2bc181a81bdc1fed3857c3e38c57a5bc5cc66dc5ef
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211030124336]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ca577b4f870e432816af8a2bc181a81bdc1fed3857c3e38c57a5bc5cc66dc5ef]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211030124336] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ca577b4f870e432816af8a2bc181a81bdc1fed3857c3e38c57a5bc5cc66dc5ef])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ca577b4f870e432816af8a2bc181a81bdc1fed3857c3e38c57a5bc5cc66dc5ef
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ca577b4f870e432816af8a2bc181a81bdc1fed3857c3e38c57a5bc5cc66dc5ef
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ca577b4f870e432816af8a2bc181a81bdc1fed3857c3e38c57a5bc5cc66dc5ef].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 33s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/xdi223qrw356q

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #134

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/134/display/redirect?page=changes>

Changes:

[artur.khanin] Implemented GetCompileOutput and covered with unit tests

[artur.khanin] Replaced logging library regarding PR comment

[daria.malkova] fix executor tests

[mmack] [BEAM-13138] Update code to retrieve container endpoint

[aydar.zaynutdinov] [BEAM-13120][Playground]

[noreply] Merge pull request #15709 from [BEAM-12967] [Playground] Create Example

[noreply] Merge pull request #15294 from [BEAM-11986] Spanner write metric

[noreply] [BEAM-13130] Remove persistent references to stateKeyReaders (#15815)

[Brian Hulette] [BEAM-13099] Replace call to RelNode.metadata with BeamRelMetadataQuery

[Brian Hulette] [BEAM-13099] Update all rels to use BeamRelMetadataQuery

[Kyle Weaver] [BEAM-13143] Fix python doc generator error.

[Pablo Estrada] Handle runner-provided shards for TextIO

[noreply] Merge pull request #15721 from [BEAM-13023][Playground] Implement Redis

[noreply] Merge pull request #15802 from [BEAM-12970][Playground] Implement gRPC

[noreply] Merge pull request #15800 from [BEAM-12970][Playground] Implement gRPC


------------------------------------------
[...truncated 49.03 KB...]
0b3c02b5d746: Preparing
62a747bf1719: Preparing
c30b6163b38e: Waiting
8ce6e5c48199: Waiting
5ecec3e50f1b: Waiting
36e0782f1159: Waiting
b30ebf97fccb: Waiting
37f8420a0833: Waiting
78700b6b35d0: Waiting
62a747bf1719: Waiting
ba6e5ff31f23: Waiting
9f9f651e9303: Waiting
ec6ed34ef00f: Waiting
dcbb5dc233cc: Waiting
62a5b8741e83: Waiting
582395212cc7: Pushed
26d172e729b8: Pushed
c89c93be1ca3: Pushed
258bf8261d25: Pushed
c30b6163b38e: Pushed
3ae428a4dd9b: Pushed
8ce6e5c48199: Pushed
37f8420a0833: Pushed
78700b6b35d0: Layer already exists
62a5b8741e83: Layer already exists
36e0782f1159: Layer already exists
ba6e5ff31f23: Layer already exists
ec6ed34ef00f: Pushed
5ecec3e50f1b: Pushed
0b3c02b5d746: Layer already exists
62a747bf1719: Layer already exists
9f9f651e9303: Layer already exists
dcbb5dc233cc: Pushed
b30ebf97fccb: Pushed
20211029124334: digest: sha256:4f56c4f2248507a6ef032216b3cba8d4f325946a16fa7090a368d818948b0dd2 size: 4311

> Task :sdks:java:testing:load-tests:run
Oct 29, 2021 12:45:22 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Oct 29, 2021 12:45:22 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 195 files. Enable logging at DEBUG level to see which files will be staged.
Oct 29, 2021 12:45:23 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Oct 29, 2021 12:45:23 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Oct 29, 2021 12:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 195 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Oct 29, 2021 12:45:26 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 195 files cached, 0 files newly uploaded in 0 seconds
Oct 29, 2021 12:45:26 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Oct 29, 2021 12:45:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <108949 bytes, hash 206278b525d10e9af199c1546290e40d06e1bfcd6a4d1e491b2df840adbdc45d> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-IGJ4tSXRDprxmcFUYpDkDQbhv81qTR5JGy34QK29xF0.pb
Oct 29, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Oct 29, 2021 12:45:27 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4b74b35, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@e4e1ef5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6d11ceef, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4cb2918c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@72e295cc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@c2584d3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6fa0450e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@37468787, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@51ec2856, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@714f3da4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1caa9eb6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1f53481b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2fcd7d3f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@27e7c77f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6f70a21b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6ae62c7e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2e869098, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@37c36608, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5d497a91, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@617389a]
Oct 29, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Oct 29, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Oct 29, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Oct 29, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Oct 29, 2021 12:45:27 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6a87026, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@ef60710, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@600f5704, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2503ec73, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@606f81b5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5e1fc42f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@44b21f9f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7ee8130e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6296474f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4288d98e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@169268a7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@285c6918, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@78a0ff63, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7c601d50, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@79b2852b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@326d27ac, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4d499d65, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@313f8301, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5cc9d3d0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7c2dfa2]
Oct 29, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Oct 29, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Oct 29, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Oct 29, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Oct 29, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Oct 29, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Oct 29, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Oct 29, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Oct 29, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Oct 29, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Oct 29, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Oct 29, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.35.0-SNAPSHOT
Oct 29, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-10-29_05_45_28-16658831343352750248?project=apache-beam-testing
Oct 29, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-10-29_05_45_28-16658831343352750248
Oct 29, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-10-29_05_45_28-16658831343352750248
Oct 29, 2021 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-10-29T12:45:40.721Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-10-5bwi. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Oct 29, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-29T12:45:52.807Z: Worker configuration: e2-standard-2 in us-central1-a.
Oct 29, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-29T12:45:53.753Z: Expanding SplittableParDo operations into optimizable parts.
Oct 29, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-29T12:45:53.853Z: Expanding CollectionToSingleton operations into optimizable parts.
Oct 29, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-29T12:45:53.935Z: Expanding CoGroupByKey operations into optimizable parts.
Oct 29, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-29T12:45:53.994Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Oct 29, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-29T12:45:54.030Z: Expanding GroupByKey operations into streaming Read/Write steps
Oct 29, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-29T12:45:54.094Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Oct 29, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-29T12:45:54.204Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Oct 29, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-29T12:45:54.239Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Oct 29, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-29T12:45:54.263Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Oct 29, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-29T12:45:54.297Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Oct 29, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-29T12:45:54.328Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Oct 29, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-29T12:45:54.359Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Oct 29, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-29T12:45:54.386Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Oct 29, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-29T12:45:54.421Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Oct 29, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-29T12:45:54.444Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Oct 29, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-29T12:45:54.467Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Oct 29, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-29T12:45:54.499Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Oct 29, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-29T12:45:54.532Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Oct 29, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-29T12:45:54.561Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Oct 29, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-29T12:45:54.597Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Oct 29, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-29T12:45:54.619Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Oct 29, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-29T12:45:54.643Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Oct 29, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-29T12:45:54.680Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Oct 29, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-29T12:45:54.713Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Oct 29, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-29T12:45:54.749Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Oct 29, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-29T12:45:54.774Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Oct 29, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-29T12:45:54.814Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Oct 29, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-29T12:45:54.840Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Oct 29, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-29T12:45:54.877Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Oct 29, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-29T12:45:55.352Z: Starting 5 ****s in us-central1-a...
Oct 29, 2021 12:46:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-29T12:46:09.258Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Oct 29, 2021 12:46:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-29T12:46:40.686Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Oct 29, 2021 12:47:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-29T12:47:35.788Z: Workers have started successfully.
Oct 29, 2021 12:47:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-29T12:47:35.820Z: Workers have started successfully.
Oct 29, 2021 3:51:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-29T15:51:55.911Z: Staged package amazon-kinesis-client-1.14.2-qtBh3OuKuLE7S26ITTJzYsoYfG-mWjqbGmv3VJPhec8.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/amazon-kinesis-client-1.14.2-qtBh3OuKuLE7S26ITTJzYsoYfG-mWjqbGmv3VJPhec8.jar' is inaccessible.
Oct 29, 2021 3:51:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-29T15:51:58.406Z: Staged package jakarta.activation-api-1.2.2-oYepORA671hJp6-EvX4nvi0SDEEK8pFDc3X_4GH08J0.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/jakarta.activation-api-1.2.2-oYepORA671hJp6-EvX4nvi0SDEEK8pFDc3X_4GH08J0.jar' is inaccessible.
Oct 29, 2021 3:51:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-29T15:51:58.454Z: Staged package jakarta.xml.bind-api-2.3.3-wEU59HLppt0MdoXqgtZ3KCJpq457rKLhRQDjgeDGzsU.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/jakarta.xml.bind-api-2.3.3-wEU59HLppt0MdoXqgtZ3KCJpq457rKLhRQDjgeDGzsU.jar' is inaccessible.
Oct 29, 2021 3:52:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-10-29T15:51:59.536Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Oct 29, 2021 3:55:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-10-29T15:54:59.216Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Oct 29, 2021 3:57:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-29T15:57:55.516Z: Staged package amazon-kinesis-client-1.14.2-qtBh3OuKuLE7S26ITTJzYsoYfG-mWjqbGmv3VJPhec8.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/amazon-kinesis-client-1.14.2-qtBh3OuKuLE7S26ITTJzYsoYfG-mWjqbGmv3VJPhec8.jar' is inaccessible.
Oct 29, 2021 3:57:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-29T15:57:58.185Z: Staged package jakarta.activation-api-1.2.2-oYepORA671hJp6-EvX4nvi0SDEEK8pFDc3X_4GH08J0.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/jakarta.activation-api-1.2.2-oYepORA671hJp6-EvX4nvi0SDEEK8pFDc3X_4GH08J0.jar' is inaccessible.
Oct 29, 2021 3:57:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-29T15:57:58.225Z: Staged package jakarta.xml.bind-api-2.3.3-wEU59HLppt0MdoXqgtZ3KCJpq457rKLhRQDjgeDGzsU.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/jakarta.xml.bind-api-2.3.3-wEU59HLppt0MdoXqgtZ3KCJpq457rKLhRQDjgeDGzsU.jar' is inaccessible.
Oct 29, 2021 3:58:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-10-29T15:57:59.313Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Oct 29, 2021 4:00:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-29T16:00:28.573Z: Cancel request is committed for workflow job: 2021-10-29_05_45_28-16658831343352750248.
Oct 29, 2021 4:00:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-29T16:00:28.606Z: Cleaning up.
Oct 29, 2021 4:00:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-29T16:00:28.671Z: Stopping **** pool...
Oct 29, 2021 4:00:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-29T16:00:28.725Z: Stopping **** pool...
Oct 29, 2021 4:02:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-29T16:02:51.239Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Oct 29, 2021 4:02:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-29T16:02:51.282Z: Worker pool stopped.
Oct 29, 2021 4:02:58 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-10-29_05_45_28-16658831343352750248 finished with status CANCELLED.
Load test results for test (ID): be533bbc-7af2-421b-9f7e-d37632c77237 and timestamp: 2021-10-29T12:45:23.184000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11535.407
dataflow_v2_java11_total_bytes_count             2.60500954E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211029124334
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:4f56c4f2248507a6ef032216b3cba8d4f325946a16fa7090a368d818948b0dd2
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211029124334]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:4f56c4f2248507a6ef032216b3cba8d4f325946a16fa7090a368d818948b0dd2]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211029124334] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:4f56c4f2248507a6ef032216b3cba8d4f325946a16fa7090a368d818948b0dd2])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:4f56c4f2248507a6ef032216b3cba8d4f325946a16fa7090a368d818948b0dd2
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:4f56c4f2248507a6ef032216b3cba8d4f325946a16fa7090a368d818948b0dd2
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:4f56c4f2248507a6ef032216b3cba8d4f325946a16fa7090a368d818948b0dd2].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 41s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/mbbhbhsxp2a3q

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #133

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/133/display/redirect?page=changes>

Changes:

[relax] support json

[aydar.zaynutdinov] [BEAM-12970][Playground]

[Andrew Pilloud] [BEAM-13118] Don't force push to master

[aydar.zaynutdinov] [BEAM-12970][Playground]

[eugene.nikolayev] Fix Python docs build pre-commit failures.

[aydar.zaynutdinov] [BEAM-12970][Playground]

[noreply] Merge pull request #15731: [BEAM-13067]  Mark GroupIntoBatches output as

[noreply] [BEAM-13066, BEAM-13087] Workaround for coder type combiner packing

[noreply] [BEAM-13066] Disable using abstract iterable by default. (#15805)

[aydar.zaynutdinov] [BEAM-12970][Playground]


------------------------------------------
[...truncated 49.66 KB...]
e26e7b75e334: Waiting
8228897593b2: Waiting
62a747bf1719: Waiting
b1a9e1733fc1: Waiting
0b3c02b5d746: Waiting
00ab2a787668: Pushed
388e7180b083: Pushed
5ebf909675aa: Pushed
56fb06668e82: Pushed
78a36dbe60ed: Pushed
f90c75b6eb72: Pushed
a0ac71b586b2: Pushed
1ec9e1de6222: Pushed
78700b6b35d0: Layer already exists
62a5b8741e83: Layer already exists
36e0782f1159: Layer already exists
ba6e5ff31f23: Layer already exists
9f9f651e9303: Layer already exists
8228897593b2: Pushed
0b3c02b5d746: Layer already exists
b1a9e1733fc1: Pushed
62a747bf1719: Layer already exists
91e2eeb5b486: Pushed
e26e7b75e334: Pushed
20211028124334: digest: sha256:45b73fe66dba08219be66a8bfb80995617d5dd6f86954ea5cc1b0a716ecf80a4 size: 4311

> Task :sdks:java:testing:load-tests:run
Oct 28, 2021 12:45:46 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Oct 28, 2021 12:45:46 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 195 files. Enable logging at DEBUG level to see which files will be staged.
Oct 28, 2021 12:45:47 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Oct 28, 2021 12:45:47 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Oct 28, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 195 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Oct 28, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 195 files cached, 0 files newly uploaded in 0 seconds
Oct 28, 2021 12:45:50 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Oct 28, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <108949 bytes, hash e8c831ec15148909a4024f127ebe2cb158a84663463ea98c935c2b2399f4861d> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-6Mgx7BUUiQmkAk8Sfr4ssVioRmNGPqmMk1wrI5n0hh0.pb
Oct 28, 2021 12:45:51 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Oct 28, 2021 12:45:52 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4bd51d3e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@33425811, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4b74b35, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@e4e1ef5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6d11ceef, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4cb2918c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@72e295cc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@c2584d3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6fa0450e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@37468787, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@51ec2856, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@714f3da4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1caa9eb6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1f53481b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2fcd7d3f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@27e7c77f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6f70a21b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6ae62c7e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2e869098, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@37c36608]
Oct 28, 2021 12:45:52 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Oct 28, 2021 12:45:52 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Oct 28, 2021 12:45:52 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Oct 28, 2021 12:45:52 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Oct 28, 2021 12:45:52 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@53a7a60c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@28bdbe88, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6a87026, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@ef60710, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@600f5704, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2503ec73, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@606f81b5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5e1fc42f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@44b21f9f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7ee8130e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6296474f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4288d98e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@169268a7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@285c6918, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@78a0ff63, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7c601d50, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@79b2852b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@326d27ac, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4d499d65, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@313f8301]
Oct 28, 2021 12:45:52 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Oct 28, 2021 12:45:52 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Oct 28, 2021 12:45:52 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Oct 28, 2021 12:45:52 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Oct 28, 2021 12:45:52 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Oct 28, 2021 12:45:52 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Oct 28, 2021 12:45:52 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Oct 28, 2021 12:45:52 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Oct 28, 2021 12:45:52 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Oct 28, 2021 12:45:52 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Oct 28, 2021 12:45:52 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Oct 28, 2021 12:45:52 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.35.0-SNAPSHOT
Oct 28, 2021 12:45:53 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-10-28_05_45_52-1818316849038594164?project=apache-beam-testing
Oct 28, 2021 12:45:53 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-10-28_05_45_52-1818316849038594164
Oct 28, 2021 12:45:53 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-10-28_05_45_52-1818316849038594164
Oct 28, 2021 12:46:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-10-28T12:45:59.114Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-10-69kp. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Oct 28, 2021 12:46:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-28T12:46:02.703Z: Worker configuration: e2-standard-2 in us-central1-a.
Oct 28, 2021 12:46:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-28T12:46:03.428Z: Expanding SplittableParDo operations into optimizable parts.
Oct 28, 2021 12:46:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-28T12:46:03.463Z: Expanding CollectionToSingleton operations into optimizable parts.
Oct 28, 2021 12:46:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-28T12:46:03.529Z: Expanding CoGroupByKey operations into optimizable parts.
Oct 28, 2021 12:46:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-28T12:46:03.597Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Oct 28, 2021 12:46:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-28T12:46:03.636Z: Expanding GroupByKey operations into streaming Read/Write steps
Oct 28, 2021 12:46:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-28T12:46:03.715Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Oct 28, 2021 12:46:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-28T12:46:03.835Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Oct 28, 2021 12:46:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-28T12:46:03.882Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Oct 28, 2021 12:46:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-28T12:46:03.913Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Oct 28, 2021 12:46:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-28T12:46:03.947Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Oct 28, 2021 12:46:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-28T12:46:03.973Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Oct 28, 2021 12:46:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-28T12:46:04.005Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Oct 28, 2021 12:46:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-28T12:46:04.039Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Oct 28, 2021 12:46:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-28T12:46:04.074Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Oct 28, 2021 12:46:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-28T12:46:04.114Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Oct 28, 2021 12:46:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-28T12:46:04.151Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Oct 28, 2021 12:46:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-28T12:46:04.196Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Oct 28, 2021 12:46:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-28T12:46:04.222Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Oct 28, 2021 12:46:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-28T12:46:04.282Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Oct 28, 2021 12:46:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-28T12:46:04.322Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Oct 28, 2021 12:46:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-28T12:46:04.344Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Oct 28, 2021 12:46:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-28T12:46:04.377Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Oct 28, 2021 12:46:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-28T12:46:04.413Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Oct 28, 2021 12:46:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-28T12:46:04.446Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Oct 28, 2021 12:46:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-28T12:46:04.475Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Oct 28, 2021 12:46:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-28T12:46:04.503Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Oct 28, 2021 12:46:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-28T12:46:04.561Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Oct 28, 2021 12:46:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-28T12:46:04.588Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Oct 28, 2021 12:46:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-28T12:46:04.623Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Oct 28, 2021 12:46:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-28T12:46:04.948Z: Starting 5 ****s in us-central1-a...
Oct 28, 2021 12:46:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-28T12:46:28.643Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Oct 28, 2021 12:46:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-28T12:46:55.045Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Oct 28, 2021 12:47:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-28T12:47:50.846Z: Workers have started successfully.
Oct 28, 2021 12:47:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-28T12:47:50.875Z: Workers have started successfully.
Oct 28, 2021 4:00:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-28T16:00:31.765Z: Cancel request is committed for workflow job: 2021-10-28_05_45_52-1818316849038594164.
Oct 28, 2021 4:00:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-28T16:00:31.826Z: Cleaning up.
Oct 28, 2021 4:00:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-28T16:00:31.906Z: Stopping **** pool...
Oct 28, 2021 4:00:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-28T16:00:31.969Z: Stopping **** pool...
Oct 28, 2021 4:02:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-28T16:02:54.314Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Oct 28, 2021 4:02:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-28T16:02:54.353Z: Worker pool stopped.
Oct 28, 2021 4:03:01 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-10-28_05_45_52-1818316849038594164 finished with status CANCELLED.
Load test results for test (ID): 86586faa-85c5-475e-bb26-f279091994f4 and timestamp: 2021-10-28T12:45:47.153000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11535.023
dataflow_v2_java11_total_bytes_count             2.01960346E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211028124334
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:45b73fe66dba08219be66a8bfb80995617d5dd6f86954ea5cc1b0a716ecf80a4
Deleted: sha256:7b8a540f712ea0eb598ca8ddd270c5710ac891842758fa2d3455c0db502f40ab
Deleted: sha256:f372086c4f5ae7c7c0ce763fac9dc6bc4161736301f5aaec66878b99b0502529
Deleted: sha256:458ecf7a865d8eba4376c70146397e0d5b02237745ccca939932e41d96735d16
Deleted: sha256:46ea50f1866802b31d07b4ab19c4fe70eaa990075daed270f88a27ee6e9d15f3
Deleted: sha256:b07d659ec5f4ed01f6548df29c5749e72d8b2177b51846ad8b9c02b01fa86a37
Deleted: sha256:dfe0f31e6e8c8a66fcc6235dae26630e120ef103bc016d77cf6247e53ce832b3
Deleted: sha256:ff388a048e8268cb88e9cdabaf56126c685e6156af10d2fa02126cf43bd6de5f
Deleted: sha256:309a59039c3fc7e4f0961bccda512d9fd6243bdd0d93e24bb72da28e6a74d651
Deleted: sha256:1960228fd9a804111d33d6fa1a462fed3d1385ece321a13b4c9bfe7d2eacf109
Deleted: sha256:239020881b140115586d9538f1ebc55effe8ec829942ab2b0e4ccbcfbfe32e30
Deleted: sha256:0543010a40f2c412a69a538eb3f472afd1e6799de50774caa90907e1252d9c30
Deleted: sha256:48a9f8e53cb05195f2a0ba5e69090ed4dfce8fec86bb525fc3384a5bddb7de2e
Deleted: sha256:dda7348a148f4058c9ba0d141421c7458590e30619c36e3c635ab2c04bb967b6
Deleted: sha256:fc422520ced6e158c3e0d16e44679f829892d5ef7cbdd5180b18eb6ecb3e4196
Deleted: sha256:7915ed67eb5ed08f5945bb3be29e2929e6fc3a3901d601ff1ef5a7df94607b68
Deleted: sha256:04f804135e73a663e67bc2ea1ce7cdb6f4ba0433364db5bebe9b670c74d928d5
Deleted: sha256:64c78e03070c97889adea60a295fe3249cb2d0d8f8de214579679b96ed81ac3d
Deleted: sha256:20dacd2ebe0b995e99a99aeb1379845ab556f4dc4a46a7999225953e1137c125
Deleted: sha256:bc97d320597c908480ef5b16e75151a9709dd99105b7f4e7208477783354e194
Deleted: sha256:e501e220c159658f272666a412f7fa81fcecc117c2d119854abcd0f27a4b0a56
Deleted: sha256:978b6f728cb2549bc8febbdf82552535701ff9a030c62e836d3c9255b7424cc7
Deleted: sha256:661bb97f38323578ff4497b96cf563f3a3202bb5dd36399daf4c6bfa80e99e17
Deleted: sha256:0d620287c71344ac5ee1423d8e349e7e6b955774457b1fa06cbab7e3873fba96
Deleted: sha256:477cc05977e4d760baa2b03387050ece45b170c7dbde14883f16224bd8c334c1
Deleted: sha256:bfc349587255729d9565b1997d69e9b82a13c748e2913b4fb89a721b6f1fc2e9
Deleted: sha256:7600d5c7e778546d22c284952da4e6a9449e18df9190a03c598b98c10de4740d
Deleted: sha256:17814636cff4f3683b8f05b3c293d63819f462bb24b572915c82e2602c0e73d7
Deleted: sha256:bf514f3871b7fb954332f045376680a44cb3e86f572d65e8989060e699df4612
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211028124334]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:45b73fe66dba08219be66a8bfb80995617d5dd6f86954ea5cc1b0a716ecf80a4]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211028124334] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:45b73fe66dba08219be66a8bfb80995617d5dd6f86954ea5cc1b0a716ecf80a4])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:45b73fe66dba08219be66a8bfb80995617d5dd6f86954ea5cc1b0a716ecf80a4
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:45b73fe66dba08219be66a8bfb80995617d5dd6f86954ea5cc1b0a716ecf80a4
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:45b73fe66dba08219be66a8bfb80995617d5dd6f86954ea5cc1b0a716ecf80a4].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 44s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/eujrlkngy34li

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #132

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/132/display/redirect?page=changes>

Changes:

[baeminbo] [BEAM-6721] Set numShards dynamically for TextIO.write()

[heejong] [BEAM-12978] Customizable dependency for Java external transform

[melissapa] [BEAM-11758] Update basics page: Splittable DoFn

[heejong] add ConfigT param to getDependencies method

[Alexey Romanenko] [BEAM-13104] ParquetIO: SplitReadFn must read the whole block

[noreply] Change sql.Options to an interface under sqlx. (#15790)

[melissapa] Address review feedback

[noreply] [BEAM-4149] Ensure that we always provide and require the worker id.

[noreply] [BEAM-13098] Fix translation of repeated TableRow fields (#15779)

[mmack] [BEAM-8958] Use AWS credentials provider with BasicKinesisProvider (AWS


------------------------------------------
[...truncated 50.04 KB...]
0b3c02b5d746: Waiting
a1157078efa7: Pushed
f1f6158ece23: Pushed
ca9cdaf33614: Pushed
d06b9c13e0cc: Pushed
f3f8ce23f305: Pushed
b1513d622f1f: Pushed
fb827792f82e: Pushed
1a8f734413d4: Pushed
78700b6b35d0: Layer already exists
62a5b8741e83: Layer already exists
c6149d467a89: Pushed
36e0782f1159: Layer already exists
ba6e5ff31f23: Layer already exists
9f9f651e9303: Layer already exists
0b3c02b5d746: Layer already exists
62a747bf1719: Layer already exists
7c70d01ceb3f: Pushed
e95d1c8a7989: Pushed
0e9f430337a7: Pushed
20211027124331: digest: sha256:a9836a031fe12e537fab384f6e8347c580e4f4bee44384a5e4f40294e504d354 size: 4311

> Task :sdks:java:testing:load-tests:run
Oct 27, 2021 12:45:20 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Oct 27, 2021 12:45:20 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 195 files. Enable logging at DEBUG level to see which files will be staged.
Oct 27, 2021 12:45:21 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Oct 27, 2021 12:45:21 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Oct 27, 2021 12:45:23 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 195 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Oct 27, 2021 12:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 195 files cached, 0 files newly uploaded in 1 seconds
Oct 27, 2021 12:45:25 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Oct 27, 2021 12:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <108949 bytes, hash cf636970a24e703fddec73b4e5776da0dfdc9be12795ffd4683708b270884f75> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-z2NpcKJOcD_d7HO05XdtoN_cm-Enlf_UaDcIsnCIT3U.pb
Oct 27, 2021 12:45:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Oct 27, 2021 12:45:26 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1d805aa1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@176f7f3b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@30ca0779, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58740366, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@47be0f9b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2bc426f0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4bd51d3e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@33425811, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4b74b35, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@e4e1ef5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6d11ceef, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4cb2918c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@72e295cc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@c2584d3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6fa0450e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@37468787, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@51ec2856, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@714f3da4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1caa9eb6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1f53481b]
Oct 27, 2021 12:45:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Oct 27, 2021 12:45:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Oct 27, 2021 12:45:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Oct 27, 2021 12:45:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Oct 27, 2021 12:45:26 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6242ae3b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@65ddee5a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@56399b9e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@34b9eb03, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43fda8d9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@49d831c2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@53a7a60c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@28bdbe88, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6a87026, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@ef60710, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@600f5704, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2503ec73, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@606f81b5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5e1fc42f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@44b21f9f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7ee8130e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6296474f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4288d98e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@169268a7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@285c6918]
Oct 27, 2021 12:45:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Oct 27, 2021 12:45:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Oct 27, 2021 12:45:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Oct 27, 2021 12:45:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Oct 27, 2021 12:45:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Oct 27, 2021 12:45:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Oct 27, 2021 12:45:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Oct 27, 2021 12:45:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Oct 27, 2021 12:45:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Oct 27, 2021 12:45:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Oct 27, 2021 12:45:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Oct 27, 2021 12:45:26 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.35.0-SNAPSHOT
Oct 27, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-10-27_05_45_26-11863955454072432916?project=apache-beam-testing
Oct 27, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-10-27_05_45_26-11863955454072432916
Oct 27, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-10-27_05_45_26-11863955454072432916
Oct 27, 2021 12:45:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-10-27T12:45:34.064Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-10-zy8s. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Oct 27, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-27T12:45:39.151Z: Worker configuration: e2-standard-2 in us-central1-a.
Oct 27, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-27T12:45:39.972Z: Expanding SplittableParDo operations into optimizable parts.
Oct 27, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-27T12:45:40.001Z: Expanding CollectionToSingleton operations into optimizable parts.
Oct 27, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-27T12:45:40.070Z: Expanding CoGroupByKey operations into optimizable parts.
Oct 27, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-27T12:45:40.153Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Oct 27, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-27T12:45:40.187Z: Expanding GroupByKey operations into streaming Read/Write steps
Oct 27, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-27T12:45:40.266Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Oct 27, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-27T12:45:40.371Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Oct 27, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-27T12:45:40.397Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Oct 27, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-27T12:45:40.444Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Oct 27, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-27T12:45:40.481Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Oct 27, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-27T12:45:40.509Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Oct 27, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-27T12:45:40.567Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Oct 27, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-27T12:45:40.599Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Oct 27, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-27T12:45:40.643Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Oct 27, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-27T12:45:40.671Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Oct 27, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-27T12:45:40.714Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Oct 27, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-27T12:45:40.753Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Oct 27, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-27T12:45:40.786Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Oct 27, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-27T12:45:40.820Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Oct 27, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-27T12:45:40.851Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Oct 27, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-27T12:45:40.885Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Oct 27, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-27T12:45:40.912Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Oct 27, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-27T12:45:40.944Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Oct 27, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-27T12:45:40.979Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Oct 27, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-27T12:45:41.013Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Oct 27, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-27T12:45:41.045Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Oct 27, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-27T12:45:41.073Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Oct 27, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-27T12:45:41.105Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Oct 27, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-27T12:45:41.134Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Oct 27, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-27T12:45:41.551Z: Starting 5 ****s in us-central1-a...
Oct 27, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-27T12:45:54.265Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Oct 27, 2021 12:46:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-27T12:46:26.561Z: Autoscaling: Raised the number of ****s to 3 so that the pipeline can catch up with its backlog and keep up with its input rate.
Oct 27, 2021 12:46:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-27T12:46:26.649Z: Resized **** pool to 3, though goal was 5.  This could be a quota issue.
Oct 27, 2021 12:46:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-27T12:46:36.969Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Oct 27, 2021 12:47:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-27T12:47:28.303Z: Workers have started successfully.
Oct 27, 2021 12:47:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-27T12:47:28.536Z: Workers have started successfully.
Oct 27, 2021 4:00:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-27T16:00:24.885Z: Cancel request is committed for workflow job: 2021-10-27_05_45_26-11863955454072432916.
Oct 27, 2021 4:00:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-27T16:00:28.844Z: Cleaning up.
Oct 27, 2021 4:00:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-27T16:00:29.026Z: Stopping **** pool...
Oct 27, 2021 4:00:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-27T16:00:29.115Z: Stopping **** pool...
Oct 27, 2021 4:02:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-27T16:02:51.327Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Oct 27, 2021 4:02:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-27T16:02:51.363Z: Worker pool stopped.
Oct 27, 2021 4:02:58 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-10-27_05_45_26-11863955454072432916 finished with status CANCELLED.
Load test results for test (ID): 22aa9eb4-7c1b-40ef-a3fd-377cfa270688 and timestamp: 2021-10-27T12:45:21.120000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11468.212
dataflow_v2_java11_total_bytes_count             2.41295807E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211027124331
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:a9836a031fe12e537fab384f6e8347c580e4f4bee44384a5e4f40294e504d354
Deleted: sha256:29a2fb30aabd6306152aea1c69a13e6161b03e9308e3a2771c37aacdb9f95556
Deleted: sha256:4fcdc721c79e58a7657fe1f07320c5b17ca1e681701504dcb8c1118b7b204126
Deleted: sha256:3eb87ca63c0408c9821d87f75407fca2356d9d57040fdda86ba0a62c0c96841a
Deleted: sha256:6011c3fe28996fcfcfa7f524cf616d6d08b8e4bed230eff3b48a7a49a68030f3
Deleted: sha256:9c9183b684eb3424ba90fab5516b2aa37db460768d23a17cf88730a9a63178de
Deleted: sha256:1c38578c3431f8c9ee3b49b5ee2d0e9599f27a64fa6902cfd3dd2edaac4fc904
Deleted: sha256:5fde1c63f53c457a9ed24bd050bacd89a43251164ce7cfafe7accf29a31c592a
Deleted: sha256:3dbfb0a3c5ad0267c36289166b5bfe23a5bfd346932faf855af7e183074de86e
Deleted: sha256:9f059aa9a9adc643802c7a6b3dce15c9b87178be55058bac0407838803e0734a
Deleted: sha256:f708312fbf668adfda1e4e92168f2be02f9d4b5f7799517531343acdbfba3658
Deleted: sha256:0b25710b042a20e99ce0605efa0454565b2a72f9f5e46bb91e2bab7d7f1c7367
Deleted: sha256:484cdea898879b77713ed2895d89e34ba45e36138815eddd146f3beac6e3d74d
Deleted: sha256:e3e97a9fc570a6cd89cf1e29bb6684bee9981ccad0f92d4f645288d47aabd138
Deleted: sha256:84886734949dee63c66d8a28b79744627f3f281c8e6a4d17384061e32d608d66
Deleted: sha256:3caaab00c05f6dfc030aa0595e4cea9b535b772c1858b9df96a5dcdaffaf8a7b
Deleted: sha256:a3250ae93ff833ac5142c19be09a14b2c77ab6b6148033b238eba8552cfeb98b
Deleted: sha256:f694412198b1a3a88e22608bf6acfb9c72c298bdcf2e8daa1a633917f3b1b5fa
Deleted: sha256:ff21c0dab017c561d384d8a7b6cc011aab5e0d70abdde54c21ada35d9e813f57
Deleted: sha256:5cf651323ddd4b2ae4f0e1d7c1346abd51960e1dbcea714ac1b166bcb96f9fef
Deleted: sha256:a57adda0fc8c30c95eb4b3fa522d0a10d1fdb06421dcbc999062987b4531e240
Deleted: sha256:cfc7ace7196831e4cba0cffbd93bf06aa5886b07d1cc7d1231302d503a449fc5
Deleted: sha256:60cbbe023d727747a080fc87dc3542993ae473c442fd1b61a1cd512bfdd39853
Deleted: sha256:1395c040125fca7d96ceba8cc0a27a219b49534df00f5797c3d1a94e419c8963
Deleted: sha256:8dc30115a541273a64c24df0cb6fe5f83955955eab5de2da7588b376d95dc720
Deleted: sha256:ce966b3fe9d0f2837d297660d692ce3dc72c2e4589ca22f1f3c70588a4e7665f
Deleted: sha256:7bc284fc567a3e35be0f9a04d39b93adf8486074888157675da5dc8a93d5ebcc
Deleted: sha256:b677c0d8b1da00e9d43d979e6496c6fc2203fb70e34e12fd8d3193abf2d73eab
Deleted: sha256:bc7d5a1df07c51e3a4d8b95220ae0b2098a9cee9e2c6bdb725b45ba6023d134d
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211027124331]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:a9836a031fe12e537fab384f6e8347c580e4f4bee44384a5e4f40294e504d354]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211027124331] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:a9836a031fe12e537fab384f6e8347c580e4f4bee44384a5e4f40294e504d354])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:a9836a031fe12e537fab384f6e8347c580e4f4bee44384a5e4f40294e504d354
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:a9836a031fe12e537fab384f6e8347c580e4f4bee44384a5e4f40294e504d354
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:a9836a031fe12e537fab384f6e8347c580e4f4bee44384a5e4f40294e504d354].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 43s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/okbgq2jqeo3fw

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #131

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/131/display/redirect?page=changes>

Changes:

[Luke Cwik] [BEAM-12522] Enable side inputs on all splittable DoFn execution time

[aydar.zaynutdinov] [BEAM-13062][Playground]

[ilya.kozyrev] Add environment_service.go and structures for beam sdk, network envs,

[aydar.zaynutdinov] [BEAM-13062][Playground]

[noreply] Merge pull request #15772 from [BEAM-13032] [Playground] Implement the

[noreply] Merge pull request #15714 from [BEAM-13005] [Playground] Implement local

[noreply] Merge pull request #15770 from [BEAM-13095][Playground] Using working

[noreply] [BEAM-11758] Update basics page: Aggregation, Runner, UDF, Schema

[noreply] Merge pull request #15783 from [BEAM-13048] [Playground] Add shortcuts

[Luke Cwik] [BEAM-8543] Fix test filtering for Dataflow Runner V2 to exclude

[noreply] Merge pull request #15744 from [BEAM-13072][Playground] Executor builder


------------------------------------------
[...truncated 49.23 KB...]
44671907d352: Pushed
22c6cd0d30a3: Pushed
8f8c6ec877ad: Pushed
90a44032420d: Pushed
7eea6e1f20fe: Pushed
9996e7cf888a: Pushed
9ee9aabadd72: Pushed
2116646bc5a6: Pushed
c9bb102d5ff2: Pushed
78700b6b35d0: Layer already exists
b6f1f004368f: Pushed
62a5b8741e83: Layer already exists
36e0782f1159: Layer already exists
ba6e5ff31f23: Layer already exists
0b3c02b5d746: Layer already exists
9f9f651e9303: Layer already exists
62a747bf1719: Layer already exists
358de67a5df2: Pushed
20211026124331: digest: sha256:c38f0177a7ef3f57599c16a2ef5297bf189e09217ebd02d1afefe1ff41eb7208 size: 4311

> Task :sdks:java:testing:load-tests:run
Oct 26, 2021 12:45:17 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Oct 26, 2021 12:45:18 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 195 files. Enable logging at DEBUG level to see which files will be staged.
Oct 26, 2021 12:45:18 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Oct 26, 2021 12:45:18 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Oct 26, 2021 12:45:20 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 195 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Oct 26, 2021 12:45:21 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 195 files cached, 0 files newly uploaded in 0 seconds
Oct 26, 2021 12:45:21 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Oct 26, 2021 12:45:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <108949 bytes, hash 64e91be1f694bcc161b27a0aac7b95ea2b0777def76fda00e27b537035695b40> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-ZOkb4faUvMFhsnoKrHuV6isHd973b9oA4ntTcDVpW0A.pb
Oct 26, 2021 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Oct 26, 2021 12:45:22 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@176f7f3b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@30ca0779, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58740366, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@47be0f9b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2bc426f0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4bd51d3e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@33425811, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4b74b35, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@e4e1ef5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6d11ceef, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4cb2918c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@72e295cc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@c2584d3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6fa0450e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@37468787, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@51ec2856, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@714f3da4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1caa9eb6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1f53481b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2fcd7d3f]
Oct 26, 2021 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Oct 26, 2021 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Oct 26, 2021 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Oct 26, 2021 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Oct 26, 2021 12:45:22 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@65ddee5a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@56399b9e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@34b9eb03, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43fda8d9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@49d831c2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@53a7a60c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@28bdbe88, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6a87026, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@ef60710, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@600f5704, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2503ec73, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@606f81b5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5e1fc42f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@44b21f9f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7ee8130e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6296474f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4288d98e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@169268a7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@285c6918, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@78a0ff63]
Oct 26, 2021 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Oct 26, 2021 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Oct 26, 2021 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Oct 26, 2021 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Oct 26, 2021 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Oct 26, 2021 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Oct 26, 2021 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Oct 26, 2021 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Oct 26, 2021 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Oct 26, 2021 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Oct 26, 2021 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Oct 26, 2021 12:45:22 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.35.0-SNAPSHOT
Oct 26, 2021 12:45:23 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-10-26_05_45_23-5340577711597487272?project=apache-beam-testing
Oct 26, 2021 12:45:23 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-10-26_05_45_23-5340577711597487272
Oct 26, 2021 12:45:23 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-10-26_05_45_23-5340577711597487272
Oct 26, 2021 12:45:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-10-26T12:45:34.029Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-10-2trs. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Oct 26, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-26T12:45:38.516Z: Worker configuration: e2-standard-2 in us-central1-a.
Oct 26, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-26T12:45:39.311Z: Expanding SplittableParDo operations into optimizable parts.
Oct 26, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-26T12:45:39.348Z: Expanding CollectionToSingleton operations into optimizable parts.
Oct 26, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-26T12:45:39.418Z: Expanding CoGroupByKey operations into optimizable parts.
Oct 26, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-26T12:45:39.489Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Oct 26, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-26T12:45:39.533Z: Expanding GroupByKey operations into streaming Read/Write steps
Oct 26, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-26T12:45:39.598Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Oct 26, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-26T12:45:39.709Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Oct 26, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-26T12:45:39.746Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Oct 26, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-26T12:45:39.767Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Oct 26, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-26T12:45:39.800Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Oct 26, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-26T12:45:39.823Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Oct 26, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-26T12:45:39.848Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Oct 26, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-26T12:45:39.878Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Oct 26, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-26T12:45:39.922Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Oct 26, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-26T12:45:39.960Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Oct 26, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-26T12:45:39.983Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Oct 26, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-26T12:45:40.010Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Oct 26, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-26T12:45:40.041Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Oct 26, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-26T12:45:40.074Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Oct 26, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-26T12:45:40.096Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Oct 26, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-26T12:45:40.119Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Oct 26, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-26T12:45:40.144Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Oct 26, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-26T12:45:40.165Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Oct 26, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-26T12:45:40.201Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Oct 26, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-26T12:45:40.239Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Oct 26, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-26T12:45:40.266Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Oct 26, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-26T12:45:40.289Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Oct 26, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-26T12:45:40.316Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Oct 26, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-26T12:45:40.346Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Oct 26, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-26T12:45:40.675Z: Starting 5 ****s in us-central1-a...
Oct 26, 2021 12:46:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-26T12:46:13.855Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Oct 26, 2021 12:46:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-26T12:46:26.776Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Oct 26, 2021 12:47:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-26T12:47:22.202Z: Workers have started successfully.
Oct 26, 2021 12:47:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-26T12:47:22.227Z: Workers have started successfully.
Oct 26, 2021 4:00:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-26T15:59:59.746Z: Workers have started successfully.
Oct 26, 2021 4:00:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-26T16:00:05.068Z: Worker configuration: e2-standard-2 in us-central1-a.
Oct 26, 2021 4:00:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-26T16:00:14.720Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Oct 26, 2021 4:00:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-26T16:00:44.681Z: Cancel request is committed for workflow job: 2021-10-26_05_45_23-5340577711597487272.
Oct 26, 2021 4:00:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-26T16:00:44.700Z: Cleaning up.
Oct 26, 2021 4:00:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-26T16:00:44.760Z: Stopping **** pool...
Oct 26, 2021 4:00:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-26T16:00:44.812Z: Stopping **** pool...
Oct 26, 2021 4:03:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-26T16:03:07.035Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Oct 26, 2021 4:03:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-26T16:03:07.092Z: Worker pool stopped.
Oct 26, 2021 4:03:11 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-10-26_05_45_23-5340577711597487272 finished with status CANCELLED.
Load test results for test (ID): 144ef194-9ac1-45b7-ae35-bf04bf36890b and timestamp: 2021-10-26T12:45:18.359000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11561.244
dataflow_v2_java11_total_bytes_count             3.54599776E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211026124331
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:c38f0177a7ef3f57599c16a2ef5297bf189e09217ebd02d1afefe1ff41eb7208
Deleted: sha256:0173f7450c392d6bb14fb4de6d46eb67dc7d9f283ab333c9ac1037685e0915e8
Deleted: sha256:229ced64d33038238616dbc315a9d53fc6daf3977c20ae2e3d8f7ca2236fc593
Deleted: sha256:a4cb16f3b6cc8b9b6209b909d864ce561a487faccb718b0c0533052f67430a12
Deleted: sha256:af9e27166951d99e7c71fdf97683616a40d3dc21692bbb0b4e37e02ba4585be9
Deleted: sha256:75225567acd366f7ef58e52b6afa57fb82a2d5c3024de09039ca8312b65c4eb7
Deleted: sha256:405b84633693e8d19eabc2c3830bca8b24173b3caf08963d10d9d74d66ab5511
Deleted: sha256:830e2b5f97458c160a1e81d2ed8d806d923cbba194104bc0cb892938b0e9fd79
Deleted: sha256:9aa97ded24ce51414ee4a3c60a4b5488cb07fd09305a7a72e7e7f270a5959ca6
Deleted: sha256:c1a99a5069e1065f7cbc24158f6d9993d9c15a0851967949220e182fcf5a9c7f
Deleted: sha256:cbf9161771b41b1989da9f333a137358b469a5cf0eaeff3d1d65a5c23ef693b1
Deleted: sha256:85d57d9f1b685783fb657faad82c569987c70fd053f56c055062f6318dc04fc9
Deleted: sha256:a831b46ba9c1d8db15494f5bbf3e57daa8b7d867941ce1c0c8a46e7e4155c7b9
Deleted: sha256:fcf162342282b136bb7b6f72dd5fcb629aedd622be540201cb2a56bd079b8628
Deleted: sha256:1a037865db1f41761303c36a094eb61ee81cb2dd668cc186464106e908c98d6d
Deleted: sha256:959299ca5acec1334818961249f12b323f80e99cdf6a10990fc8b822fd1beecb
Deleted: sha256:182a2c6fe00c825e65dcc48aad72d7bc91fc544628dceba3f64362a06bb2b83d
Deleted: sha256:05a730a8f86d6fc7eb526041841133ddf7feb512b6b8cb341af1dfe9bd157f73
Deleted: sha256:6acb8d63dcee226436e44bb27b72b3c36a0b5f1cc4b90a7933d17d47dfd0302f
Deleted: sha256:9aaa03a5de2dfbca994aa4130f92e8be2d57ff8eb1db8fd2d48e8f6de8d756ba
Deleted: sha256:62b1e578441e94418fcc6b5644f84c2b743256520ee867e8a9603f1558950a4a
Deleted: sha256:791b3fff9c2cb56a171506e594eadc46404bd6710216c5d04c33cd1d8da57df4
Deleted: sha256:3313c6224fba6a815009d706b0e2f56fc3297a28d3521d242aa0f9087479736e
Deleted: sha256:8766115a00680c46e87d5088fac12210a6a00200c7dd73ebb6ae8fb08df5f8fe
Deleted: sha256:52b58d9c9fd05801a4f97aa1edc1e3ba080325912da21ccaf55fccad9b2066da
Deleted: sha256:9fac7ff9230aea968c486225df5c92fe094a97db6b56aef62609feac0ce63ebe
Deleted: sha256:b04f74f01c98559b002fc2a966d5b9fef7d9b8aa5f66fc7555af326b1e816db9
Deleted: sha256:0be0c5cd7c32a82e8a9ab4379990c82dc2f6cfddc1fbdf1b193aec9069842b5f
Deleted: sha256:81fa0bfb87f7a739f45a5fc2e16c1f967e84573167aa211d9ffdcacc7bd65962
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211026124331]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:c38f0177a7ef3f57599c16a2ef5297bf189e09217ebd02d1afefe1ff41eb7208]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211026124331] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:c38f0177a7ef3f57599c16a2ef5297bf189e09217ebd02d1afefe1ff41eb7208])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:c38f0177a7ef3f57599c16a2ef5297bf189e09217ebd02d1afefe1ff41eb7208
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:c38f0177a7ef3f57599c16a2ef5297bf189e09217ebd02d1afefe1ff41eb7208
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:c38f0177a7ef3f57599c16a2ef5297bf189e09217ebd02d1afefe1ff41eb7208].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 57s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/flrimtg72akcy

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #130

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/130/display/redirect?page=changes>

Changes:

[noreply] Fixing BigQueryIO request too big corner case for streaming inserts


------------------------------------------
[...truncated 48.99 KB...]
831f7f5467c5: Preparing
10919810519a: Preparing
cf43989a6960: Preparing
5b29ab1822bb: Preparing
cf9967dcd6ec: Preparing
44ba0f20ac6e: Preparing
612d8cc2e68f: Preparing
07856eba2e32: Preparing
8a20526dc2b3: Preparing
ad69949f2cbf: Preparing
6ab646f5193f: Preparing
9cfc9240f18d: Preparing
78700b6b35d0: Preparing
62a5b8741e83: Preparing
36e0782f1159: Preparing
ba6e5ff31f23: Preparing
9f9f651e9303: Preparing
0b3c02b5d746: Preparing
62a747bf1719: Preparing
44ba0f20ac6e: Waiting
612d8cc2e68f: Waiting
07856eba2e32: Waiting
8a20526dc2b3: Waiting
ba6e5ff31f23: Waiting
9f9f651e9303: Waiting
ad69949f2cbf: Waiting
0b3c02b5d746: Waiting
62a747bf1719: Waiting
62a5b8741e83: Waiting
6ab646f5193f: Waiting
36e0782f1159: Waiting
9cfc9240f18d: Waiting
78700b6b35d0: Waiting
10919810519a: Pushed
cf43989a6960: Pushed
cf9967dcd6ec: Pushed
831f7f5467c5: Pushed
44ba0f20ac6e: Pushed
5b29ab1822bb: Pushed
07856eba2e32: Pushed
8a20526dc2b3: Pushed
78700b6b35d0: Layer already exists
62a5b8741e83: Layer already exists
36e0782f1159: Layer already exists
612d8cc2e68f: Pushed
ba6e5ff31f23: Layer already exists
6ab646f5193f: Pushed
9cfc9240f18d: Pushed
9f9f651e9303: Layer already exists
0b3c02b5d746: Layer already exists
62a747bf1719: Layer already exists
ad69949f2cbf: Pushed
20211025124334: digest: sha256:c08104e2d10664a4efb5b2332b04020be951c9d078fcbc95358524b0acf4c329 size: 4311

> Task :sdks:java:testing:load-tests:run
Oct 25, 2021 12:45:27 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Oct 25, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 195 files. Enable logging at DEBUG level to see which files will be staged.
Oct 25, 2021 12:45:28 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Oct 25, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Oct 25, 2021 12:45:31 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 195 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Oct 25, 2021 12:45:32 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 195 files cached, 0 files newly uploaded in 0 seconds
Oct 25, 2021 12:45:32 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Oct 25, 2021 12:45:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <108949 bytes, hash 91230eba4866ec41d6661f52d5d707332eaf22f64fe9fb97f55f01e55f3e72c9> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-kSMOukhm7EHWZh9S1dcHMy6vIvZP6fuX9V8B5V8-csk.pb
Oct 25, 2021 12:45:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Oct 25, 2021 12:45:33 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7fb66650, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1a96d94c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2a869a16, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@ae202c6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@46aa712c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6ada9c0c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7412ed6b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6e7c351d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b4a0aef, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@45cec376, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26d41711, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@298f0a0b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2b960a7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@31dfc6f5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@37b52340, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@663bb8ef, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2f4e40d7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@60e9c3a5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7e5843db, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@459f703f]
Oct 25, 2021 12:45:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Oct 25, 2021 12:45:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Oct 25, 2021 12:45:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Oct 25, 2021 12:45:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Oct 25, 2021 12:45:34 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@367f0121, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7da39774, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@441b8382, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1df1ced0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5349b246, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@32b0876c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2aaf152b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@39bbd9e0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@27fe9713, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@11c3ff67, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4397a639, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@24841372, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77114efe, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@79a7bfbc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77f68df, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3e4e4c1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7e7f3cfd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ae126d1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@46a488c2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6242ae3b]
Oct 25, 2021 12:45:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Oct 25, 2021 12:45:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Oct 25, 2021 12:45:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Oct 25, 2021 12:45:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Oct 25, 2021 12:45:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Oct 25, 2021 12:45:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Oct 25, 2021 12:45:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Oct 25, 2021 12:45:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Oct 25, 2021 12:45:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Oct 25, 2021 12:45:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Oct 25, 2021 12:45:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Oct 25, 2021 12:45:34 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.35.0-SNAPSHOT
Oct 25, 2021 12:45:34 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-10-25_05_45_34-11476855775060073623?project=apache-beam-testing
Oct 25, 2021 12:45:34 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-10-25_05_45_34-11476855775060073623
Oct 25, 2021 12:45:34 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-10-25_05_45_34-11476855775060073623
Oct 25, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-10-25T12:45:40.912Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-10-uvmg. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Oct 25, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-25T12:45:44.648Z: Worker configuration: e2-standard-2 in us-central1-a.
Oct 25, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-25T12:45:45.375Z: Expanding SplittableParDo operations into optimizable parts.
Oct 25, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-25T12:45:45.406Z: Expanding CollectionToSingleton operations into optimizable parts.
Oct 25, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-25T12:45:45.461Z: Expanding CoGroupByKey operations into optimizable parts.
Oct 25, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-25T12:45:45.538Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Oct 25, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-25T12:45:45.563Z: Expanding GroupByKey operations into streaming Read/Write steps
Oct 25, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-25T12:45:45.639Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Oct 25, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-25T12:45:45.752Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Oct 25, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-25T12:45:45.772Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Oct 25, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-25T12:45:45.816Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Oct 25, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-25T12:45:45.839Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Oct 25, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-25T12:45:45.872Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Oct 25, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-25T12:45:45.906Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Oct 25, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-25T12:45:45.928Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Oct 25, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-25T12:45:45.957Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Oct 25, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-25T12:45:45.981Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Oct 25, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-25T12:45:46.018Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Oct 25, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-25T12:45:46.043Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Oct 25, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-25T12:45:46.076Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Oct 25, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-25T12:45:46.110Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Oct 25, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-25T12:45:46.167Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Oct 25, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-25T12:45:46.200Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Oct 25, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-25T12:45:46.226Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Oct 25, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-25T12:45:46.248Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Oct 25, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-25T12:45:46.272Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Oct 25, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-25T12:45:46.303Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Oct 25, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-25T12:45:46.327Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Oct 25, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-25T12:45:46.355Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Oct 25, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-25T12:45:46.379Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Oct 25, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-25T12:45:46.410Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Oct 25, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-25T12:45:46.769Z: Starting 5 ****s in us-central1-a...
Oct 25, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-25T12:45:51.212Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Oct 25, 2021 12:46:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-25T12:46:28.879Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Oct 25, 2021 12:47:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-25T12:47:26.727Z: Workers have started successfully.
Oct 25, 2021 12:47:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-25T12:47:26.757Z: Workers have started successfully.
Oct 25, 2021 4:00:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-25T16:00:28.761Z: Cancel request is committed for workflow job: 2021-10-25_05_45_34-11476855775060073623.
Oct 25, 2021 4:00:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-25T16:00:28.835Z: Cleaning up.
Oct 25, 2021 4:00:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-25T16:00:28.921Z: Stopping **** pool...
Oct 25, 2021 4:00:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-25T16:00:28.997Z: Stopping **** pool...
Oct 25, 2021 4:02:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-25T16:02:48.560Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Oct 25, 2021 4:02:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-25T16:02:48.598Z: Worker pool stopped.
Oct 25, 2021 4:02:55 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-10-25_05_45_34-11476855775060073623 finished with status CANCELLED.
Load test results for test (ID): 98e688ea-807b-4e48-8998-6aa6c3809566 and timestamp: 2021-10-25T12:45:28.244000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11551.233
dataflow_v2_java11_total_bytes_count             2.46477647E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211025124334
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:c08104e2d10664a4efb5b2332b04020be951c9d078fcbc95358524b0acf4c329
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211025124334]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:c08104e2d10664a4efb5b2332b04020be951c9d078fcbc95358524b0acf4c329]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211025124334] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:c08104e2d10664a4efb5b2332b04020be951c9d078fcbc95358524b0acf4c329])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:c08104e2d10664a4efb5b2332b04020be951c9d078fcbc95358524b0acf4c329
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:c08104e2d10664a4efb5b2332b04020be951c9d078fcbc95358524b0acf4c329
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:c08104e2d10664a4efb5b2332b04020be951c9d078fcbc95358524b0acf4c329].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 42s
101 actionable tasks: 71 executed, 28 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/daorjlr55yyuc

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #129

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/129/display/redirect>

Changes:


------------------------------------------
[...truncated 48.57 KB...]
The push refers to repository [us.gcr.io/apache-beam-testing/java-postcommit-it/java]
b78ae8b444a3: Preparing
62d6b0f560f1: Preparing
814668c08eb1: Preparing
a968cd95a014: Preparing
e96b5b261848: Preparing
788b0e737a68: Preparing
8cc00797d551: Preparing
70217549fe31: Preparing
60ab889751cd: Preparing
1e0b3e8484d3: Preparing
c7b06e92a90f: Preparing
e8980afecadc: Preparing
78700b6b35d0: Preparing
62a5b8741e83: Preparing
36e0782f1159: Preparing
ba6e5ff31f23: Preparing
9f9f651e9303: Preparing
0b3c02b5d746: Preparing
62a747bf1719: Preparing
60ab889751cd: Waiting
788b0e737a68: Waiting
1e0b3e8484d3: Waiting
8cc00797d551: Waiting
c7b06e92a90f: Waiting
70217549fe31: Waiting
e8980afecadc: Waiting
78700b6b35d0: Waiting
62a5b8741e83: Waiting
62a747bf1719: Waiting
36e0782f1159: Waiting
9f9f651e9303: Waiting
ba6e5ff31f23: Waiting
62d6b0f560f1: Pushed
814668c08eb1: Pushed
e96b5b261848: Pushed
788b0e737a68: Pushed
b78ae8b444a3: Pushed
70217549fe31: Pushed
a968cd95a014: Pushed
60ab889751cd: Pushed
78700b6b35d0: Layer already exists
62a5b8741e83: Layer already exists
36e0782f1159: Layer already exists
ba6e5ff31f23: Layer already exists
9f9f651e9303: Layer already exists
c7b06e92a90f: Pushed
e8980afecadc: Pushed
0b3c02b5d746: Layer already exists
62a747bf1719: Layer already exists
8cc00797d551: Pushed
1e0b3e8484d3: Pushed
20211024124337: digest: sha256:5a124b45284c3cf8a3aefbedde277e9c1caf27422178312ebe7827bfd8bb4a3d size: 4311

> Task :sdks:java:testing:load-tests:run
Oct 24, 2021 12:45:47 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Oct 24, 2021 12:45:48 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 195 files. Enable logging at DEBUG level to see which files will be staged.
Oct 24, 2021 12:45:49 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Oct 24, 2021 12:45:49 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Oct 24, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 195 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Oct 24, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 195 files cached, 0 files newly uploaded in 0 seconds
Oct 24, 2021 12:45:51 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Oct 24, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <108949 bytes, hash 82ef116bc867718c215b522534d48287a8aee43ec7cd3d11a63531af932e4df6> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-gu8Ra8hncYwhW1IlNNSCh6iu5D7HzT0RpjUxr5MuTfY.pb
Oct 24, 2021 12:45:53 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Oct 24, 2021 12:45:53 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@ae202c6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@46aa712c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6ada9c0c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7412ed6b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6e7c351d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b4a0aef, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@45cec376, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26d41711, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@298f0a0b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2b960a7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@31dfc6f5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@37b52340, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@663bb8ef, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2f4e40d7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@60e9c3a5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7e5843db, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@459f703f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@188ac8a3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3650d4fc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@aaa0f76]
Oct 24, 2021 12:45:53 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Oct 24, 2021 12:45:53 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Oct 24, 2021 12:45:53 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Oct 24, 2021 12:45:53 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Oct 24, 2021 12:45:53 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1df1ced0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5349b246, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@32b0876c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2aaf152b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@39bbd9e0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@27fe9713, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@11c3ff67, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4397a639, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@24841372, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77114efe, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@79a7bfbc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77f68df, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3e4e4c1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7e7f3cfd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ae126d1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@46a488c2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6242ae3b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@65ddee5a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@56399b9e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@34b9eb03]
Oct 24, 2021 12:45:53 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Oct 24, 2021 12:45:53 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Oct 24, 2021 12:45:53 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Oct 24, 2021 12:45:53 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Oct 24, 2021 12:45:53 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Oct 24, 2021 12:45:53 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Oct 24, 2021 12:45:53 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Oct 24, 2021 12:45:53 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Oct 24, 2021 12:45:53 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Oct 24, 2021 12:45:53 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Oct 24, 2021 12:45:53 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Oct 24, 2021 12:45:53 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.35.0-SNAPSHOT
Oct 24, 2021 12:45:54 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-10-24_05_45_53-15648205050698846936?project=apache-beam-testing
Oct 24, 2021 12:45:54 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-10-24_05_45_53-15648205050698846936
Oct 24, 2021 12:45:54 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-10-24_05_45_53-15648205050698846936
Oct 24, 2021 12:46:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-10-24T12:46:03.797Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-10-11tb. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Oct 24, 2021 12:46:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-24T12:46:08.314Z: Worker configuration: e2-standard-2 in us-central1-a.
Oct 24, 2021 12:46:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-24T12:46:09.026Z: Expanding SplittableParDo operations into optimizable parts.
Oct 24, 2021 12:46:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-24T12:46:09.050Z: Expanding CollectionToSingleton operations into optimizable parts.
Oct 24, 2021 12:46:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-24T12:46:09.111Z: Expanding CoGroupByKey operations into optimizable parts.
Oct 24, 2021 12:46:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-24T12:46:09.181Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Oct 24, 2021 12:46:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-24T12:46:09.211Z: Expanding GroupByKey operations into streaming Read/Write steps
Oct 24, 2021 12:46:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-24T12:46:09.266Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Oct 24, 2021 12:46:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-24T12:46:09.375Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Oct 24, 2021 12:46:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-24T12:46:09.412Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Oct 24, 2021 12:46:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-24T12:46:09.437Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Oct 24, 2021 12:46:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-24T12:46:09.475Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Oct 24, 2021 12:46:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-24T12:46:09.506Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Oct 24, 2021 12:46:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-24T12:46:09.530Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Oct 24, 2021 12:46:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-24T12:46:09.555Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Oct 24, 2021 12:46:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-24T12:46:09.578Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Oct 24, 2021 12:46:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-24T12:46:09.602Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Oct 24, 2021 12:46:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-24T12:46:09.628Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Oct 24, 2021 12:46:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-24T12:46:09.663Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Oct 24, 2021 12:46:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-24T12:46:09.693Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Oct 24, 2021 12:46:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-24T12:46:09.720Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Oct 24, 2021 12:46:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-24T12:46:09.753Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Oct 24, 2021 12:46:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-24T12:46:09.780Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Oct 24, 2021 12:46:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-24T12:46:09.807Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Oct 24, 2021 12:46:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-24T12:46:09.828Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Oct 24, 2021 12:46:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-24T12:46:09.861Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Oct 24, 2021 12:46:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-24T12:46:09.895Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Oct 24, 2021 12:46:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-24T12:46:09.928Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Oct 24, 2021 12:46:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-24T12:46:09.961Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Oct 24, 2021 12:46:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-24T12:46:09.991Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Oct 24, 2021 12:46:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-24T12:46:10.027Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Oct 24, 2021 12:46:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-24T12:46:10.379Z: Starting 5 ****s in us-central1-a...
Oct 24, 2021 12:46:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-24T12:46:14.214Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Oct 24, 2021 12:46:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-24T12:46:50.812Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Oct 24, 2021 12:47:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-24T12:47:44.534Z: Workers have started successfully.
Oct 24, 2021 12:47:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-24T12:47:44.556Z: Workers have started successfully.
Oct 24, 2021 4:00:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-24T16:00:26.295Z: Cancel request is committed for workflow job: 2021-10-24_05_45_53-15648205050698846936.
Oct 24, 2021 4:00:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-24T16:00:26.367Z: Cleaning up.
Oct 24, 2021 4:00:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-24T16:00:26.457Z: Stopping **** pool...
Oct 24, 2021 4:00:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-24T16:00:26.534Z: Stopping **** pool...
Oct 24, 2021 4:02:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-24T16:02:50.100Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Oct 24, 2021 4:02:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-24T16:02:50.141Z: Worker pool stopped.
Oct 24, 2021 4:02:55 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-10-24_05_45_53-15648205050698846936 finished with status CANCELLED.
Load test results for test (ID): ce635266-db81-4432-a184-265a9ed66a37 and timestamp: 2021-10-24T12:45:48.716000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11518.388
dataflow_v2_java11_total_bytes_count              1.5485639E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211024124337
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:5a124b45284c3cf8a3aefbedde277e9c1caf27422178312ebe7827bfd8bb4a3d
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211024124337]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:5a124b45284c3cf8a3aefbedde277e9c1caf27422178312ebe7827bfd8bb4a3d]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211024124337] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:5a124b45284c3cf8a3aefbedde277e9c1caf27422178312ebe7827bfd8bb4a3d])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:5a124b45284c3cf8a3aefbedde277e9c1caf27422178312ebe7827bfd8bb4a3d
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:5a124b45284c3cf8a3aefbedde277e9c1caf27422178312ebe7827bfd8bb4a3d
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:5a124b45284c3cf8a3aefbedde277e9c1caf27422178312ebe7827bfd8bb4a3d].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 38s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/4q3tm3yqk3k7m

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #128

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/128/display/redirect>

Changes:


------------------------------------------
[...truncated 49.31 KB...]
62a747bf1719: Waiting
0b3c02b5d746: Waiting
9f9f651e9303: Waiting
1813b4329c90: Pushed
5d8f284749f8: Pushed
58999aee1641: Pushed
cf476769792b: Pushed
57b76b70e2e9: Pushed
b61a7f33d3bd: Pushed
661388b38f1f: Pushed
54d9132de0ae: Pushed
1e5bbac8f158: Pushed
78700b6b35d0: Layer already exists
62a5b8741e83: Layer already exists
36e0782f1159: Layer already exists
ba6e5ff31f23: Layer already exists
2160cddcd869: Pushed
9f9f651e9303: Layer already exists
fa72c85d8524: Pushed
0b3c02b5d746: Layer already exists
62a747bf1719: Layer already exists
dcddb0ae3ae3: Pushed
20211023124333: digest: sha256:885d360e00e774e70efd4d3431bb813b11e1f2f3ec001fcceb8e64d4c922855b size: 4311

> Task :sdks:java:testing:load-tests:run
Oct 23, 2021 12:45:20 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Oct 23, 2021 12:45:21 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 195 files. Enable logging at DEBUG level to see which files will be staged.
Oct 23, 2021 12:45:21 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Oct 23, 2021 12:45:21 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Oct 23, 2021 12:45:23 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 195 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Oct 23, 2021 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 195 files cached, 0 files newly uploaded in 0 seconds
Oct 23, 2021 12:45:24 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Oct 23, 2021 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <108949 bytes, hash 7150d364fc0b39096a5e436a8ffa17d66347c9d2bc148256d8e76041b4a6df51> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-cVDTZPwLOQlqXkNqj_oX1mNHydK8FIJW2OdgQbSm31E.pb
Oct 23, 2021 12:45:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Oct 23, 2021 12:45:25 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7fb66650, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1a96d94c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2a869a16, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@ae202c6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@46aa712c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6ada9c0c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7412ed6b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6e7c351d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b4a0aef, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@45cec376, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26d41711, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@298f0a0b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2b960a7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@31dfc6f5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@37b52340, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@663bb8ef, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2f4e40d7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@60e9c3a5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7e5843db, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@459f703f]
Oct 23, 2021 12:45:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Oct 23, 2021 12:45:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Oct 23, 2021 12:45:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Oct 23, 2021 12:45:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Oct 23, 2021 12:45:26 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@367f0121, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7da39774, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@441b8382, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1df1ced0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5349b246, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@32b0876c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2aaf152b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@39bbd9e0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@27fe9713, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@11c3ff67, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4397a639, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@24841372, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77114efe, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@79a7bfbc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77f68df, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3e4e4c1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7e7f3cfd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ae126d1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@46a488c2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6242ae3b]
Oct 23, 2021 12:45:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Oct 23, 2021 12:45:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Oct 23, 2021 12:45:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Oct 23, 2021 12:45:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Oct 23, 2021 12:45:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Oct 23, 2021 12:45:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Oct 23, 2021 12:45:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Oct 23, 2021 12:45:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Oct 23, 2021 12:45:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Oct 23, 2021 12:45:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Oct 23, 2021 12:45:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Oct 23, 2021 12:45:26 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.35.0-SNAPSHOT
Oct 23, 2021 12:45:26 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-10-23_05_45_26-11254189268418989776?project=apache-beam-testing
Oct 23, 2021 12:45:26 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-10-23_05_45_26-11254189268418989776
Oct 23, 2021 12:45:26 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-10-23_05_45_26-11254189268418989776
Oct 23, 2021 12:45:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-10-23T12:45:33.710Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-10-gmis. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Oct 23, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-23T12:45:38.016Z: Worker configuration: e2-standard-2 in us-central1-a.
Oct 23, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-23T12:45:38.753Z: Expanding SplittableParDo operations into optimizable parts.
Oct 23, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-23T12:45:38.792Z: Expanding CollectionToSingleton operations into optimizable parts.
Oct 23, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-23T12:45:38.867Z: Expanding CoGroupByKey operations into optimizable parts.
Oct 23, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-23T12:45:38.939Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Oct 23, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-23T12:45:38.965Z: Expanding GroupByKey operations into streaming Read/Write steps
Oct 23, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-23T12:45:39.048Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Oct 23, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-23T12:45:39.141Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Oct 23, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-23T12:45:39.182Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Oct 23, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-23T12:45:39.207Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Oct 23, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-23T12:45:39.229Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Oct 23, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-23T12:45:39.264Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Oct 23, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-23T12:45:39.289Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Oct 23, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-23T12:45:39.323Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Oct 23, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-23T12:45:39.348Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Oct 23, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-23T12:45:39.384Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Oct 23, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-23T12:45:39.422Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Oct 23, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-23T12:45:39.455Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Oct 23, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-23T12:45:39.492Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Oct 23, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-23T12:45:39.515Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Oct 23, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-23T12:45:39.536Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Oct 23, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-23T12:45:39.571Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Oct 23, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-23T12:45:39.595Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Oct 23, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-23T12:45:39.619Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Oct 23, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-23T12:45:39.647Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Oct 23, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-23T12:45:39.674Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Oct 23, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-23T12:45:39.706Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Oct 23, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-23T12:45:39.729Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Oct 23, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-23T12:45:39.760Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Oct 23, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-23T12:45:39.807Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Oct 23, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-23T12:45:40.135Z: Starting 5 ****s in us-central1-a...
Oct 23, 2021 12:46:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-23T12:46:10.073Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Oct 23, 2021 12:46:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-23T12:46:23.009Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Oct 23, 2021 12:47:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-23T12:47:17.865Z: Workers have started successfully.
Oct 23, 2021 12:47:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-23T12:47:17.898Z: Workers have started successfully.
Oct 23, 2021 3:42:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-23T15:42:40.431Z: Staged package arrow-format-5.0.0-FzXW0Kc0wcM-4qPMnXkmsxGom9LbiofNT9gHF14a_l4.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/arrow-format-5.0.0-FzXW0Kc0wcM-4qPMnXkmsxGom9LbiofNT9gHF14a_l4.jar' is inaccessible.
Oct 23, 2021 3:42:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-23T15:42:40.531Z: Staged package arrow-memory-core-5.0.0-cTMnF35Hj_AzHUn7iWyDPIq8rG9o8ZjREpZtIg17P4M.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/arrow-memory-core-5.0.0-cTMnF35Hj_AzHUn7iWyDPIq8rG9o8ZjREpZtIg17P4M.jar' is inaccessible.
Oct 23, 2021 3:42:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-23T15:42:40.576Z: Staged package arrow-vector-5.0.0-qCLc2jGU2wVSqfy4RSUPuN0xKMz05B4sYNeFxQPEwnI.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/arrow-vector-5.0.0-qCLc2jGU2wVSqfy4RSUPuN0xKMz05B4sYNeFxQPEwnI.jar' is inaccessible.
Oct 23, 2021 3:42:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-10-23T15:42:44.341Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Oct 23, 2021 3:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-10-23T15:45:44.131Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Oct 23, 2021 3:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-23T15:48:40.486Z: Staged package arrow-format-5.0.0-FzXW0Kc0wcM-4qPMnXkmsxGom9LbiofNT9gHF14a_l4.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/arrow-format-5.0.0-FzXW0Kc0wcM-4qPMnXkmsxGom9LbiofNT9gHF14a_l4.jar' is inaccessible.
Oct 23, 2021 3:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-23T15:48:40.568Z: Staged package arrow-memory-core-5.0.0-cTMnF35Hj_AzHUn7iWyDPIq8rG9o8ZjREpZtIg17P4M.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/arrow-memory-core-5.0.0-cTMnF35Hj_AzHUn7iWyDPIq8rG9o8ZjREpZtIg17P4M.jar' is inaccessible.
Oct 23, 2021 3:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-23T15:48:40.600Z: Staged package arrow-vector-5.0.0-qCLc2jGU2wVSqfy4RSUPuN0xKMz05B4sYNeFxQPEwnI.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/arrow-vector-5.0.0-qCLc2jGU2wVSqfy4RSUPuN0xKMz05B4sYNeFxQPEwnI.jar' is inaccessible.
Oct 23, 2021 3:48:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-10-23T15:48:44.120Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Oct 23, 2021 3:51:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-10-23T15:51:43.991Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Oct 23, 2021 3:54:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-23T15:54:40.447Z: Staged package arrow-format-5.0.0-FzXW0Kc0wcM-4qPMnXkmsxGom9LbiofNT9gHF14a_l4.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/arrow-format-5.0.0-FzXW0Kc0wcM-4qPMnXkmsxGom9LbiofNT9gHF14a_l4.jar' is inaccessible.
Oct 23, 2021 3:54:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-23T15:54:40.488Z: Staged package arrow-memory-core-5.0.0-cTMnF35Hj_AzHUn7iWyDPIq8rG9o8ZjREpZtIg17P4M.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/arrow-memory-core-5.0.0-cTMnF35Hj_AzHUn7iWyDPIq8rG9o8ZjREpZtIg17P4M.jar' is inaccessible.
Oct 23, 2021 3:54:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-23T15:54:40.565Z: Staged package arrow-vector-5.0.0-qCLc2jGU2wVSqfy4RSUPuN0xKMz05B4sYNeFxQPEwnI.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/arrow-vector-5.0.0-qCLc2jGU2wVSqfy4RSUPuN0xKMz05B4sYNeFxQPEwnI.jar' is inaccessible.
Oct 23, 2021 3:54:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-10-23T15:54:44.183Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Oct 23, 2021 3:57:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-10-23T15:57:44.040Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Oct 23, 2021 4:00:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-23T16:00:33.057Z: Cancel request is committed for workflow job: 2021-10-23_05_45_26-11254189268418989776.
Oct 23, 2021 4:00:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-23T16:00:33.085Z: Cleaning up.
Oct 23, 2021 4:00:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-23T16:00:33.159Z: Stopping **** pool...
Oct 23, 2021 4:00:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-23T16:00:33.217Z: Stopping **** pool...
Oct 23, 2021 4:02:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-23T16:02:57.201Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Oct 23, 2021 4:02:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-23T16:02:57.236Z: Worker pool stopped.
Oct 23, 2021 4:03:02 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-10-23_05_45_26-11254189268418989776 finished with status CANCELLED.
Load test results for test (ID): 2e74a720-71a4-4c83-ab57-addad6fa1cad and timestamp: 2021-10-23T12:45:21.373000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11554.335
dataflow_v2_java11_total_bytes_count             2.49740768E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211023124333
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:885d360e00e774e70efd4d3431bb813b11e1f2f3ec001fcceb8e64d4c922855b
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211023124333]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:885d360e00e774e70efd4d3431bb813b11e1f2f3ec001fcceb8e64d4c922855b]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211023124333] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:885d360e00e774e70efd4d3431bb813b11e1f2f3ec001fcceb8e64d4c922855b])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:885d360e00e774e70efd4d3431bb813b11e1f2f3ec001fcceb8e64d4c922855b
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:885d360e00e774e70efd4d3431bb813b11e1f2f3ec001fcceb8e64d4c922855b
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:885d360e00e774e70efd4d3431bb813b11e1f2f3ec001fcceb8e64d4c922855b].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 45s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/v6q6rf57l3neo

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #127

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/127/display/redirect?page=changes>

Changes:

[noreply] Merge pull request #15761 from [BEAM-13008] Create gradle tasks for the

[noreply] [BEAM-13096] Double test timeout. (#15774)

[noreply] [BEAM-13019] Add `containsInAnyOrder` with matchers to the

[Daniel Oliveira] Avoiding read-only Go module cache in Gradle config.

[noreply] [BEAM-11758] Update basics page: Pipeline, PCollection, PTransform

[noreply] Test SetState addIfAbsent with no read (#15776)

[noreply] lazy creation of source splits for export-based ReadFromBigQuery

[noreply] [BEAM-11275] Support remote package download from remote filesystems in

[noreply] [BEAM-13015] Create a multiplexer that sends Elements based upon


------------------------------------------
[...truncated 53.59 KB...]
Oct 22, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Oct 22, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Oct 22, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Oct 22, 2021 12:45:27 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@441b8382, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1df1ced0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5349b246, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@32b0876c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2aaf152b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@39bbd9e0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@27fe9713, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@11c3ff67, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4397a639, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@24841372, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77114efe, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@79a7bfbc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77f68df, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3e4e4c1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7e7f3cfd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ae126d1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@46a488c2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6242ae3b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@65ddee5a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@56399b9e]
Oct 22, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Oct 22, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Oct 22, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Oct 22, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Oct 22, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Oct 22, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Oct 22, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Oct 22, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Oct 22, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Oct 22, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Oct 22, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Oct 22, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.35.0-SNAPSHOT
Oct 22, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-10-22_05_45_27-8754089351573128322?project=apache-beam-testing
Oct 22, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-10-22_05_45_27-8754089351573128322
Oct 22, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-10-22_05_45_27-8754089351573128322
Oct 22, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-10-22T12:45:40.746Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-10-2smy. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Oct 22, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-22T12:45:45.008Z: Worker configuration: e2-standard-2 in us-central1-a.
Oct 22, 2021 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-22T12:45:45.769Z: Expanding SplittableParDo operations into optimizable parts.
Oct 22, 2021 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-22T12:45:45.790Z: Expanding CollectionToSingleton operations into optimizable parts.
Oct 22, 2021 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-22T12:45:45.839Z: Expanding CoGroupByKey operations into optimizable parts.
Oct 22, 2021 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-22T12:45:45.898Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Oct 22, 2021 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-22T12:45:45.929Z: Expanding GroupByKey operations into streaming Read/Write steps
Oct 22, 2021 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-22T12:45:45.988Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Oct 22, 2021 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-22T12:45:46.089Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Oct 22, 2021 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-22T12:45:46.127Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Oct 22, 2021 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-22T12:45:46.159Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Oct 22, 2021 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-22T12:45:46.194Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Oct 22, 2021 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-22T12:45:46.219Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Oct 22, 2021 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-22T12:45:46.242Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Oct 22, 2021 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-22T12:45:46.268Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Oct 22, 2021 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-22T12:45:46.298Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Oct 22, 2021 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-22T12:45:46.334Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Oct 22, 2021 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-22T12:45:46.357Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Oct 22, 2021 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-22T12:45:46.390Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Oct 22, 2021 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-22T12:45:46.413Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Oct 22, 2021 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-22T12:45:46.430Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Oct 22, 2021 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-22T12:45:46.456Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Oct 22, 2021 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-22T12:45:46.478Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Oct 22, 2021 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-22T12:45:46.510Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Oct 22, 2021 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-22T12:45:46.542Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Oct 22, 2021 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-22T12:45:46.567Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Oct 22, 2021 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-22T12:45:46.594Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Oct 22, 2021 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-22T12:45:46.623Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Oct 22, 2021 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-22T12:45:46.644Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Oct 22, 2021 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-22T12:45:46.677Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Oct 22, 2021 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-22T12:45:46.708Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Oct 22, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-22T12:45:47.043Z: Starting 5 ****s in us-central1-a...
Oct 22, 2021 12:46:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-22T12:46:03.956Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Oct 22, 2021 12:46:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-22T12:46:31.151Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Oct 22, 2021 12:47:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-22T12:47:23.177Z: Workers have started successfully.
Oct 22, 2021 12:47:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-22T12:47:23.204Z: Workers have started successfully.
Oct 22, 2021 3:51:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-22T15:51:47.504Z: Staged package aws-java-sdk-cloudwatch-1.11.974-Ub2yicrz1p27ushPndFcVBigc3SuIZ4G-Yaq45B1iUo.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/aws-java-sdk-cloudwatch-1.11.974-Ub2yicrz1p27ushPndFcVBigc3SuIZ4G-Yaq45B1iUo.jar' is inaccessible.
Oct 22, 2021 3:51:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-22T15:51:47.578Z: Staged package aws-java-sdk-core-1.11.974-s3lPhICKsq1xExGOyj03VQHRqOsuX5YUc-UUlnhiA64.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/aws-java-sdk-core-1.11.974-s3lPhICKsq1xExGOyj03VQHRqOsuX5YUc-UUlnhiA64.jar' is inaccessible.
Oct 22, 2021 3:51:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-22T15:51:47.616Z: Staged package aws-java-sdk-dynamodb-1.11.974-RLvIDrX-TDIDN06gNnaHm9dQbuHCD_XSBesOuniSTk4.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/aws-java-sdk-dynamodb-1.11.974-RLvIDrX-TDIDN06gNnaHm9dQbuHCD_XSBesOuniSTk4.jar' is inaccessible.
Oct 22, 2021 3:51:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-22T15:51:47.674Z: Staged package aws-java-sdk-kinesis-1.11.974-kbU_CjbRpkrhhFeoyIMQA81nTYmniJY77w8Smd7PLpY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/aws-java-sdk-kinesis-1.11.974-kbU_CjbRpkrhhFeoyIMQA81nTYmniJY77w8Smd7PLpY.jar' is inaccessible.
Oct 22, 2021 3:51:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-22T15:51:47.728Z: Staged package aws-java-sdk-kms-1.11.974-HWH_mHclw3Wv4_GJ56x62xFDHSPSkZWjeKAlx40XrTk.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/aws-java-sdk-kms-1.11.974-HWH_mHclw3Wv4_GJ56x62xFDHSPSkZWjeKAlx40XrTk.jar' is inaccessible.
Oct 22, 2021 3:51:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-22T15:51:47.768Z: Staged package aws-java-sdk-s3-1.11.974-Qgd0wgaOuI_uyxxQI4Ulljk9bZEq-3JZArQF0zU5yzA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/aws-java-sdk-s3-1.11.974-Qgd0wgaOuI_uyxxQI4Ulljk9bZEq-3JZArQF0zU5yzA.jar' is inaccessible.
Oct 22, 2021 3:51:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-22T15:51:48.423Z: Staged package commons-io-2.6--HfTBGYKwqFC84ZbrfyXHex-1zx0fH-NXS9ROcpzZRM.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/commons-io-2.6--HfTBGYKwqFC84ZbrfyXHex-1zx0fH-NXS9ROcpzZRM.jar' is inaccessible.
Oct 22, 2021 3:51:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-22T15:51:48.458Z: Staged package commons-lang-2.6-UPEbCfh3wpTVbyRGP0fSj5Kc9QRPZIZhwPDPuumi9Jw.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/commons-lang-2.6-UPEbCfh3wpTVbyRGP0fSj5Kc9QRPZIZhwPDPuumi9Jw.jar' is inaccessible.
Oct 22, 2021 3:51:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-22T15:51:48.605Z: Staged package error_prone_annotations-2.3.4-uvfW6pfOYGxT4RtoVLpfLOfvXCTd3wr6GNEmC9JbACw.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/error_prone_annotations-2.3.4-uvfW6pfOYGxT4RtoVLpfLOfvXCTd3wr6GNEmC9JbACw.jar' is inaccessible.
Oct 22, 2021 3:51:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-22T15:51:49.908Z: Staged package ion-java-1.0.2-DRJ7IFofzgq8KjdXoEF0hlG8ZsFc9MBZusWDOyfUcaU.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/ion-java-1.0.2-DRJ7IFofzgq8KjdXoEF0hlG8ZsFc9MBZusWDOyfUcaU.jar' is inaccessible.
Oct 22, 2021 3:51:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-22T15:51:50.036Z: Staged package jackson-dataformat-cbor-2.12.4-xpFy2QEl4dUh55AUzgOueyVcIoNGuAdPK6G3l4P-yCc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/jackson-dataformat-cbor-2.12.4-xpFy2QEl4dUh55AUzgOueyVcIoNGuAdPK6G3l4P-yCc.jar' is inaccessible.
Oct 22, 2021 3:51:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-22T15:51:50.186Z: Staged package jmespath-java-1.11.974-QaUt5Hnw6wd7ULTQ5Oj4j_zpRr-rnMCdfGwiwkSo8-0.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/jmespath-java-1.11.974-QaUt5Hnw6wd7ULTQ5Oj4j_zpRr-rnMCdfGwiwkSo8-0.jar' is inaccessible.
Oct 22, 2021 3:51:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-10-22T15:51:51.457Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Oct 22, 2021 3:54:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-10-22T15:54:50.652Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Oct 22, 2021 3:57:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-22T15:57:47.484Z: Staged package aws-java-sdk-cloudwatch-1.11.974-Ub2yicrz1p27ushPndFcVBigc3SuIZ4G-Yaq45B1iUo.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/aws-java-sdk-cloudwatch-1.11.974-Ub2yicrz1p27ushPndFcVBigc3SuIZ4G-Yaq45B1iUo.jar' is inaccessible.
Oct 22, 2021 3:57:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-22T15:57:47.521Z: Staged package aws-java-sdk-core-1.11.974-s3lPhICKsq1xExGOyj03VQHRqOsuX5YUc-UUlnhiA64.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/aws-java-sdk-core-1.11.974-s3lPhICKsq1xExGOyj03VQHRqOsuX5YUc-UUlnhiA64.jar' is inaccessible.
Oct 22, 2021 3:57:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-22T15:57:47.568Z: Staged package aws-java-sdk-dynamodb-1.11.974-RLvIDrX-TDIDN06gNnaHm9dQbuHCD_XSBesOuniSTk4.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/aws-java-sdk-dynamodb-1.11.974-RLvIDrX-TDIDN06gNnaHm9dQbuHCD_XSBesOuniSTk4.jar' is inaccessible.
Oct 22, 2021 3:57:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-22T15:57:47.618Z: Staged package aws-java-sdk-kinesis-1.11.974-kbU_CjbRpkrhhFeoyIMQA81nTYmniJY77w8Smd7PLpY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/aws-java-sdk-kinesis-1.11.974-kbU_CjbRpkrhhFeoyIMQA81nTYmniJY77w8Smd7PLpY.jar' is inaccessible.
Oct 22, 2021 3:57:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-22T15:57:47.668Z: Staged package aws-java-sdk-kms-1.11.974-HWH_mHclw3Wv4_GJ56x62xFDHSPSkZWjeKAlx40XrTk.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/aws-java-sdk-kms-1.11.974-HWH_mHclw3Wv4_GJ56x62xFDHSPSkZWjeKAlx40XrTk.jar' is inaccessible.
Oct 22, 2021 3:57:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-22T15:57:47.713Z: Staged package aws-java-sdk-s3-1.11.974-Qgd0wgaOuI_uyxxQI4Ulljk9bZEq-3JZArQF0zU5yzA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/aws-java-sdk-s3-1.11.974-Qgd0wgaOuI_uyxxQI4Ulljk9bZEq-3JZArQF0zU5yzA.jar' is inaccessible.
Oct 22, 2021 3:57:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-22T15:57:48.366Z: Staged package commons-io-2.6--HfTBGYKwqFC84ZbrfyXHex-1zx0fH-NXS9ROcpzZRM.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/commons-io-2.6--HfTBGYKwqFC84ZbrfyXHex-1zx0fH-NXS9ROcpzZRM.jar' is inaccessible.
Oct 22, 2021 3:57:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-22T15:57:48.417Z: Staged package commons-lang-2.6-UPEbCfh3wpTVbyRGP0fSj5Kc9QRPZIZhwPDPuumi9Jw.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/commons-lang-2.6-UPEbCfh3wpTVbyRGP0fSj5Kc9QRPZIZhwPDPuumi9Jw.jar' is inaccessible.
Oct 22, 2021 3:57:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-22T15:57:48.571Z: Staged package error_prone_annotations-2.3.4-uvfW6pfOYGxT4RtoVLpfLOfvXCTd3wr6GNEmC9JbACw.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/error_prone_annotations-2.3.4-uvfW6pfOYGxT4RtoVLpfLOfvXCTd3wr6GNEmC9JbACw.jar' is inaccessible.
Oct 22, 2021 3:57:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-22T15:57:49.949Z: Staged package ion-java-1.0.2-DRJ7IFofzgq8KjdXoEF0hlG8ZsFc9MBZusWDOyfUcaU.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/ion-java-1.0.2-DRJ7IFofzgq8KjdXoEF0hlG8ZsFc9MBZusWDOyfUcaU.jar' is inaccessible.
Oct 22, 2021 3:57:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-22T15:57:50.097Z: Staged package jackson-dataformat-cbor-2.12.4-xpFy2QEl4dUh55AUzgOueyVcIoNGuAdPK6G3l4P-yCc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/jackson-dataformat-cbor-2.12.4-xpFy2QEl4dUh55AUzgOueyVcIoNGuAdPK6G3l4P-yCc.jar' is inaccessible.
Oct 22, 2021 3:57:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-22T15:57:50.281Z: Staged package jmespath-java-1.11.974-QaUt5Hnw6wd7ULTQ5Oj4j_zpRr-rnMCdfGwiwkSo8-0.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/jmespath-java-1.11.974-QaUt5Hnw6wd7ULTQ5Oj4j_zpRr-rnMCdfGwiwkSo8-0.jar' is inaccessible.
Oct 22, 2021 3:57:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-10-22T15:57:51.242Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Oct 22, 2021 4:00:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-22T16:00:30.705Z: Cancel request is committed for workflow job: 2021-10-22_05_45_27-8754089351573128322.
Oct 22, 2021 4:00:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-22T16:00:30.739Z: Cleaning up.
Oct 22, 2021 4:00:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-22T16:00:30.821Z: Stopping **** pool...
Oct 22, 2021 4:00:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-22T16:00:30.901Z: Stopping **** pool...
Oct 22, 2021 4:02:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-22T16:02:50.988Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Oct 22, 2021 4:02:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-22T16:02:51.059Z: Worker pool stopped.
Oct 22, 2021 4:02:56 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-10-22_05_45_27-8754089351573128322 finished with status CANCELLED.
Load test results for test (ID): 461d56ac-b960-479f-83e1-9186f0683a7e and timestamp: 2021-10-22T12:45:22.419000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11519.193
dataflow_v2_java11_total_bytes_count             2.22514959E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211022124333
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:fde408f783525f15f7516cdd3955d0a75837d3b11f9031be4247590ff832a8a7
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211022124333]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:fde408f783525f15f7516cdd3955d0a75837d3b11f9031be4247590ff832a8a7]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211022124333] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:fde408f783525f15f7516cdd3955d0a75837d3b11f9031be4247590ff832a8a7])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:fde408f783525f15f7516cdd3955d0a75837d3b11f9031be4247590ff832a8a7
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:fde408f783525f15f7516cdd3955d0a75837d3b11f9031be4247590ff832a8a7
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:fde408f783525f15f7516cdd3955d0a75837d3b11f9031be4247590ff832a8a7].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:600e0b7a283b0c8b376be8fe9960594503735a5f3a285f91083d3aec31b13b63
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:600e0b7a283b0c8b376be8fe9960594503735a5f3a285f91083d3aec31b13b63
ERROR: (gcloud.container.images.delete) Not found: response: {'status': '404', 'content-length': '168', 'x-xss-protection': '0', 'transfer-encoding': 'chunked', 'server': 'Docker Registry', '-content-encoding': 'gzip', 'docker-distribution-api-version': 'registry/2.0', 'cache-control': 'private', 'date': 'Fri, 22 Oct 2021 16:03:00 GMT', 'x-frame-options': 'SAMEORIGIN', 'content-type': 'application/json'}
Failed to compute blob liveness for manifest: 'sha256:600e0b7a283b0c8b376be8fe9960594503735a5f3a285f91083d3aec31b13b63': None

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages FAILED

FAILURE: Build completed with 2 failures.

1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

2: Task failed with an exception.
-----------
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src/runners/google-cloud-dataflow-java/build.gradle'> line: 282

* What went wrong:
Execution failed for task ':runners:google-cloud-dataflow-java:cleanUpDockerImages'.
> Process 'command './scripts/cleanup_untagged_gcr_images.sh'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 40s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/ao3ubinetr776

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #126

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/126/display/redirect?page=changes>

Changes:

[noreply] Add -XX:+AlwaysActAsServerClassMachine to Java SDK container

[moritz] adhoc: Minor update to flink runner docs

[noreply] [BEAM-11087] Add default WindowMappingFn from Main to Side Input

[noreply] [BEAM-13082] Re-use dataWriter buffer. (#15762)


------------------------------------------
[...truncated 48.66 KB...]
27d7589082a3: Preparing
df718bf95b1f: Preparing
fdbee3fc0ed7: Preparing
13f89c075c80: Preparing
c2e45a9a6908: Preparing
211736d7fc9d: Preparing
62a5b8741e83: Preparing
36e0782f1159: Preparing
ba6e5ff31f23: Preparing
9f9f651e9303: Preparing
0b3c02b5d746: Preparing
62a747bf1719: Preparing
fdbee3fc0ed7: Waiting
13f89c075c80: Waiting
c2e45a9a6908: Waiting
9f9f651e9303: Waiting
0b3c02b5d746: Waiting
211736d7fc9d: Waiting
62a5b8741e83: Waiting
b46796526af9: Waiting
36e0782f1159: Waiting
ba6e5ff31f23: Waiting
62a747bf1719: Waiting
df718bf95b1f: Waiting
7f341d529cec: Waiting
27d7589082a3: Waiting
e75773a91867: Pushed
c2cd88aa4da1: Pushed
d81f99b93918: Pushed
bf560bac8f42: Pushed
7f341d529cec: Pushed
27d7589082a3: Pushed
9efe5625e8ba: Pushed
df718bf95b1f: Pushed
211736d7fc9d: Layer already exists
62a5b8741e83: Layer already exists
36e0782f1159: Layer already exists
ba6e5ff31f23: Layer already exists
9f9f651e9303: Layer already exists
b46796526af9: Pushed
0b3c02b5d746: Layer already exists
62a747bf1719: Layer already exists
13f89c075c80: Pushed
c2e45a9a6908: Pushed
fdbee3fc0ed7: Pushed
20211021124334: digest: sha256:050ac8bca4e318f4189ef746a2f9d28ab1c6c5c69b77e4460d07ae18147826dc size: 4311

> Task :sdks:java:testing:load-tests:run
Oct 21, 2021 12:45:22 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Oct 21, 2021 12:45:22 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 195 files. Enable logging at DEBUG level to see which files will be staged.
Oct 21, 2021 12:45:23 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Oct 21, 2021 12:45:23 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Oct 21, 2021 12:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 195 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Oct 21, 2021 12:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 195 files cached, 0 files newly uploaded in 0 seconds
Oct 21, 2021 12:45:25 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Oct 21, 2021 12:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <108949 bytes, hash 6e8f69c3b865aabbc467c2863b517519c648dd1e38a1a9650bdb957396b1aecf> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-bo9pw7hlqrvEZ8KGO1F1GcZI3R44oallC9uVc5axrs8.pb
Oct 21, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Oct 21, 2021 12:45:27 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7fb66650, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1a96d94c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2a869a16, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@ae202c6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@46aa712c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6ada9c0c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7412ed6b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6e7c351d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b4a0aef, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@45cec376, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26d41711, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@298f0a0b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2b960a7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@31dfc6f5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@37b52340, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@663bb8ef, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2f4e40d7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@60e9c3a5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7e5843db, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@459f703f]
Oct 21, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Oct 21, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Oct 21, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Oct 21, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Oct 21, 2021 12:45:27 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@367f0121, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7da39774, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@441b8382, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1df1ced0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5349b246, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@32b0876c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2aaf152b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@39bbd9e0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@27fe9713, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@11c3ff67, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4397a639, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@24841372, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77114efe, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@79a7bfbc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77f68df, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3e4e4c1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7e7f3cfd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ae126d1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@46a488c2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6242ae3b]
Oct 21, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Oct 21, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Oct 21, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Oct 21, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Oct 21, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Oct 21, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Oct 21, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Oct 21, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Oct 21, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Oct 21, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Oct 21, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Oct 21, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.35.0-SNAPSHOT
Oct 21, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-10-21_05_45_27-12856435589279297628?project=apache-beam-testing
Oct 21, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-10-21_05_45_27-12856435589279297628
Oct 21, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-10-21_05_45_27-12856435589279297628
Oct 21, 2021 12:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-10-21T12:45:33.549Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-10-sgei. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Oct 21, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-21T12:45:42.447Z: Worker configuration: e2-standard-2 in us-central1-a.
Oct 21, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-21T12:45:43.223Z: Expanding SplittableParDo operations into optimizable parts.
Oct 21, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-21T12:45:43.247Z: Expanding CollectionToSingleton operations into optimizable parts.
Oct 21, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-21T12:45:43.318Z: Expanding CoGroupByKey operations into optimizable parts.
Oct 21, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-21T12:45:43.392Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Oct 21, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-21T12:45:43.459Z: Expanding GroupByKey operations into streaming Read/Write steps
Oct 21, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-21T12:45:43.519Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Oct 21, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-21T12:45:43.623Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Oct 21, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-21T12:45:43.660Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Oct 21, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-21T12:45:43.698Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Oct 21, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-21T12:45:43.726Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Oct 21, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-21T12:45:43.752Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Oct 21, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-21T12:45:43.795Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Oct 21, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-21T12:45:43.826Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Oct 21, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-21T12:45:43.860Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Oct 21, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-21T12:45:43.889Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Oct 21, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-21T12:45:43.927Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Oct 21, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-21T12:45:43.961Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Oct 21, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-21T12:45:43.995Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Oct 21, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-21T12:45:44.025Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Oct 21, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-21T12:45:44.049Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Oct 21, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-21T12:45:44.083Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Oct 21, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-21T12:45:44.128Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Oct 21, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-21T12:45:44.150Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Oct 21, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-21T12:45:44.177Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Oct 21, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-21T12:45:44.205Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Oct 21, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-21T12:45:44.229Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Oct 21, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-21T12:45:44.253Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Oct 21, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-21T12:45:44.284Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Oct 21, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-21T12:45:44.317Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Oct 21, 2021 12:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-21T12:45:44.700Z: Starting 5 ****s in us-central1-a...
Oct 21, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-21T12:46:01.901Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Oct 21, 2021 12:46:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-21T12:46:24.356Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Oct 21, 2021 12:47:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-21T12:47:28.300Z: Workers have started successfully.
Oct 21, 2021 12:47:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-21T12:47:28.333Z: Workers have started successfully.
Oct 21, 2021 2:11:48 PM org.apache.beam.sdk.metrics.MetricsEnvironment getCurrentContainer
WARNING: Reporting metrics are not supported in the current execution environment.
Oct 21, 2021 3:53:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-21T15:53:43.830Z: generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:1089
Oct 21, 2021 4:01:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-21T16:01:00.167Z: Cancel request is committed for workflow job: 2021-10-21_05_45_27-12856435589279297628.
Oct 21, 2021 4:01:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-21T16:01:00.210Z: Cleaning up.
Oct 21, 2021 4:01:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-21T16:01:00.299Z: Stopping **** pool...
Oct 21, 2021 4:01:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-21T16:01:00.353Z: Stopping **** pool...
Oct 21, 2021 4:03:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-21T16:03:28.019Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Oct 21, 2021 4:03:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-21T16:03:28.111Z: Worker pool stopped.
Oct 21, 2021 4:03:35 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-10-21_05_45_27-12856435589279297628 finished with status CANCELLED.
Load test results for test (ID): 40bd8b67-6ded-44c3-948c-8e43c1b7f652 and timestamp: 2021-10-21T12:45:22.796000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11555.628
dataflow_v2_java11_total_bytes_count             2.51379893E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211021124334
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:050ac8bca4e318f4189ef746a2f9d28ab1c6c5c69b77e4460d07ae18147826dc
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211021124334]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:050ac8bca4e318f4189ef746a2f9d28ab1c6c5c69b77e4460d07ae18147826dc]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211021124334] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:050ac8bca4e318f4189ef746a2f9d28ab1c6c5c69b77e4460d07ae18147826dc])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:050ac8bca4e318f4189ef746a2f9d28ab1c6c5c69b77e4460d07ae18147826dc
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:050ac8bca4e318f4189ef746a2f9d28ab1c6c5c69b77e4460d07ae18147826dc
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:050ac8bca4e318f4189ef746a2f9d28ab1c6c5c69b77e4460d07ae18147826dc].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 20m 19s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/cfsjnax2g2pws

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #125

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/125/display/redirect?page=changes>

Changes:

[samuelw] [BEAM-13042] Prevent unexpected blocking in

[ilya.kozyrev] Implement initial server with grpcweb wrapper

[aydar.zaynutdinov] [BEAM_13077][Playground]

[egalpin] [BEAM-10990] Adds response filtering for ElasticsearchIO

[egalpin] [BEAM-5172] Tries to reduce ES uTest flakiness

[noreply] [BEAM-13068] Add a SQL API in Beam Go SDK (#15746)

[Kyle Weaver] [BEAM-13055] Use unshallow clone to create PR.

[noreply] [BEAM-13079] Updates cross-language transform URNs to use the new


------------------------------------------
[...truncated 49.57 KB...]
a1d4aca25996: Preparing
d3f5757701a4: Preparing
6fdee1d11a8b: Preparing
c59a11cff868: Preparing
fa54838fc732: Preparing
aa6ce0101937: Preparing
1ea7135b02a7: Preparing
3c762d54a262: Preparing
211736d7fc9d: Preparing
62a5b8741e83: Preparing
36e0782f1159: Preparing
ba6e5ff31f23: Preparing
9f9f651e9303: Preparing
0b3c02b5d746: Preparing
62a747bf1719: Preparing
c59a11cff868: Waiting
3c762d54a262: Waiting
1ea7135b02a7: Waiting
9f9f651e9303: Waiting
fa54838fc732: Waiting
62a5b8741e83: Waiting
0b3c02b5d746: Waiting
aa6ce0101937: Waiting
36e0782f1159: Waiting
62a747bf1719: Waiting
ba6e5ff31f23: Waiting
d3f5757701a4: Waiting
6fdee1d11a8b: Waiting
8b0817e7e651: Pushed
ce09958f4a5a: Pushed
a1d4aca25996: Pushed
d3f5757701a4: Pushed
505ede38669d: Pushed
c59a11cff868: Pushed
e054861bdeb5: Pushed
fa54838fc732: Pushed
1ea7135b02a7: Pushed
6fdee1d11a8b: Pushed
211736d7fc9d: Layer already exists
62a5b8741e83: Layer already exists
36e0782f1159: Layer already exists
3c762d54a262: Pushed
ba6e5ff31f23: Layer already exists
9f9f651e9303: Layer already exists
62a747bf1719: Layer already exists
0b3c02b5d746: Layer already exists
aa6ce0101937: Pushed
20211020124336: digest: sha256:c99d7dfb89e708a2d4953010321e187e0e4e0a1065f5b207fcb4e0b004a8d204 size: 4311

> Task :sdks:java:testing:load-tests:run
Oct 20, 2021 12:45:24 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Oct 20, 2021 12:45:25 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 195 files. Enable logging at DEBUG level to see which files will be staged.
Oct 20, 2021 12:45:26 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Oct 20, 2021 12:45:26 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Oct 20, 2021 12:45:28 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 195 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Oct 20, 2021 12:45:28 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 195 files cached, 0 files newly uploaded in 0 seconds
Oct 20, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Oct 20, 2021 12:45:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <108949 bytes, hash 5bb77f16ed7b654ea65c4a37191e7b3d0bae694f379fa1116a1cf2c6b79634b9> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-W7d_Fu17ZU6mXEo3GR57PQuuaU83n6ERahzyxreWNLk.pb
Oct 20, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Oct 20, 2021 12:45:30 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7fb66650, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1a96d94c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2a869a16, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@ae202c6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@46aa712c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6ada9c0c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7412ed6b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6e7c351d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b4a0aef, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@45cec376, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26d41711, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@298f0a0b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2b960a7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@31dfc6f5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@37b52340, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@663bb8ef, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2f4e40d7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@60e9c3a5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7e5843db, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@459f703f]
Oct 20, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Oct 20, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Oct 20, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Oct 20, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Oct 20, 2021 12:45:30 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@367f0121, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7da39774, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@441b8382, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1df1ced0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5349b246, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@32b0876c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2aaf152b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@39bbd9e0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@27fe9713, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@11c3ff67, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4397a639, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@24841372, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77114efe, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@79a7bfbc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77f68df, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3e4e4c1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7e7f3cfd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ae126d1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@46a488c2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6242ae3b]
Oct 20, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Oct 20, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Oct 20, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Oct 20, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Oct 20, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Oct 20, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Oct 20, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Oct 20, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Oct 20, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Oct 20, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Oct 20, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Oct 20, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.35.0-SNAPSHOT
Oct 20, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-10-20_05_45_30-8985529173814672987?project=apache-beam-testing
Oct 20, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-10-20_05_45_30-8985529173814672987
Oct 20, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-10-20_05_45_30-8985529173814672987
Oct 20, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-10-20T12:45:38.043Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-10-k46o. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Oct 20, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-20T12:45:43.410Z: Worker configuration: e2-standard-2 in us-central1-a.
Oct 20, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-20T12:45:44.107Z: Expanding SplittableParDo operations into optimizable parts.
Oct 20, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-20T12:45:44.128Z: Expanding CollectionToSingleton operations into optimizable parts.
Oct 20, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-20T12:45:44.186Z: Expanding CoGroupByKey operations into optimizable parts.
Oct 20, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-20T12:45:44.256Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Oct 20, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-20T12:45:44.277Z: Expanding GroupByKey operations into streaming Read/Write steps
Oct 20, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-20T12:45:44.338Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Oct 20, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-20T12:45:44.486Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Oct 20, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-20T12:45:44.513Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Oct 20, 2021 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-20T12:45:44.537Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Oct 20, 2021 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-20T12:45:44.561Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Oct 20, 2021 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-20T12:45:44.596Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Oct 20, 2021 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-20T12:45:44.694Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Oct 20, 2021 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-20T12:45:44.739Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Oct 20, 2021 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-20T12:45:44.798Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Oct 20, 2021 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-20T12:45:44.844Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Oct 20, 2021 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-20T12:45:44.891Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Oct 20, 2021 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-20T12:45:44.932Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Oct 20, 2021 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-20T12:45:44.964Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Oct 20, 2021 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-20T12:45:45.001Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Oct 20, 2021 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-20T12:45:45.025Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Oct 20, 2021 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-20T12:45:45.068Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Oct 20, 2021 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-20T12:45:45.105Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Oct 20, 2021 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-20T12:45:45.128Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Oct 20, 2021 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-20T12:45:45.153Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Oct 20, 2021 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-20T12:45:45.179Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Oct 20, 2021 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-20T12:45:45.205Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Oct 20, 2021 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-20T12:45:45.239Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Oct 20, 2021 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-20T12:45:45.262Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Oct 20, 2021 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-20T12:45:45.286Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Oct 20, 2021 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-20T12:45:45.612Z: Starting 5 ****s in us-central1-a...
Oct 20, 2021 12:46:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-20T12:46:07.329Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Oct 20, 2021 12:46:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-20T12:46:30.487Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Oct 20, 2021 12:47:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-20T12:47:23.889Z: Workers have started successfully.
Oct 20, 2021 12:47:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-20T12:47:23.927Z: Workers have started successfully.
Oct 20, 2021 4:00:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-20T16:00:27.510Z: Cancel request is committed for workflow job: 2021-10-20_05_45_30-8985529173814672987.
Oct 20, 2021 4:00:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-20T16:00:27.969Z: Cleaning up.
Oct 20, 2021 4:00:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-20T16:00:28.043Z: Stopping **** pool...
Oct 20, 2021 4:00:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-20T16:00:28.112Z: Stopping **** pool...
Oct 20, 2021 4:00:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-20T16:00:36.518Z: generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:1089
Oct 20, 2021 4:02:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-20T16:02:51.502Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Oct 20, 2021 4:02:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-20T16:02:51.536Z: Worker pool stopped.
Oct 20, 2021 4:02:58 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-10-20_05_45_30-8985529173814672987 finished with status CANCELLED.
Load test results for test (ID): 22762324-71b2-4662-9b2b-f910527e588a and timestamp: 2021-10-20T12:45:25.606000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11561.399
dataflow_v2_java11_total_bytes_count             2.57977194E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211020124336
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:c99d7dfb89e708a2d4953010321e187e0e4e0a1065f5b207fcb4e0b004a8d204
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211020124336]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:c99d7dfb89e708a2d4953010321e187e0e4e0a1065f5b207fcb4e0b004a8d204]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211020124336] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:c99d7dfb89e708a2d4953010321e187e0e4e0a1065f5b207fcb4e0b004a8d204])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:c99d7dfb89e708a2d4953010321e187e0e4e0a1065f5b207fcb4e0b004a8d204
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:c99d7dfb89e708a2d4953010321e187e0e4e0a1065f5b207fcb4e0b004a8d204
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:c99d7dfb89e708a2d4953010321e187e0e4e0a1065f5b207fcb4e0b004a8d204].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 42s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/i4w5smwlnqsey

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #124

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/124/display/redirect?page=changes>

Changes:

[zhoufek] [BEAM-9487] Multiple Trigger.may_lose_data fixes

[zhoufek] [BEAM-9487] Remove CONDITION_NOT_GUARANTEED as potential data loss

[zhoufek] [BEAM-9487] Do AfterAny, AfterAll, and AfterEach checks properly (i.e.

[zhoufek] [BEAM-9487] Remove unused import

[zhoufek] [BEAM-9487] Reintroduce flag but do not use it

[zhoufek] [BEAM-9487] Add test that shows AfterCount finishing

[zhoufek] [BEAM-9487] Make _ParallelTriggerFn.may_finish clearer

[Robert Bradshaw] Revert "Merge pull request #15441 from [BEAM-8823] Make FnApiRunner work

[Robert Bradshaw] [BEAM-13040] Add some test cases enforcing side input waiting.

[Robert Bradshaw] lint

[brachipa] [BEAM-12393] sql support for Zeta Sql

[aydar.zaynutdinov] [BEAM-12988] [Playground]

[brachipa] [BEAM-12393] package private

[brachipa] [BEAM-12393] returning more generic interface

[noreply] [BEAM-13066] Produce abstract iterables from IterableCoder. (#15662)

[noreply] [BEAM-11936] Fix some errorprone warnings (#15648)

[noreply] [BEAM-13068] Add xlangx.DecodeStructPayload (#15741)

[Luke Cwik] [BEAM-13015] Implement a simplified cancellable blocking queue with


------------------------------------------
[...truncated 48.75 KB...]
1d979abf3de2: Preparing
289798ca79de: Preparing
f750d5c064cb: Preparing
d19373d00b9e: Preparing
b93a3cfa0410: Preparing
c21938f48ffa: Preparing
211736d7fc9d: Preparing
62a5b8741e83: Preparing
36e0782f1159: Preparing
ba6e5ff31f23: Preparing
9f9f651e9303: Preparing
0b3c02b5d746: Preparing
62a747bf1719: Preparing
289798ca79de: Waiting
9f9f651e9303: Waiting
0b3c02b5d746: Waiting
f750d5c064cb: Waiting
62a747bf1719: Waiting
ba6e5ff31f23: Waiting
d19373d00b9e: Waiting
c21938f48ffa: Waiting
b93a3cfa0410: Waiting
36e0782f1159: Waiting
62a5b8741e83: Waiting
237c7227b067: Waiting
1d979abf3de2: Waiting
211736d7fc9d: Waiting
e54700390d44: Pushed
15dda34baecf: Pushed
8c1506596a62: Pushed
10f2325e684e: Pushed
237c7227b067: Pushed
b97be556d78d: Pushed
289798ca79de: Pushed
f750d5c064cb: Pushed
211736d7fc9d: Layer already exists
62a5b8741e83: Layer already exists
36e0782f1159: Layer already exists
ba6e5ff31f23: Layer already exists
9f9f651e9303: Layer already exists
b93a3cfa0410: Pushed
0b3c02b5d746: Layer already exists
c21938f48ffa: Pushed
62a747bf1719: Layer already exists
1d979abf3de2: Pushed
d19373d00b9e: Pushed
20211019124332: digest: sha256:d91e16e0bd057c6131ee1f035da528600156546eee09db267f179a80e445ef1b size: 4311

> Task :sdks:java:testing:load-tests:run
Oct 19, 2021 12:45:26 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Oct 19, 2021 12:45:26 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 195 files. Enable logging at DEBUG level to see which files will be staged.
Oct 19, 2021 12:45:27 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Oct 19, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Oct 19, 2021 12:45:29 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 195 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Oct 19, 2021 12:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 195 files cached, 0 files newly uploaded in 0 seconds
Oct 19, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Oct 19, 2021 12:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <108949 bytes, hash 555cfa9d337e4b57346b563d5ebcc30af3d429d6d26afa106e3c27f11463c136> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-VVz6nTN-S1c0a1Y9XrzDCvPUKdbSavoQbjwn8RRjwTY.pb
Oct 19, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Oct 19, 2021 12:45:31 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2a869a16, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@ae202c6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@46aa712c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6ada9c0c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7412ed6b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6e7c351d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b4a0aef, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@45cec376, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26d41711, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@298f0a0b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2b960a7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@31dfc6f5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@37b52340, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@663bb8ef, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2f4e40d7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@60e9c3a5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7e5843db, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@459f703f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@188ac8a3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3650d4fc]
Oct 19, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Oct 19, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Oct 19, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Oct 19, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Oct 19, 2021 12:45:31 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@441b8382, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1df1ced0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5349b246, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@32b0876c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2aaf152b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@39bbd9e0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@27fe9713, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@11c3ff67, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4397a639, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@24841372, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77114efe, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@79a7bfbc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77f68df, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3e4e4c1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7e7f3cfd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ae126d1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@46a488c2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6242ae3b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@65ddee5a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@56399b9e]
Oct 19, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Oct 19, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Oct 19, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Oct 19, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Oct 19, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Oct 19, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Oct 19, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Oct 19, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Oct 19, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Oct 19, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Oct 19, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Oct 19, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.35.0-SNAPSHOT
Oct 19, 2021 12:45:32 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-10-19_05_45_31-9238490290909742448?project=apache-beam-testing
Oct 19, 2021 12:45:32 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-10-19_05_45_31-9238490290909742448
Oct 19, 2021 12:45:32 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-10-19_05_45_31-9238490290909742448
Oct 19, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-10-19T12:45:43.025Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-10-watl. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Oct 19, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-19T12:45:47.853Z: Worker configuration: e2-standard-2 in us-central1-a.
Oct 19, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-19T12:45:48.708Z: Expanding SplittableParDo operations into optimizable parts.
Oct 19, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-19T12:45:48.742Z: Expanding CollectionToSingleton operations into optimizable parts.
Oct 19, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-19T12:45:48.798Z: Expanding CoGroupByKey operations into optimizable parts.
Oct 19, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-19T12:45:48.869Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Oct 19, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-19T12:45:48.898Z: Expanding GroupByKey operations into streaming Read/Write steps
Oct 19, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-19T12:45:48.952Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Oct 19, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-19T12:45:49.055Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Oct 19, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-19T12:45:49.095Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Oct 19, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-19T12:45:49.160Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Oct 19, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-19T12:45:49.194Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Oct 19, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-19T12:45:49.226Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Oct 19, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-19T12:45:49.258Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Oct 19, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-19T12:45:49.294Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Oct 19, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-19T12:45:49.327Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Oct 19, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-19T12:45:49.359Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Oct 19, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-19T12:45:49.393Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Oct 19, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-19T12:45:49.425Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Oct 19, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-19T12:45:49.458Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Oct 19, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-19T12:45:49.495Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Oct 19, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-19T12:45:49.516Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Oct 19, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-19T12:45:49.549Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Oct 19, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-19T12:45:49.572Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Oct 19, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-19T12:45:49.604Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Oct 19, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-19T12:45:49.636Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Oct 19, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-19T12:45:49.671Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Oct 19, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-19T12:45:49.705Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Oct 19, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-19T12:45:49.744Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Oct 19, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-19T12:45:49.776Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Oct 19, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-19T12:45:49.809Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Oct 19, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-19T12:45:50.218Z: Starting 5 ****s in us-central1-a...
Oct 19, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-19T12:46:01.361Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Oct 19, 2021 12:46:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-19T12:46:26.099Z: Autoscaling: Raised the number of ****s to 2 so that the pipeline can catch up with its backlog and keep up with its input rate.
Oct 19, 2021 12:46:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-19T12:46:26.134Z: Resized **** pool to 2, though goal was 5.  This could be a quota issue.
Oct 19, 2021 12:46:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-19T12:46:36.426Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Oct 19, 2021 12:47:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-19T12:47:28.817Z: Workers have started successfully.
Oct 19, 2021 12:47:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-19T12:47:28.846Z: Workers have started successfully.
Oct 19, 2021 2:59:38 PM org.apache.beam.sdk.metrics.MetricsEnvironment getCurrentContainer
WARNING: Reporting metrics are not supported in the current execution environment.
Oct 19, 2021 4:00:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-19T16:00:29.077Z: Cancel request is committed for workflow job: 2021-10-19_05_45_31-9238490290909742448.
Oct 19, 2021 4:00:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-19T16:00:29.153Z: Cleaning up.
Oct 19, 2021 4:00:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-19T16:00:29.252Z: Stopping **** pool...
Oct 19, 2021 4:00:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-19T16:00:29.309Z: Stopping **** pool...
Oct 19, 2021 4:02:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-19T16:02:48.758Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Oct 19, 2021 4:02:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-19T16:02:48.794Z: Worker pool stopped.
Oct 19, 2021 4:02:54 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-10-19_05_45_31-9238490290909742448 finished with status CANCELLED.
Load test results for test (ID): d5a994cd-f589-493b-8f96-f8f5246c719b and timestamp: 2021-10-19T12:45:27.262000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                  11532.16
dataflow_v2_java11_total_bytes_count             3.33109931E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211019124332
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:d91e16e0bd057c6131ee1f035da528600156546eee09db267f179a80e445ef1b
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211019124332]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:d91e16e0bd057c6131ee1f035da528600156546eee09db267f179a80e445ef1b]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211019124332] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:d91e16e0bd057c6131ee1f035da528600156546eee09db267f179a80e445ef1b])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:d91e16e0bd057c6131ee1f035da528600156546eee09db267f179a80e445ef1b
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:d91e16e0bd057c6131ee1f035da528600156546eee09db267f179a80e445ef1b
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:d91e16e0bd057c6131ee1f035da528600156546eee09db267f179a80e445ef1b].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 40s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/kmcjt2z7qm2fm

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #123

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/123/display/redirect>

Changes:


------------------------------------------
[...truncated 56.61 KB...]
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Oct 18, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Oct 18, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Oct 18, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Oct 18, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Oct 18, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Oct 18, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Oct 18, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Oct 18, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.35.0-SNAPSHOT
Oct 18, 2021 12:45:32 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-10-18_05_45_31-14203049816433255547?project=apache-beam-testing
Oct 18, 2021 12:45:32 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-10-18_05_45_31-14203049816433255547
Oct 18, 2021 12:45:32 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-10-18_05_45_31-14203049816433255547
Oct 18, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-10-18T12:45:38.957Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-10-sqas. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Oct 18, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-18T12:45:43.143Z: Worker configuration: e2-standard-2 in us-central1-a.
Oct 18, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-18T12:45:43.857Z: Expanding SplittableParDo operations into optimizable parts.
Oct 18, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-18T12:45:43.882Z: Expanding CollectionToSingleton operations into optimizable parts.
Oct 18, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-18T12:45:43.949Z: Expanding CoGroupByKey operations into optimizable parts.
Oct 18, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-18T12:45:44.014Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Oct 18, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-18T12:45:44.039Z: Expanding GroupByKey operations into streaming Read/Write steps
Oct 18, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-18T12:45:44.095Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Oct 18, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-18T12:45:44.187Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Oct 18, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-18T12:45:44.208Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Oct 18, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-18T12:45:44.235Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Oct 18, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-18T12:45:44.260Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Oct 18, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-18T12:45:44.292Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Oct 18, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-18T12:45:44.324Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Oct 18, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-18T12:45:44.363Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Oct 18, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-18T12:45:44.390Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Oct 18, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-18T12:45:44.416Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Oct 18, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-18T12:45:44.431Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Oct 18, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-18T12:45:44.501Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Oct 18, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-18T12:45:44.530Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Oct 18, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-18T12:45:44.565Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Oct 18, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-18T12:45:44.597Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Oct 18, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-18T12:45:44.624Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Oct 18, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-18T12:45:44.656Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Oct 18, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-18T12:45:44.689Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Oct 18, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-18T12:45:44.714Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Oct 18, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-18T12:45:44.741Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Oct 18, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-18T12:45:44.785Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Oct 18, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-18T12:45:44.814Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Oct 18, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-18T12:45:44.851Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Oct 18, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-18T12:45:44.881Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Oct 18, 2021 12:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-18T12:45:45.244Z: Starting 5 ****s in us-central1-a...
Oct 18, 2021 12:46:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-18T12:46:14.278Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Oct 18, 2021 12:46:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-18T12:46:36.203Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Oct 18, 2021 12:47:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-18T12:47:37.482Z: Workers have started successfully.
Oct 18, 2021 12:47:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-18T12:47:37.509Z: Workers have started successfully.
Oct 18, 2021 3:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-18T15:45:46.425Z: Staged package checker-qual-3.10.0-pNyILKaqxJbTM4Hl5esGBMRUg7AEvD6sk2jxuxfLJRI.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/checker-qual-3.10.0-pNyILKaqxJbTM4Hl5esGBMRUg7AEvD6sk2jxuxfLJRI.jar' is inaccessible.
Oct 18, 2021 3:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-18T15:45:47.367Z: Staged package google-http-client-1.39.2-arSWeGCSfwUd1_K1deS3qO8182fcLjQIBzqk3uMo4ic.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-http-client-1.39.2-arSWeGCSfwUd1_K1deS3qO8182fcLjQIBzqk3uMo4ic.jar' is inaccessible.
Oct 18, 2021 3:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-18T15:45:47.409Z: Staged package google-http-client-apache-v2-1.39.2-UtOSCfZav29XfhyrRmrbmfEtHNK6TpjYblAfddf4UM8.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-http-client-apache-v2-1.39.2-UtOSCfZav29XfhyrRmrbmfEtHNK6TpjYblAfddf4UM8.jar' is inaccessible.
Oct 18, 2021 3:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-18T15:45:47.454Z: Staged package google-http-client-appengine-1.39.2-9jINwNsDWv9uqy4odIVg2nGKtGX7w0gbZT6j8nqlysk.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-http-client-appengine-1.39.2-9jINwNsDWv9uqy4odIVg2nGKtGX7w0gbZT6j8nqlysk.jar' is inaccessible.
Oct 18, 2021 3:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-18T15:45:47.505Z: Staged package google-http-client-gson-1.39.2-z4GUT9Qf4NqoabkTvQqq7w5Nj-Z3ShlVQPRU7NPPULY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-http-client-gson-1.39.2-z4GUT9Qf4NqoabkTvQqq7w5Nj-Z3ShlVQPRU7NPPULY.jar' is inaccessible.
Oct 18, 2021 3:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-18T15:45:47.580Z: Staged package google-http-client-jackson2-1.39.2-dKSTp7ol4d-z06Td9xMwbVo3htsAo0idWQYdyM_zIbE.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-http-client-jackson2-1.39.2-dKSTp7ol4d-z06Td9xMwbVo3htsAo0idWQYdyM_zIbE.jar' is inaccessible.
Oct 18, 2021 3:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-18T15:45:47.644Z: Staged package google-http-client-protobuf-1.39.2-48CRMH_IJ9gY8fZ3-5ANC0qF7rqbijaXD90xuQTMr5s.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-http-client-protobuf-1.39.2-48CRMH_IJ9gY8fZ3-5ANC0qF7rqbijaXD90xuQTMr5s.jar' is inaccessible.
Oct 18, 2021 3:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-18T15:45:48.974Z: Staged package opencensus-contrib-grpc-util-0.28.0-uRaDRuavZZMwChvCfvdCVKofJAGYhZON2PuFK4d9VfA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/opencensus-contrib-grpc-util-0.28.0-uRaDRuavZZMwChvCfvdCVKofJAGYhZON2PuFK4d9VfA.jar' is inaccessible.
Oct 18, 2021 3:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-18T15:45:49.550Z: Staged package zstd-jni-1.4.5-2-lzQxwUtNCahuI7cYQRb8rC2FUB60p0MPfRhczhr0YFA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/zstd-jni-1.4.5-2-lzQxwUtNCahuI7cYQRb8rC2FUB60p0MPfRhczhr0YFA.jar' is inaccessible.
Oct 18, 2021 3:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-10-18T15:45:49.590Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Oct 18, 2021 3:48:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-10-18T15:48:48.937Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Oct 18, 2021 3:51:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-18T15:51:46.458Z: Staged package checker-qual-3.10.0-pNyILKaqxJbTM4Hl5esGBMRUg7AEvD6sk2jxuxfLJRI.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/checker-qual-3.10.0-pNyILKaqxJbTM4Hl5esGBMRUg7AEvD6sk2jxuxfLJRI.jar' is inaccessible.
Oct 18, 2021 3:51:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-18T15:51:47.619Z: Staged package google-http-client-1.39.2-arSWeGCSfwUd1_K1deS3qO8182fcLjQIBzqk3uMo4ic.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-http-client-1.39.2-arSWeGCSfwUd1_K1deS3qO8182fcLjQIBzqk3uMo4ic.jar' is inaccessible.
Oct 18, 2021 3:51:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-18T15:51:47.659Z: Staged package google-http-client-apache-v2-1.39.2-UtOSCfZav29XfhyrRmrbmfEtHNK6TpjYblAfddf4UM8.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-http-client-apache-v2-1.39.2-UtOSCfZav29XfhyrRmrbmfEtHNK6TpjYblAfddf4UM8.jar' is inaccessible.
Oct 18, 2021 3:51:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-18T15:51:47.694Z: Staged package google-http-client-appengine-1.39.2-9jINwNsDWv9uqy4odIVg2nGKtGX7w0gbZT6j8nqlysk.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-http-client-appengine-1.39.2-9jINwNsDWv9uqy4odIVg2nGKtGX7w0gbZT6j8nqlysk.jar' is inaccessible.
Oct 18, 2021 3:51:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-18T15:51:47.745Z: Staged package google-http-client-gson-1.39.2-z4GUT9Qf4NqoabkTvQqq7w5Nj-Z3ShlVQPRU7NPPULY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-http-client-gson-1.39.2-z4GUT9Qf4NqoabkTvQqq7w5Nj-Z3ShlVQPRU7NPPULY.jar' is inaccessible.
Oct 18, 2021 3:51:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-18T15:51:47.792Z: Staged package google-http-client-jackson2-1.39.2-dKSTp7ol4d-z06Td9xMwbVo3htsAo0idWQYdyM_zIbE.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-http-client-jackson2-1.39.2-dKSTp7ol4d-z06Td9xMwbVo3htsAo0idWQYdyM_zIbE.jar' is inaccessible.
Oct 18, 2021 3:51:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-18T15:51:47.841Z: Staged package google-http-client-protobuf-1.39.2-48CRMH_IJ9gY8fZ3-5ANC0qF7rqbijaXD90xuQTMr5s.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-http-client-protobuf-1.39.2-48CRMH_IJ9gY8fZ3-5ANC0qF7rqbijaXD90xuQTMr5s.jar' is inaccessible.
Oct 18, 2021 3:51:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-18T15:51:51.684Z: Staged package opencensus-contrib-grpc-util-0.28.0-uRaDRuavZZMwChvCfvdCVKofJAGYhZON2PuFK4d9VfA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/opencensus-contrib-grpc-util-0.28.0-uRaDRuavZZMwChvCfvdCVKofJAGYhZON2PuFK4d9VfA.jar' is inaccessible.
Oct 18, 2021 3:51:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-18T15:51:52.173Z: Staged package zstd-jni-1.4.5-2-lzQxwUtNCahuI7cYQRb8rC2FUB60p0MPfRhczhr0YFA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/zstd-jni-1.4.5-2-lzQxwUtNCahuI7cYQRb8rC2FUB60p0MPfRhczhr0YFA.jar' is inaccessible.
Oct 18, 2021 3:51:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-10-18T15:51:52.196Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Oct 18, 2021 3:54:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-10-18T15:54:49.008Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Oct 18, 2021 3:57:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-18T15:57:46.185Z: Staged package checker-qual-3.10.0-pNyILKaqxJbTM4Hl5esGBMRUg7AEvD6sk2jxuxfLJRI.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/checker-qual-3.10.0-pNyILKaqxJbTM4Hl5esGBMRUg7AEvD6sk2jxuxfLJRI.jar' is inaccessible.
Oct 18, 2021 3:57:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-18T15:57:46.940Z: Staged package google-http-client-1.39.2-arSWeGCSfwUd1_K1deS3qO8182fcLjQIBzqk3uMo4ic.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-http-client-1.39.2-arSWeGCSfwUd1_K1deS3qO8182fcLjQIBzqk3uMo4ic.jar' is inaccessible.
Oct 18, 2021 3:57:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-18T15:57:46.987Z: Staged package google-http-client-apache-v2-1.39.2-UtOSCfZav29XfhyrRmrbmfEtHNK6TpjYblAfddf4UM8.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-http-client-apache-v2-1.39.2-UtOSCfZav29XfhyrRmrbmfEtHNK6TpjYblAfddf4UM8.jar' is inaccessible.
Oct 18, 2021 3:57:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-18T15:57:47.029Z: Staged package google-http-client-appengine-1.39.2-9jINwNsDWv9uqy4odIVg2nGKtGX7w0gbZT6j8nqlysk.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-http-client-appengine-1.39.2-9jINwNsDWv9uqy4odIVg2nGKtGX7w0gbZT6j8nqlysk.jar' is inaccessible.
Oct 18, 2021 3:57:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-18T15:57:47.066Z: Staged package google-http-client-gson-1.39.2-z4GUT9Qf4NqoabkTvQqq7w5Nj-Z3ShlVQPRU7NPPULY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-http-client-gson-1.39.2-z4GUT9Qf4NqoabkTvQqq7w5Nj-Z3ShlVQPRU7NPPULY.jar' is inaccessible.
Oct 18, 2021 3:57:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-18T15:57:47.101Z: Staged package google-http-client-jackson2-1.39.2-dKSTp7ol4d-z06Td9xMwbVo3htsAo0idWQYdyM_zIbE.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-http-client-jackson2-1.39.2-dKSTp7ol4d-z06Td9xMwbVo3htsAo0idWQYdyM_zIbE.jar' is inaccessible.
Oct 18, 2021 3:57:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-18T15:57:47.135Z: Staged package google-http-client-protobuf-1.39.2-48CRMH_IJ9gY8fZ3-5ANC0qF7rqbijaXD90xuQTMr5s.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-http-client-protobuf-1.39.2-48CRMH_IJ9gY8fZ3-5ANC0qF7rqbijaXD90xuQTMr5s.jar' is inaccessible.
Oct 18, 2021 3:57:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-18T15:57:48.240Z: Staged package opencensus-contrib-grpc-util-0.28.0-uRaDRuavZZMwChvCfvdCVKofJAGYhZON2PuFK4d9VfA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/opencensus-contrib-grpc-util-0.28.0-uRaDRuavZZMwChvCfvdCVKofJAGYhZON2PuFK4d9VfA.jar' is inaccessible.
Oct 18, 2021 3:57:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-18T15:57:48.720Z: Staged package zstd-jni-1.4.5-2-lzQxwUtNCahuI7cYQRb8rC2FUB60p0MPfRhczhr0YFA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/zstd-jni-1.4.5-2-lzQxwUtNCahuI7cYQRb8rC2FUB60p0MPfRhczhr0YFA.jar' is inaccessible.
Oct 18, 2021 3:57:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-10-18T15:57:48.744Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Oct 18, 2021 4:00:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-18T16:00:25.846Z: Cancel request is committed for workflow job: 2021-10-18_05_45_31-14203049816433255547.
Oct 18, 2021 4:00:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-18T16:00:25.920Z: Cleaning up.
Oct 18, 2021 4:00:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-18T16:00:26.018Z: Stopping **** pool...
Oct 18, 2021 4:00:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-18T16:00:26.077Z: Stopping **** pool...
Oct 18, 2021 4:02:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-18T16:02:50.941Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Oct 18, 2021 4:02:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-18T16:02:50.979Z: Worker pool stopped.
Oct 18, 2021 4:02:56 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-10-18_05_45_31-14203049816433255547 finished with status CANCELLED.
Load test results for test (ID): 43d5a22c-7d4e-467d-b8ba-360457bd8fb0 and timestamp: 2021-10-18T12:45:26.825000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11541.005
dataflow_v2_java11_total_bytes_count             2.66347914E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211018124331
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:4df8ffb0b0f0f953f605731eb08a71de653af8fe1b3e9b93e4f42638fc4a24ba
Deleted: sha256:a9c8e6a10e40c6458c674ad3df0f71f874203fde2d4f097f76e4ca126c5f5bd7
Deleted: sha256:3dbaaf904543b89a81c99997561f9fc1e1ca04d0465c13a9cbeffb7117a418ed
Deleted: sha256:9be7aff646e3ecac7ffe5b5729cb2f10468ec343294a5e6f89a18c0f84538b56
Deleted: sha256:17dc3c4c167c47ec274269d5a7e338afa813d3a86c1cb1ee730a3d4c685dde9c
Deleted: sha256:7d7172892a76fa790a898b2b8225479efe3d59f0038cfe69b08af3b3f9e1625b
Deleted: sha256:6e94d0611cec08dbe29f947ec16346fff79538800eb712873a0a20b7b7b65283
Deleted: sha256:12a0a42cefa779ad517829f79c3b844598a7378b8c32d699f6eafc70b17f8d62
Deleted: sha256:cd9835914c02a5d47c5c13a47ad0299c215d3b70a986e0ac9f2a7bcc0d05f388
Deleted: sha256:b969294e2397f76eab4d2a31e5c95404627e4133f65a5c39d4cd120471f9bff8
Deleted: sha256:35bc0b3e42753c7df41a4fe00330a9c0c74ef569077002af11b98ca45de41a98
Deleted: sha256:d888356de3ffc6ab1621e26e546f78e96a903e24a948443834a2578d808312f7
Deleted: sha256:1e5f414dd9725a4a7aca61ee9b38e356995bc8d62454f3aacd414deb4c2a9659
Deleted: sha256:0b91fe418ba6384286cab87ebe78460c1adc96cf39684041f0ba53c27d2eef2a
Deleted: sha256:ed6dc7e7255bc6a359c7cc86243dcd53f8a67c29168767b9be9fedc5f29ebfff
Deleted: sha256:8852fc44a380cf00ad8b3aff3e65f40b368191cdabf6a026e6efdc79cbd30051
Deleted: sha256:9ae3ec8d051530b90a6ea38063bcef8e42b01742c5236195c7c2d71a839db82c
Deleted: sha256:2e0b196ed93369c3f6c1713b45347ac214fa5d848e9e2b04245ec7a980884c9f
Deleted: sha256:d9127b1356b05acc490eddd987582d0db8b2ac018747b3dd0dc4a09c28c56048
Deleted: sha256:9884a28cd1a12d5a25c815afc824da9c1462284ceee324befebaaa04ec1ebbce
Deleted: sha256:581a651601c9058540e65efc34628708043a174c672613ec6e9f701f9aba5769
Deleted: sha256:56abf364cb5b331fe81c2e0ed53998a978e4b867ac646c6884af15cc6039c914
Deleted: sha256:3a9c8ca970db3476d1318117e075652be3dc54a17ff4c6dcbea91b69ba6a809d
Deleted: sha256:af1d7e3142536074b5e92da4f36546c65c316d645f471bb64bcf2023b56d6351
Deleted: sha256:394935694891be25b97736cfa134e8122d07066e7158e6d987430f3680a444b4
Deleted: sha256:26496153c8d40746d3ad5ff311ca6020ea4db804a87a59b2f94958136c12f1be
Deleted: sha256:f008f4d0af97cd55a2f3e41031a1a25c8b32795803aefeef41bd7aa20ab38867
Deleted: sha256:6f929a762cf92dcaaa08e1a66e1f937f1eff277f7bea37497364820d8e613c1a
Deleted: sha256:b114444ee7535e21dec502a8781fa7fb1b96ad41d139c4aa142398911e2f12b1
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211018124331]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:4df8ffb0b0f0f953f605731eb08a71de653af8fe1b3e9b93e4f42638fc4a24ba]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211018124331] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:4df8ffb0b0f0f953f605731eb08a71de653af8fe1b3e9b93e4f42638fc4a24ba])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:4df8ffb0b0f0f953f605731eb08a71de653af8fe1b3e9b93e4f42638fc4a24ba
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:4df8ffb0b0f0f953f605731eb08a71de653af8fe1b3e9b93e4f42638fc4a24ba
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:4df8ffb0b0f0f953f605731eb08a71de653af8fe1b3e9b93e4f42638fc4a24ba].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 40s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/p7s2jbh462rrk

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #122

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/122/display/redirect>

Changes:


------------------------------------------
[...truncated 48.52 KB...]
The push refers to repository [us.gcr.io/apache-beam-testing/java-postcommit-it/java]
0d03cb6de735: Preparing
373cfc536b92: Preparing
d7d59b119c13: Preparing
08731deb5899: Preparing
6f863fa754a0: Preparing
735ad4aee5b9: Preparing
57c28a12ac5a: Preparing
66b571650cb9: Preparing
ef13a8f3645d: Preparing
42f9c59d4994: Preparing
be05c80060cb: Preparing
e63be041c4ce: Preparing
211736d7fc9d: Preparing
62a5b8741e83: Preparing
36e0782f1159: Preparing
ba6e5ff31f23: Preparing
9f9f651e9303: Preparing
735ad4aee5b9: Waiting
0b3c02b5d746: Preparing
66b571650cb9: Waiting
62a747bf1719: Preparing
57c28a12ac5a: Waiting
42f9c59d4994: Waiting
ef13a8f3645d: Waiting
be05c80060cb: Waiting
e63be041c4ce: Waiting
211736d7fc9d: Waiting
ba6e5ff31f23: Waiting
36e0782f1159: Waiting
0b3c02b5d746: Waiting
62a747bf1719: Waiting
62a5b8741e83: Waiting
d7d59b119c13: Pushed
6f863fa754a0: Pushed
373cfc536b92: Pushed
735ad4aee5b9: Pushed
0d03cb6de735: Pushed
08731deb5899: Pushed
66b571650cb9: Pushed
ef13a8f3645d: Pushed
211736d7fc9d: Layer already exists
62a5b8741e83: Layer already exists
36e0782f1159: Layer already exists
be05c80060cb: Pushed
ba6e5ff31f23: Layer already exists
57c28a12ac5a: Pushed
9f9f651e9303: Layer already exists
0b3c02b5d746: Layer already exists
62a747bf1719: Layer already exists
e63be041c4ce: Pushed
42f9c59d4994: Pushed
20211017124332: digest: sha256:2179fd19be59fd1b3625061f91acd7781b377b4b53fa39b9c539dfee16c5ecb9 size: 4311

> Task :sdks:java:testing:load-tests:run
Oct 17, 2021 12:45:21 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Oct 17, 2021 12:45:21 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 195 files. Enable logging at DEBUG level to see which files will be staged.
Oct 17, 2021 12:45:22 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Oct 17, 2021 12:45:22 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Oct 17, 2021 12:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 195 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Oct 17, 2021 12:45:26 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 195 files cached, 0 files newly uploaded in 0 seconds
Oct 17, 2021 12:45:26 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Oct 17, 2021 12:45:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <108949 bytes, hash 5e28b9a098b764a4bcbb22a3f47c2d450db954c77953aa6fead07d9d8106cb9c> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-Xii5oJi3ZKS8uyKj9HwtRQ25VMd5U6pv6tB9nYEGy5w.pb
Oct 17, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Oct 17, 2021 12:45:27 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@ae202c6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@46aa712c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6ada9c0c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7412ed6b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6e7c351d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b4a0aef, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@45cec376, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26d41711, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@298f0a0b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2b960a7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@31dfc6f5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@37b52340, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@663bb8ef, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2f4e40d7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@60e9c3a5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7e5843db, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@459f703f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@188ac8a3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3650d4fc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@aaa0f76]
Oct 17, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Oct 17, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Oct 17, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Oct 17, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Oct 17, 2021 12:45:27 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1df1ced0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5349b246, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@32b0876c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2aaf152b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@39bbd9e0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@27fe9713, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@11c3ff67, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4397a639, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@24841372, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77114efe, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@79a7bfbc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77f68df, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3e4e4c1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7e7f3cfd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ae126d1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@46a488c2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6242ae3b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@65ddee5a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@56399b9e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@34b9eb03]
Oct 17, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Oct 17, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Oct 17, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Oct 17, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Oct 17, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Oct 17, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Oct 17, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Oct 17, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Oct 17, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Oct 17, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Oct 17, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Oct 17, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.35.0-SNAPSHOT
Oct 17, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-10-17_05_45_28-12751250368988367149?project=apache-beam-testing
Oct 17, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-10-17_05_45_28-12751250368988367149
Oct 17, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-10-17_05_45_28-12751250368988367149
Oct 17, 2021 12:45:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-10-17T12:45:33.331Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-10-1npm. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Oct 17, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-17T12:45:36.516Z: Worker configuration: e2-standard-2 in us-central1-a.
Oct 17, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-17T12:45:37.321Z: Expanding SplittableParDo operations into optimizable parts.
Oct 17, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-17T12:45:37.347Z: Expanding CollectionToSingleton operations into optimizable parts.
Oct 17, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-17T12:45:37.402Z: Expanding CoGroupByKey operations into optimizable parts.
Oct 17, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-17T12:45:37.476Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Oct 17, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-17T12:45:37.505Z: Expanding GroupByKey operations into streaming Read/Write steps
Oct 17, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-17T12:45:37.559Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Oct 17, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-17T12:45:37.652Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Oct 17, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-17T12:45:37.687Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Oct 17, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-17T12:45:37.717Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Oct 17, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-17T12:45:37.751Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Oct 17, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-17T12:45:37.782Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Oct 17, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-17T12:45:37.799Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Oct 17, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-17T12:45:37.830Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Oct 17, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-17T12:45:37.866Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Oct 17, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-17T12:45:37.899Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Oct 17, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-17T12:45:37.926Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Oct 17, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-17T12:45:37.956Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Oct 17, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-17T12:45:37.981Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Oct 17, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-17T12:45:38.003Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Oct 17, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-17T12:45:38.034Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Oct 17, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-17T12:45:38.056Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Oct 17, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-17T12:45:38.089Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Oct 17, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-17T12:45:38.120Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Oct 17, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-17T12:45:38.155Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Oct 17, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-17T12:45:38.189Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Oct 17, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-17T12:45:38.217Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Oct 17, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-17T12:45:38.244Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Oct 17, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-17T12:45:38.278Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Oct 17, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-17T12:45:38.321Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Oct 17, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-17T12:45:38.691Z: Starting 5 ****s in us-central1-a...
Oct 17, 2021 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-17T12:45:54.570Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Oct 17, 2021 12:46:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-17T12:46:22.041Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Oct 17, 2021 12:47:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-17T12:47:16.455Z: Workers have started successfully.
Oct 17, 2021 12:47:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-17T12:47:16.484Z: Workers have started successfully.
Oct 17, 2021 4:00:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-17T16:00:26.861Z: Cancel request is committed for workflow job: 2021-10-17_05_45_28-12751250368988367149.
Oct 17, 2021 4:00:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-17T16:00:26.939Z: Cleaning up.
Oct 17, 2021 4:00:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-17T16:00:27.104Z: Stopping **** pool...
Oct 17, 2021 4:00:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-17T16:00:27.163Z: Stopping **** pool...
Oct 17, 2021 4:03:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-17T16:03:06.035Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Oct 17, 2021 4:03:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-17T16:03:06.072Z: Worker pool stopped.
Oct 17, 2021 4:03:12 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-10-17_05_45_28-12751250368988367149 finished with status CANCELLED.
Load test results for test (ID): 31c80f47-4469-4607-afe5-9082afe97d61 and timestamp: 2021-10-17T12:45:22.254000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11539.799
dataflow_v2_java11_total_bytes_count             2.36353573E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211017124332
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:2179fd19be59fd1b3625061f91acd7781b377b4b53fa39b9c539dfee16c5ecb9
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211017124332]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:2179fd19be59fd1b3625061f91acd7781b377b4b53fa39b9c539dfee16c5ecb9]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211017124332] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:2179fd19be59fd1b3625061f91acd7781b377b4b53fa39b9c539dfee16c5ecb9])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:2179fd19be59fd1b3625061f91acd7781b377b4b53fa39b9c539dfee16c5ecb9
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:2179fd19be59fd1b3625061f91acd7781b377b4b53fa39b9c539dfee16c5ecb9
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:2179fd19be59fd1b3625061f91acd7781b377b4b53fa39b9c539dfee16c5ecb9].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 57s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/sd4pab4htlagc

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #121

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/121/display/redirect?page=changes>

Changes:

[noreply] Corrected Join Example

[noreply] Minor: Add more links to DataFrame API documentation (#15661)

[noreply] [BEAM-11480] Use snippets for DataFrame examples (#15600)

[noreply] Allow multiple Python worker processe to share the same VM. (#15642)

[noreply] [BEAM-12564] Implement Series.hasnans (#15729)

[noreply] Minor: Fix frames_test.py equality check for non-frame outputs (#15734)

[noreply] [BEAM-12769] Adds integration tests for Java Class Lookup based

[Brian Hulette] Address BEAM-4028 cleanup TODOs

[Brian Hulette] Bump Dataflow container to beam-master-20211015

[noreply] [BEAM-13052] Restructure pubsublite folder to move non-user interface


------------------------------------------
[...truncated 58.09 KB...]
INFO: Adding Ungroup and reiterate as step s14
Oct 16, 2021 12:45:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Oct 16, 2021 12:45:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Oct 16, 2021 12:45:48 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.35.0-SNAPSHOT
Oct 16, 2021 12:45:49 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-10-16_05_45_49-17323798727383694038?project=apache-beam-testing
Oct 16, 2021 12:45:49 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-10-16_05_45_49-17323798727383694038
Oct 16, 2021 12:45:49 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-10-16_05_45_49-17323798727383694038
Oct 16, 2021 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-10-16T12:45:54.539Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-10-pft5. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Oct 16, 2021 12:46:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-16T12:45:59.700Z: Worker configuration: e2-standard-2 in us-central1-a.
Oct 16, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-16T12:46:00.574Z: Expanding SplittableParDo operations into optimizable parts.
Oct 16, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-16T12:46:00.622Z: Expanding CollectionToSingleton operations into optimizable parts.
Oct 16, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-16T12:46:00.694Z: Expanding CoGroupByKey operations into optimizable parts.
Oct 16, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-16T12:46:00.765Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Oct 16, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-16T12:46:00.793Z: Expanding GroupByKey operations into streaming Read/Write steps
Oct 16, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-16T12:46:00.851Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Oct 16, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-16T12:46:00.952Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Oct 16, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-16T12:46:00.987Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Oct 16, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-16T12:46:01.019Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Oct 16, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-16T12:46:01.052Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Oct 16, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-16T12:46:01.086Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Oct 16, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-16T12:46:01.110Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Oct 16, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-16T12:46:01.143Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Oct 16, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-16T12:46:01.174Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Oct 16, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-16T12:46:01.209Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Oct 16, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-16T12:46:01.244Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Oct 16, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-16T12:46:01.275Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Oct 16, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-16T12:46:01.318Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Oct 16, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-16T12:46:01.351Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Oct 16, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-16T12:46:01.391Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Oct 16, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-16T12:46:01.427Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Oct 16, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-16T12:46:01.464Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Oct 16, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-16T12:46:01.492Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Oct 16, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-16T12:46:01.518Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Oct 16, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-16T12:46:01.546Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Oct 16, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-16T12:46:01.602Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Oct 16, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-16T12:46:01.629Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Oct 16, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-16T12:46:01.673Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Oct 16, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-16T12:46:01.714Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Oct 16, 2021 12:46:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-16T12:46:02.125Z: Starting 5 ****s in us-central1-a...
Oct 16, 2021 12:46:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-16T12:46:27.138Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Oct 16, 2021 12:46:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-16T12:46:35.807Z: Autoscaling: Raised the number of ****s to 3 so that the pipeline can catch up with its backlog and keep up with its input rate.
Oct 16, 2021 12:46:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-16T12:46:35.837Z: Resized **** pool to 3, though goal was 5.  This could be a quota issue.
Oct 16, 2021 12:46:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-16T12:46:46.060Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Oct 16, 2021 12:47:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-16T12:47:40.119Z: Workers have started successfully.
Oct 16, 2021 12:47:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-16T12:47:40.153Z: Workers have started successfully.
Oct 16, 2021 1:19:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-16T13:19:03.749Z: Staged package failureaccess-1.0.1-oXHuTHNN0tqDfksWvp30Zhr6typBra8x64Tf2vk2yiY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/failureaccess-1.0.1-oXHuTHNN0tqDfksWvp30Zhr6typBra8x64Tf2vk2yiY.jar' is inaccessible.
Oct 16, 2021 1:19:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-16T13:19:04.603Z: Staged package google-oauth-client-1.31.0-9fiaR9DCEOJ3VPxypLVvfxhohPwMB0pyoLY-Gc52QtA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-oauth-client-1.31.0-9fiaR9DCEOJ3VPxypLVvfxhohPwMB0pyoLY-Gc52QtA.jar' is inaccessible.
Oct 16, 2021 1:19:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-16T13:19:05.723Z: Staged package listenablefuture-9999.0-empty-to-avoid-conflict-with-guava-s3KgN9QjCqV_vv_e8w_WEj-cDC24XQrO0AyRuXTzP5k.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava-s3KgN9QjCqV_vv_e8w_WEj-cDC24XQrO0AyRuXTzP5k.jar' is inaccessible.
Oct 16, 2021 1:19:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-16T13:19:06.050Z: Staged package netty-tcnative-boringssl-static-2.0.33.Final-q3CryMTeke7zEAdYTla2hvWe2cxEwrM38_6C8EAtOH0.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-tcnative-boringssl-static-2.0.33.Final-q3CryMTeke7zEAdYTla2hvWe2cxEwrM38_6C8EAtOH0.jar' is inaccessible.
Oct 16, 2021 1:19:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-16T13:19:06.698Z: Staged package threetenbp-1.5.1-Q0LuBNhwQPcbCqkYjulgeA7y2nNOMqjUOlIqWAteDzs.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/threetenbp-1.5.1-Q0LuBNhwQPcbCqkYjulgeA7y2nNOMqjUOlIqWAteDzs.jar' is inaccessible.
Oct 16, 2021 1:19:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-10-16T13:19:06.811Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Oct 16, 2021 1:22:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-10-16T13:22:06.787Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Oct 16, 2021 1:25:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-16T13:25:03.744Z: Staged package failureaccess-1.0.1-oXHuTHNN0tqDfksWvp30Zhr6typBra8x64Tf2vk2yiY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/failureaccess-1.0.1-oXHuTHNN0tqDfksWvp30Zhr6typBra8x64Tf2vk2yiY.jar' is inaccessible.
Oct 16, 2021 1:25:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-16T13:25:04.634Z: Staged package google-oauth-client-1.31.0-9fiaR9DCEOJ3VPxypLVvfxhohPwMB0pyoLY-Gc52QtA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-oauth-client-1.31.0-9fiaR9DCEOJ3VPxypLVvfxhohPwMB0pyoLY-Gc52QtA.jar' is inaccessible.
Oct 16, 2021 1:25:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-16T13:25:05.897Z: Staged package listenablefuture-9999.0-empty-to-avoid-conflict-with-guava-s3KgN9QjCqV_vv_e8w_WEj-cDC24XQrO0AyRuXTzP5k.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava-s3KgN9QjCqV_vv_e8w_WEj-cDC24XQrO0AyRuXTzP5k.jar' is inaccessible.
Oct 16, 2021 1:25:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-16T13:25:06.215Z: Staged package netty-tcnative-boringssl-static-2.0.33.Final-q3CryMTeke7zEAdYTla2hvWe2cxEwrM38_6C8EAtOH0.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-tcnative-boringssl-static-2.0.33.Final-q3CryMTeke7zEAdYTla2hvWe2cxEwrM38_6C8EAtOH0.jar' is inaccessible.
Oct 16, 2021 1:25:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-16T13:25:06.753Z: Staged package threetenbp-1.5.1-Q0LuBNhwQPcbCqkYjulgeA7y2nNOMqjUOlIqWAteDzs.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/threetenbp-1.5.1-Q0LuBNhwQPcbCqkYjulgeA7y2nNOMqjUOlIqWAteDzs.jar' is inaccessible.
Oct 16, 2021 1:25:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-10-16T13:25:06.889Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Oct 16, 2021 1:28:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-10-16T13:28:06.865Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Oct 16, 2021 1:31:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-16T13:31:03.590Z: Staged package failureaccess-1.0.1-oXHuTHNN0tqDfksWvp30Zhr6typBra8x64Tf2vk2yiY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/failureaccess-1.0.1-oXHuTHNN0tqDfksWvp30Zhr6typBra8x64Tf2vk2yiY.jar' is inaccessible.
Oct 16, 2021 1:31:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-16T13:31:04.668Z: Staged package google-oauth-client-1.31.0-9fiaR9DCEOJ3VPxypLVvfxhohPwMB0pyoLY-Gc52QtA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-oauth-client-1.31.0-9fiaR9DCEOJ3VPxypLVvfxhohPwMB0pyoLY-Gc52QtA.jar' is inaccessible.
Oct 16, 2021 1:31:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-16T13:31:05.756Z: Staged package listenablefuture-9999.0-empty-to-avoid-conflict-with-guava-s3KgN9QjCqV_vv_e8w_WEj-cDC24XQrO0AyRuXTzP5k.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava-s3KgN9QjCqV_vv_e8w_WEj-cDC24XQrO0AyRuXTzP5k.jar' is inaccessible.
Oct 16, 2021 1:31:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-16T13:31:06.063Z: Staged package netty-tcnative-boringssl-static-2.0.33.Final-q3CryMTeke7zEAdYTla2hvWe2cxEwrM38_6C8EAtOH0.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-tcnative-boringssl-static-2.0.33.Final-q3CryMTeke7zEAdYTla2hvWe2cxEwrM38_6C8EAtOH0.jar' is inaccessible.
Oct 16, 2021 1:31:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-16T13:31:06.693Z: Staged package threetenbp-1.5.1-Q0LuBNhwQPcbCqkYjulgeA7y2nNOMqjUOlIqWAteDzs.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/threetenbp-1.5.1-Q0LuBNhwQPcbCqkYjulgeA7y2nNOMqjUOlIqWAteDzs.jar' is inaccessible.
Oct 16, 2021 1:31:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-10-16T13:31:06.797Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Oct 16, 2021 1:34:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-10-16T13:34:07.023Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Oct 16, 2021 1:37:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-16T13:37:03.471Z: Staged package failureaccess-1.0.1-oXHuTHNN0tqDfksWvp30Zhr6typBra8x64Tf2vk2yiY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/failureaccess-1.0.1-oXHuTHNN0tqDfksWvp30Zhr6typBra8x64Tf2vk2yiY.jar' is inaccessible.
Oct 16, 2021 1:37:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-16T13:37:04.262Z: Staged package google-oauth-client-1.31.0-9fiaR9DCEOJ3VPxypLVvfxhohPwMB0pyoLY-Gc52QtA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-oauth-client-1.31.0-9fiaR9DCEOJ3VPxypLVvfxhohPwMB0pyoLY-Gc52QtA.jar' is inaccessible.
Oct 16, 2021 1:37:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-16T13:37:05.367Z: Staged package listenablefuture-9999.0-empty-to-avoid-conflict-with-guava-s3KgN9QjCqV_vv_e8w_WEj-cDC24XQrO0AyRuXTzP5k.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava-s3KgN9QjCqV_vv_e8w_WEj-cDC24XQrO0AyRuXTzP5k.jar' is inaccessible.
Oct 16, 2021 1:37:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-16T13:37:05.671Z: Staged package netty-tcnative-boringssl-static-2.0.33.Final-q3CryMTeke7zEAdYTla2hvWe2cxEwrM38_6C8EAtOH0.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-tcnative-boringssl-static-2.0.33.Final-q3CryMTeke7zEAdYTla2hvWe2cxEwrM38_6C8EAtOH0.jar' is inaccessible.
Oct 16, 2021 1:37:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-16T13:37:06.252Z: Staged package threetenbp-1.5.1-Q0LuBNhwQPcbCqkYjulgeA7y2nNOMqjUOlIqWAteDzs.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/threetenbp-1.5.1-Q0LuBNhwQPcbCqkYjulgeA7y2nNOMqjUOlIqWAteDzs.jar' is inaccessible.
Oct 16, 2021 1:37:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-10-16T13:37:06.366Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Oct 16, 2021 1:40:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-10-16T13:40:06.274Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Oct 16, 2021 1:43:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-16T13:43:03.533Z: Staged package failureaccess-1.0.1-oXHuTHNN0tqDfksWvp30Zhr6typBra8x64Tf2vk2yiY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/failureaccess-1.0.1-oXHuTHNN0tqDfksWvp30Zhr6typBra8x64Tf2vk2yiY.jar' is inaccessible.
Oct 16, 2021 1:43:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-16T13:43:04.274Z: Staged package google-oauth-client-1.31.0-9fiaR9DCEOJ3VPxypLVvfxhohPwMB0pyoLY-Gc52QtA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-oauth-client-1.31.0-9fiaR9DCEOJ3VPxypLVvfxhohPwMB0pyoLY-Gc52QtA.jar' is inaccessible.
Oct 16, 2021 1:43:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-16T13:43:05.544Z: Staged package listenablefuture-9999.0-empty-to-avoid-conflict-with-guava-s3KgN9QjCqV_vv_e8w_WEj-cDC24XQrO0AyRuXTzP5k.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava-s3KgN9QjCqV_vv_e8w_WEj-cDC24XQrO0AyRuXTzP5k.jar' is inaccessible.
Oct 16, 2021 1:43:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-16T13:43:05.963Z: Staged package netty-tcnative-boringssl-static-2.0.33.Final-q3CryMTeke7zEAdYTla2hvWe2cxEwrM38_6C8EAtOH0.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-tcnative-boringssl-static-2.0.33.Final-q3CryMTeke7zEAdYTla2hvWe2cxEwrM38_6C8EAtOH0.jar' is inaccessible.
Oct 16, 2021 1:43:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-16T13:43:06.816Z: Staged package threetenbp-1.5.1-Q0LuBNhwQPcbCqkYjulgeA7y2nNOMqjUOlIqWAteDzs.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/threetenbp-1.5.1-Q0LuBNhwQPcbCqkYjulgeA7y2nNOMqjUOlIqWAteDzs.jar' is inaccessible.
Oct 16, 2021 1:43:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-10-16T13:43:06.953Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Oct 16, 2021 1:46:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-10-16T13:46:06.252Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Oct 16, 2021 4:00:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-16T16:00:27.167Z: Cancel request is committed for workflow job: 2021-10-16_05_45_49-17323798727383694038.
Oct 16, 2021 4:00:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-16T16:00:27.227Z: Cleaning up.
Oct 16, 2021 4:00:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-16T16:00:27.374Z: Stopping **** pool...
Oct 16, 2021 4:00:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-16T16:00:27.445Z: Stopping **** pool...
Oct 16, 2021 4:02:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-16T16:02:49.591Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Oct 16, 2021 4:02:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-16T16:02:49.638Z: Worker pool stopped.
Oct 16, 2021 4:02:57 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-10-16_05_45_49-17323798727383694038 finished with status CANCELLED.
Load test results for test (ID): 13df6ea1-35df-4668-b89d-87eb2f85798e and timestamp: 2021-10-16T12:45:43.518000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                  11528.11
dataflow_v2_java11_total_bytes_count             2.56458986E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211016124335
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:1139a919877fc8a186c189c0b8d5e7658db9db5dace1af8b78d16323f55be1e6
Deleted: sha256:c6608b6371c4637110e361d876b767919e20107a3d6e982557fca038825b21a0
Deleted: sha256:a5692feace293c810ad0f53d96f307288c301534a4aa63c8aa8c3e14959b9233
Deleted: sha256:13d26b6f316d4a433baa86d59e4a468137c6dabf5fb5788bfebe01382496b190
Deleted: sha256:3c9134dbc80125648493f303549a6f88fa321d85e497fd652861b9a2551318af
Deleted: sha256:5bdb43ffdccd3275cb844aed35217d28fe43b54bda72c14edc31d8a958a1339c
Deleted: sha256:f0137e512f65a6f8b679f5ea8ec98cbc22c94ef0ec57d0b5f2f201279ccedadf
Deleted: sha256:efdad8ce1b34f47f2e737402df6a80b86d0a188ded817b79b30ef2483c026a93
Deleted: sha256:c301f0738a9ac93cca8d154a89b7234f75151600b9f76ca734f1c031e090e34b
Deleted: sha256:6f115d7dcccc1934213dade5be75ec199935f6c01c09ad418bc9d59f8e21dbc5
Deleted: sha256:3393a42e8a1ee7e56d2a5158862df7dcef6e80493b0aa8b48ae72cc7678f3228
Deleted: sha256:f711be6ab86448ce20c8351c0848aab43f265948989121a7369e53f559edc729
Deleted: sha256:58ff9fb58d37de28f1cba77f31b8431e90df3290df2818e2f1af0bb887bdf192
Deleted: sha256:2e00d8c9ca6557b4be89e4e146a2b127eae08fbd1628653b6c8468062f06c001
Deleted: sha256:46804d896e74d4f1e95531875be0142af5291abe916f04064b27dac263839cd8
Deleted: sha256:34b4caed1a3bdf5be866dbabcd9ca98f7b38033f0dd995de6eb3c622e711672b
Deleted: sha256:5908e43a92de689066a54b33fb14e1ad226b96fc554bd164c9a6914c3c619db2
Deleted: sha256:eb932af945883c033fa3fc6b93da740b9b77596e157c423e84a78ba36dbecf62
Deleted: sha256:04ecf2be4164234ff927925174181832e863583b836dbd98647f2958f38f3522
Deleted: sha256:403ab567820fe7129b2d117c8663e5b852e341ff5003ad5d83777384cb9b3abe
Deleted: sha256:ddb1bf0bbbbd3f35f354ff392659720ddfd292f1ac80c03931ef3baa1bcaebea
Deleted: sha256:8f68508c4657b06b94a02853e848fef202bf2b8ba3eefaaeb7fb5a330743675e
Deleted: sha256:2a05e65b8c777c3462ad9574123dc3cd316856740ea67ab92e27d3343bbce2e6
Deleted: sha256:c6021de9fbb391c81380278cf46d13fd09c05247b350549e989deae8b1bad323
Deleted: sha256:2c3032d4a45f19e4e9ab451c4def71962133f19904e230b250acb781fea9366b
Deleted: sha256:36691e24c1412315338563b813baa922710ec6e4d385ffe3290c7ce3ab161032
Deleted: sha256:f84fd20473a63ee5f8b1db36aec0cd0987006d32729bd52fe96cdb39bd821fc7
Deleted: sha256:964ec6d0fd593c23dc6dc382addf4d87f9c770583321e430facdde78a8985b3a
Deleted: sha256:6ab16c16fe924bef92b502027c8594854970a7e7bb7e6bf5454e73504eaf43c6
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211016124335]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:1139a919877fc8a186c189c0b8d5e7658db9db5dace1af8b78d16323f55be1e6]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211016124335] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:1139a919877fc8a186c189c0b8d5e7658db9db5dace1af8b78d16323f55be1e6])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:1139a919877fc8a186c189c0b8d5e7658db9db5dace1af8b78d16323f55be1e6
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:1139a919877fc8a186c189c0b8d5e7658db9db5dace1af8b78d16323f55be1e6
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:1139a919877fc8a186c189c0b8d5e7658db9db5dace1af8b78d16323f55be1e6].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 38s
101 actionable tasks: 71 executed, 28 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/khlinekl22gz2

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #120

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/120/display/redirect?page=changes>

Changes:

[ilya.kozyrev] Add java executor to compile java code in created file system

[zyichi] Use daemon thread to report lull operation and move code to

[noreply] Merge pull request #15722 from [BEAM-13022] [Playground] Run code on the

[Luke Cwik] [BEAM-12640] Skip checkerframework on generated code/test libraries for

[Luke Cwik] [BEAM-13015] Add tiny bundle processing benchmark for Java SDK harness.

[noreply] [BEAM-12922] Test MapState putIfAbsent with no read (#15704)

[Robert Bradshaw] [BEAM-13053] Avoid runner v2 when streaming engine explicitly disabled.

[noreply] [Go SDK] Small typo fixes. (#15725)

[noreply] [BEAM-13038] Migrate to use a context object within

[noreply] [BEAM-12135] Avoid windowing overhead in Spark global GBK. (#15637)

[noreply] [BEAM-12907] Run DataFrame API tests with multiple pandas versions

[vachan] Remove unnecessary ERROR logging.


------------------------------------------
[...truncated 49.96 KB...]
9f9f651e9303: Waiting
36e0782f1159: Waiting
0b3c02b5d746: Waiting
211736d7fc9d: Waiting
62a747bf1719: Waiting
627ba6eec1f0: Pushed
72c54bb5cb63: Pushed
46d865f218ac: Pushed
95547c4bcd85: Pushed
7f18771badf7: Pushed
67b322ae068a: Pushed
ac04a64b7850: Pushed
e36537bc2320: Pushed
211736d7fc9d: Layer already exists
1fbcbef2a719: Pushed
62a5b8741e83: Layer already exists
fc415c7af1e8: Pushed
36e0782f1159: Layer already exists
0b3c02b5d746: Layer already exists
ba6e5ff31f23: Layer already exists
62a747bf1719: Layer already exists
9f9f651e9303: Layer already exists
db7df9806082: Pushed
65cd9fd5b8f1: Pushed
20211015124339: digest: sha256:37492b5808443f2d5949274093061710a47346c5d4c3f6de22a40f37b8862c95 size: 4311

> Task :sdks:java:testing:load-tests:run
Oct 15, 2021 12:46:01 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Oct 15, 2021 12:46:02 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 195 files. Enable logging at DEBUG level to see which files will be staged.
Oct 15, 2021 12:46:03 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Oct 15, 2021 12:46:03 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Oct 15, 2021 12:46:05 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 195 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Oct 15, 2021 12:46:05 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 195 files cached, 0 files newly uploaded in 0 seconds
Oct 15, 2021 12:46:05 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Oct 15, 2021 12:46:05 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <108949 bytes, hash e8c6a57bd9cef213102fb818e3b4321873673d1af67d4258a2236e133f392a4e> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-6Male9nO8hMQL7gY47QyGHNnPRr2fUJYoiNuEz85Kk4.pb
Oct 15, 2021 12:46:07 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Oct 15, 2021 12:46:07 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2a869a16, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@ae202c6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@46aa712c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6ada9c0c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7412ed6b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6e7c351d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b4a0aef, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@45cec376, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26d41711, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@298f0a0b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2b960a7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@31dfc6f5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@37b52340, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@663bb8ef, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2f4e40d7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@60e9c3a5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7e5843db, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@459f703f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@188ac8a3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3650d4fc]
Oct 15, 2021 12:46:07 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Oct 15, 2021 12:46:07 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Oct 15, 2021 12:46:07 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Oct 15, 2021 12:46:07 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Oct 15, 2021 12:46:07 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@441b8382, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1df1ced0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5349b246, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@32b0876c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2aaf152b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@39bbd9e0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@27fe9713, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@11c3ff67, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4397a639, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@24841372, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77114efe, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@79a7bfbc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77f68df, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3e4e4c1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7e7f3cfd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ae126d1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@46a488c2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6242ae3b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@65ddee5a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@56399b9e]
Oct 15, 2021 12:46:07 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Oct 15, 2021 12:46:07 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Oct 15, 2021 12:46:07 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Oct 15, 2021 12:46:07 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Oct 15, 2021 12:46:07 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Oct 15, 2021 12:46:07 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Oct 15, 2021 12:46:07 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Oct 15, 2021 12:46:07 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Oct 15, 2021 12:46:07 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Oct 15, 2021 12:46:07 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Oct 15, 2021 12:46:07 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Oct 15, 2021 12:46:07 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.35.0-SNAPSHOT
Oct 15, 2021 12:46:08 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-10-15_05_46_07-5099732834723193030?project=apache-beam-testing
Oct 15, 2021 12:46:08 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-10-15_05_46_07-5099732834723193030
Oct 15, 2021 12:46:08 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-10-15_05_46_07-5099732834723193030
Oct 15, 2021 12:46:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-10-15T12:46:14.163Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-10-gywb. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Oct 15, 2021 12:46:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-15T12:46:16.904Z: Worker configuration: e2-standard-2 in us-central1-a.
Oct 15, 2021 12:46:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-15T12:46:17.643Z: Expanding SplittableParDo operations into optimizable parts.
Oct 15, 2021 12:46:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-15T12:46:17.677Z: Expanding CollectionToSingleton operations into optimizable parts.
Oct 15, 2021 12:46:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-15T12:46:17.743Z: Expanding CoGroupByKey operations into optimizable parts.
Oct 15, 2021 12:46:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-15T12:46:17.810Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Oct 15, 2021 12:46:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-15T12:46:17.840Z: Expanding GroupByKey operations into streaming Read/Write steps
Oct 15, 2021 12:46:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-15T12:46:17.889Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Oct 15, 2021 12:46:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-15T12:46:17.989Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Oct 15, 2021 12:46:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-15T12:46:18.018Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Oct 15, 2021 12:46:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-15T12:46:18.039Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Oct 15, 2021 12:46:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-15T12:46:18.065Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Oct 15, 2021 12:46:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-15T12:46:18.087Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Oct 15, 2021 12:46:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-15T12:46:18.115Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Oct 15, 2021 12:46:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-15T12:46:18.137Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Oct 15, 2021 12:46:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-15T12:46:18.160Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Oct 15, 2021 12:46:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-15T12:46:18.195Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Oct 15, 2021 12:46:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-15T12:46:18.222Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Oct 15, 2021 12:46:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-15T12:46:18.248Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Oct 15, 2021 12:46:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-15T12:46:18.275Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Oct 15, 2021 12:46:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-15T12:46:18.313Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Oct 15, 2021 12:46:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-15T12:46:18.348Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Oct 15, 2021 12:46:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-15T12:46:18.369Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Oct 15, 2021 12:46:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-15T12:46:18.398Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Oct 15, 2021 12:46:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-15T12:46:18.415Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Oct 15, 2021 12:46:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-15T12:46:18.438Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Oct 15, 2021 12:46:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-15T12:46:18.468Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Oct 15, 2021 12:46:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-15T12:46:18.501Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Oct 15, 2021 12:46:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-15T12:46:18.528Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Oct 15, 2021 12:46:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-15T12:46:18.559Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Oct 15, 2021 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-15T12:46:18.585Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Oct 15, 2021 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-15T12:46:18.901Z: Starting 5 ****s in us-central1-a...
Oct 15, 2021 12:46:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-15T12:46:26.979Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Oct 15, 2021 12:47:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-15T12:47:05.880Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Oct 15, 2021 12:48:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-15T12:48:07.024Z: Workers have started successfully.
Oct 15, 2021 12:48:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-15T12:48:07.051Z: Workers have started successfully.
Oct 15, 2021 4:00:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-15T16:00:33.261Z: Cancel request is committed for workflow job: 2021-10-15_05_46_07-5099732834723193030.
Oct 15, 2021 4:00:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-15T16:00:33.299Z: Cleaning up.
Oct 15, 2021 4:00:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-15T16:00:33.392Z: Stopping **** pool...
Oct 15, 2021 4:00:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-15T16:00:33.465Z: Stopping **** pool...
Oct 15, 2021 4:02:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-15T16:02:52.630Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Oct 15, 2021 4:02:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-15T16:02:52.689Z: Worker pool stopped.
Oct 15, 2021 4:02:59 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-10-15_05_46_07-5099732834723193030 finished with status CANCELLED.
Load test results for test (ID): 6596d9b1-0af1-4cd3-b7c0-db94f0667212 and timestamp: 2021-10-15T12:46:02.621000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11526.378
dataflow_v2_java11_total_bytes_count             3.17600769E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211015124339
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:37492b5808443f2d5949274093061710a47346c5d4c3f6de22a40f37b8862c95
Deleted: sha256:b0c907852126d6cb7a13515ede17d8d4d200012b8683a1647db3cc0b08bb9ae0
Deleted: sha256:27c5f14935511d28286684c8599aeebf7a6a453e7b52b43d32f38f7aa1d5ad1f
Deleted: sha256:176290c3fe5259eba93d5debe5dff5a464e35a78b2a0fa4c0ff6ddfb0edf709b
Deleted: sha256:0f0f1c4e7e253e139b8a2a47f13fb876660de84016503577d7cacac021f192bb
Deleted: sha256:3ee6217474a1e24a3edb08089bfa4bdb19b2603dd7808671a3da3a6749d1be1b
Deleted: sha256:5d12a97bd9a097c769513067f40c0a9bd3f4a2f09709645d30e1d0757e90dc18
Deleted: sha256:448f9ff4615e854aed0135bec8512f1c639bfbd8a439bd6aebac2297f4f3a913
Deleted: sha256:087553de6f51ae15ef3c64ce8a226e16857f54045fc95b754c9f4ce444c746e7
Deleted: sha256:00ace44f9bc60221ab49cc4b2101f25d06657b9dc9b6c54ba6489b278a36ae2e
Deleted: sha256:0693fbf5155d16f83c68601c0eec72707f4dd62c6a5c95dc695c3732fe58ddfe
Deleted: sha256:d3fb5c1969a27379ad47c7ba1dd038e5cd99aaff4a73a4e7ce011f5654102ce6
Deleted: sha256:90ee2b715475d8d31392e671085f13e7c36ff4c7538a0e9651768e5c32611ffb
Deleted: sha256:03ee277a07cd025a21936a629389a83a8ff2ff68e5dcfd531673961d4b23e96c
Deleted: sha256:a83ca80aff119c02d1de27d14bc220a739f0f0fa19aa6f25b468e5d847a56c86
Deleted: sha256:d5c6e403906f542d4b7ad462d4cccd63917e5612223f57d2276aecd002894bd3
Deleted: sha256:3f19505a498a646b7d1d15e58984c1e7c0b3b6fde52083e303bdc701d754dab9
Deleted: sha256:aeeed2b4a1581948ac94bead7a2dc31129c1cdf5df4cb142a98e0256db3023eb
Deleted: sha256:89c77848d2b63b6111f5eff2cf81c79b136a851327484de38a704c5bf56376c4
Deleted: sha256:4e14f9fd4348fb8ebeb2dcd17ea6e2a3b18996717a0246a4acc0197d994fc908
Deleted: sha256:fe4650ab5ddacfc3f154e00efab3eecfa700adaf41f7c578e6c942a1ca411a99
Deleted: sha256:e65af55e3b3ee008f31b3bedcce5fdf077a8ac904ae5072ba28c93b7a079d4cb
Deleted: sha256:03c0f63d9d98017f31ced6b135d7f8f6bd995f1b262d81474692d716d5e6862c
Deleted: sha256:642977761a622ce0819e1ad2eba7438a70f165cbf6dbb68d06dbdd0ec6e9b1fa
Deleted: sha256:1099a3ca149d5cf61a79d6fd5fb97a6ebd55af680a9462ff70277a067f93a47c
Deleted: sha256:0ed7259d6bb12aa03171b77f0abb5babb7689b81bd68ff2bef28a07e8b0fa52b
Deleted: sha256:670aa8b914db5b5779af2e27d3b2b767d13e71f5e94d6ce0f83b75f8cbd325a1
Deleted: sha256:bc06fa176310809f50267375306769dc6e06103a412a904c0162d801f6442c84
Deleted: sha256:88d4212a8fef6fd7d7cc9b6b044e71085d3041f49756a79ca576ccc2e65bc088
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211015124339]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:37492b5808443f2d5949274093061710a47346c5d4c3f6de22a40f37b8862c95]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211015124339] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:37492b5808443f2d5949274093061710a47346c5d4c3f6de22a40f37b8862c95])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:37492b5808443f2d5949274093061710a47346c5d4c3f6de22a40f37b8862c95
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:37492b5808443f2d5949274093061710a47346c5d4c3f6de22a40f37b8862c95
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:37492b5808443f2d5949274093061710a47346c5d4c3f6de22a40f37b8862c95].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 40s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/ss7fflnnu6e7g

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #119

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/119/display/redirect?page=changes>

Changes:

[Udi Meiri] Release guide: minor updates

[dmitrii_kuzin] [BEAM-12730] Custom delimiter always bytes

[noreply] Minor: Add base_func to Series.aggregate expression names (#15706)

[noreply] [BEAM-13013] Extend cross language expansion. (#15698)

[noreply] Minor: Clarify confusing boolean logic for selecting PerWindowInvoker

[noreply] [BEAM-4424] Execute hooks in enabling order (#15713)

[noreply] [BEAM-11097] Update cache with metrics counting, windowing (#15717)

[kawaigin] Prepare release of apache-beam-jupyterlab-sidepanel v2.0.0

[noreply] [BEAM-12644] Javascript files moved to assets (#15653)


------------------------------------------
[...truncated 49.23 KB...]
fc92b19a474c: Preparing
a4201d4818e2: Preparing
37328c6b84db: Preparing
305bb936906e: Preparing
c1b02b399900: Preparing
fa030dac2f06: Preparing
f5ac1c8f92d2: Preparing
211736d7fc9d: Preparing
62a5b8741e83: Preparing
36e0782f1159: Preparing
ba6e5ff31f23: Preparing
9f9f651e9303: Preparing
0b3c02b5d746: Preparing
62a747bf1719: Preparing
a4201d4818e2: Waiting
305bb936906e: Waiting
211736d7fc9d: Waiting
c1b02b399900: Waiting
37328c6b84db: Waiting
62a5b8741e83: Waiting
fa030dac2f06: Waiting
f5ac1c8f92d2: Waiting
9f9f651e9303: Waiting
0b3c02b5d746: Waiting
ba6e5ff31f23: Waiting
62a747bf1719: Waiting
36e0782f1159: Waiting
b3998852e35b: Pushed
bfcfb0b3af60: Pushed
19115141af1a: Pushed
fc92b19a474c: Pushed
1a70880f1ea8: Pushed
2c38276686a8: Pushed
37328c6b84db: Pushed
305bb936906e: Pushed
211736d7fc9d: Layer already exists
fa030dac2f06: Pushed
62a5b8741e83: Layer already exists
36e0782f1159: Layer already exists
f5ac1c8f92d2: Pushed
a4201d4818e2: Pushed
ba6e5ff31f23: Layer already exists
0b3c02b5d746: Layer already exists
9f9f651e9303: Layer already exists
62a747bf1719: Layer already exists
c1b02b399900: Pushed
20211014124331: digest: sha256:0010396bb221b6eb520a8658a9ed24f2b07aa8fb9e7f2087001876e1ea010d23 size: 4311

> Task :sdks:java:testing:load-tests:run
Oct 14, 2021 12:45:16 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Oct 14, 2021 12:45:17 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 195 files. Enable logging at DEBUG level to see which files will be staged.
Oct 14, 2021 12:45:17 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Oct 14, 2021 12:45:17 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Oct 14, 2021 12:45:20 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 195 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Oct 14, 2021 12:45:20 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 195 files cached, 0 files newly uploaded in 0 seconds
Oct 14, 2021 12:45:20 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Oct 14, 2021 12:45:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <108949 bytes, hash 9950a4903cb02103c09974ea3d237e9d1a1625294d1944271b38f71744b1fc5d> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-mVCkkDywIQPAmXTqPSN-nRoWJSlNGUQnGzj3F0Sx_F0.pb
Oct 14, 2021 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Oct 14, 2021 12:45:22 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2a869a16, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@ae202c6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@46aa712c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6ada9c0c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7412ed6b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6e7c351d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b4a0aef, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@45cec376, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26d41711, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@298f0a0b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2b960a7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@31dfc6f5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@37b52340, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@663bb8ef, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2f4e40d7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@60e9c3a5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7e5843db, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@459f703f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@188ac8a3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3650d4fc]
Oct 14, 2021 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Oct 14, 2021 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Oct 14, 2021 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Oct 14, 2021 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Oct 14, 2021 12:45:22 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@441b8382, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1df1ced0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5349b246, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@32b0876c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2aaf152b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@39bbd9e0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@27fe9713, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@11c3ff67, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4397a639, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@24841372, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77114efe, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@79a7bfbc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77f68df, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3e4e4c1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7e7f3cfd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ae126d1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@46a488c2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6242ae3b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@65ddee5a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@56399b9e]
Oct 14, 2021 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Oct 14, 2021 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Oct 14, 2021 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Oct 14, 2021 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Oct 14, 2021 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Oct 14, 2021 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Oct 14, 2021 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Oct 14, 2021 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Oct 14, 2021 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Oct 14, 2021 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Oct 14, 2021 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Oct 14, 2021 12:45:22 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.35.0-SNAPSHOT
Oct 14, 2021 12:45:23 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-10-14_05_45_22-2948433723216611995?project=apache-beam-testing
Oct 14, 2021 12:45:23 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-10-14_05_45_22-2948433723216611995
Oct 14, 2021 12:45:23 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-10-14_05_45_22-2948433723216611995
Oct 14, 2021 12:45:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-10-14T12:45:28.808Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-10-xsnp. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Oct 14, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-14T12:45:44.096Z: Worker configuration: e2-standard-2 in us-central1-a.
Oct 14, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-14T12:45:44.767Z: Expanding SplittableParDo operations into optimizable parts.
Oct 14, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-14T12:45:44.790Z: Expanding CollectionToSingleton operations into optimizable parts.
Oct 14, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-14T12:45:44.848Z: Expanding CoGroupByKey operations into optimizable parts.
Oct 14, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-14T12:45:44.918Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Oct 14, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-14T12:45:44.947Z: Expanding GroupByKey operations into streaming Read/Write steps
Oct 14, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-14T12:45:45.012Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Oct 14, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-14T12:45:45.101Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Oct 14, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-14T12:45:45.135Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Oct 14, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-14T12:45:45.171Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Oct 14, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-14T12:45:45.197Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Oct 14, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-14T12:45:45.230Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Oct 14, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-14T12:45:45.255Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Oct 14, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-14T12:45:45.285Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Oct 14, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-14T12:45:45.310Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Oct 14, 2021 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-14T12:45:45.334Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Oct 14, 2021 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-14T12:45:45.375Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Oct 14, 2021 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-14T12:45:45.408Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Oct 14, 2021 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-14T12:45:45.444Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Oct 14, 2021 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-14T12:45:45.472Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Oct 14, 2021 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-14T12:45:45.493Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Oct 14, 2021 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-14T12:45:45.518Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Oct 14, 2021 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-14T12:45:45.563Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Oct 14, 2021 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-14T12:45:45.595Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Oct 14, 2021 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-14T12:45:45.629Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Oct 14, 2021 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-14T12:45:45.664Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Oct 14, 2021 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-14T12:45:45.698Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Oct 14, 2021 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-14T12:45:45.730Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Oct 14, 2021 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-14T12:45:45.775Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Oct 14, 2021 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-14T12:45:45.798Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Oct 14, 2021 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-14T12:45:46.179Z: Starting 5 ****s in us-central1-a...
Oct 14, 2021 12:45:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-14T12:45:59.019Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Oct 14, 2021 12:46:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-14T12:46:33.082Z: Autoscaling: Raised the number of ****s to 4 so that the pipeline can catch up with its backlog and keep up with its input rate.
Oct 14, 2021 12:46:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-14T12:46:33.109Z: Resized **** pool to 4, though goal was 5.  This could be a quota issue.
Oct 14, 2021 12:47:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-14T12:47:03.810Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Oct 14, 2021 12:47:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-14T12:47:31.701Z: Workers have started successfully.
Oct 14, 2021 12:47:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-14T12:47:31.737Z: Workers have started successfully.
Oct 14, 2021 1:41:30 PM org.apache.beam.sdk.metrics.MetricsEnvironment getCurrentContainer
WARNING: Reporting metrics are not supported in the current execution environment.
Oct 14, 2021 4:00:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-14T16:00:26.932Z: Cancel request is committed for workflow job: 2021-10-14_05_45_22-2948433723216611995.
Oct 14, 2021 4:00:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-14T16:00:26.980Z: Cleaning up.
Oct 14, 2021 4:00:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-14T16:00:27.063Z: Stopping **** pool...
Oct 14, 2021 4:00:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-14T16:00:27.109Z: Stopping **** pool...
Oct 14, 2021 4:02:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-14T16:02:44.617Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Oct 14, 2021 4:02:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-14T16:02:44.694Z: Worker pool stopped.
Oct 14, 2021 4:02:51 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-10-14_05_45_22-2948433723216611995 finished with status CANCELLED.
Load test results for test (ID): 9c9961b7-6df9-4864-bdf7-46431390b3bb and timestamp: 2021-10-14T12:45:17.325000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11493.503
dataflow_v2_java11_total_bytes_count              3.0893436E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211014124331
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:0010396bb221b6eb520a8658a9ed24f2b07aa8fb9e7f2087001876e1ea010d23
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211014124331]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:0010396bb221b6eb520a8658a9ed24f2b07aa8fb9e7f2087001876e1ea010d23]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211014124331] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:0010396bb221b6eb520a8658a9ed24f2b07aa8fb9e7f2087001876e1ea010d23])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:0010396bb221b6eb520a8658a9ed24f2b07aa8fb9e7f2087001876e1ea010d23
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:0010396bb221b6eb520a8658a9ed24f2b07aa8fb9e7f2087001876e1ea010d23
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:0010396bb221b6eb520a8658a9ed24f2b07aa8fb9e7f2087001876e1ea010d23].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 37s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/32peok6pgqug4

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #118

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/118/display/redirect?page=changes>

Changes:

[cgray] [BEAM-7169] Fix singleton assertion in PAssert javadoc.

[noreply] Revert "[BEAM-10913] - Forcing update of YAML file by running kubectl

[heejong] Update Dataflow Python container tag

[noreply] Merge pull request #15667 from [BEAM-12730] Add custom delimiters to

[noreply] Merge pull request #15700 from [BEAM-12987] [Playground] Add page UI

[kawaigin] [BEAM-12997] Upgraded labextension to V3

[noreply] [BEAM-12513] Add JIRAs for missing Go SDK features (#15658)


------------------------------------------
[...truncated 49.00 KB...]
48e502b38c40: Preparing
9fe49eb8f698: Preparing
21bba6fc7db0: Preparing
04e5cba339ad: Preparing
8bfabf5d869b: Preparing
020514861326: Preparing
365d9dfe3001: Preparing
2329b590aa4e: Preparing
527eb80015ed: Preparing
f505a9e8cbc6: Preparing
f04d52741e62: Preparing
341f0e6e941d: Preparing
211736d7fc9d: Preparing
62a5b8741e83: Preparing
36e0782f1159: Preparing
ba6e5ff31f23: Preparing
9f9f651e9303: Preparing
0b3c02b5d746: Preparing
62a747bf1719: Preparing
211736d7fc9d: Waiting
62a5b8741e83: Waiting
365d9dfe3001: Waiting
36e0782f1159: Waiting
9f9f651e9303: Waiting
ba6e5ff31f23: Waiting
0b3c02b5d746: Waiting
2329b590aa4e: Waiting
62a747bf1719: Waiting
527eb80015ed: Waiting
020514861326: Waiting
f505a9e8cbc6: Waiting
341f0e6e941d: Waiting
f04d52741e62: Waiting
21bba6fc7db0: Pushed
8bfabf5d869b: Pushed
9fe49eb8f698: Pushed
48e502b38c40: Pushed
020514861326: Pushed
04e5cba339ad: Pushed
2329b590aa4e: Pushed
527eb80015ed: Pushed
211736d7fc9d: Layer already exists
62a5b8741e83: Layer already exists
36e0782f1159: Layer already exists
365d9dfe3001: Pushed
9f9f651e9303: Layer already exists
ba6e5ff31f23: Layer already exists
341f0e6e941d: Pushed
0b3c02b5d746: Layer already exists
62a747bf1719: Layer already exists
f04d52741e62: Pushed
f505a9e8cbc6: Pushed
20211013124333: digest: sha256:40f67b11010e01baf38404e6a41e5f9346ec0a2575d61edf1310938d4ed891ad size: 4311

> Task :sdks:java:testing:load-tests:run
Oct 13, 2021 12:46:26 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Oct 13, 2021 12:46:26 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 195 files. Enable logging at DEBUG level to see which files will be staged.
Oct 13, 2021 12:46:27 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Oct 13, 2021 12:46:27 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Oct 13, 2021 12:46:30 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 195 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Oct 13, 2021 12:46:31 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 195 files cached, 0 files newly uploaded in 0 seconds
Oct 13, 2021 12:46:31 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Oct 13, 2021 12:46:31 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <108949 bytes, hash 8ecf4624414404a248a31cca3f8be5d470069840d823c09bb168c1eca9625d45> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-js9GJEFEBKJIoxzKP4vl1HAGmEDYI8CbsWjB7KliXUU.pb
Oct 13, 2021 12:46:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Oct 13, 2021 12:46:33 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@46aa712c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6ada9c0c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7412ed6b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6e7c351d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b4a0aef, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@45cec376, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26d41711, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@298f0a0b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2b960a7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@31dfc6f5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@37b52340, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@663bb8ef, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2f4e40d7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@60e9c3a5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7e5843db, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@459f703f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@188ac8a3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3650d4fc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@aaa0f76, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@340a8894]
Oct 13, 2021 12:46:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Oct 13, 2021 12:46:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Oct 13, 2021 12:46:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Oct 13, 2021 12:46:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Oct 13, 2021 12:46:33 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5349b246, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@32b0876c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2aaf152b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@39bbd9e0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@27fe9713, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@11c3ff67, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4397a639, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@24841372, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77114efe, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@79a7bfbc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77f68df, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3e4e4c1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7e7f3cfd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ae126d1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@46a488c2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6242ae3b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@65ddee5a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@56399b9e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@34b9eb03, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43fda8d9]
Oct 13, 2021 12:46:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Oct 13, 2021 12:46:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Oct 13, 2021 12:46:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Oct 13, 2021 12:46:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Oct 13, 2021 12:46:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Oct 13, 2021 12:46:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Oct 13, 2021 12:46:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Oct 13, 2021 12:46:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Oct 13, 2021 12:46:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Oct 13, 2021 12:46:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Oct 13, 2021 12:46:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Oct 13, 2021 12:46:33 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.35.0-SNAPSHOT
Oct 13, 2021 12:46:34 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-10-13_05_46_33-9020637892299962970?project=apache-beam-testing
Oct 13, 2021 12:46:34 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-10-13_05_46_33-9020637892299962970
Oct 13, 2021 12:46:34 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-10-13_05_46_33-9020637892299962970
Oct 13, 2021 12:46:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-10-13T12:46:49.979Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-10-yw0n. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Oct 13, 2021 12:46:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-13T12:46:53.964Z: Worker configuration: e2-standard-2 in us-central1-a.
Oct 13, 2021 12:46:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-13T12:46:54.661Z: Expanding SplittableParDo operations into optimizable parts.
Oct 13, 2021 12:46:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-13T12:46:54.695Z: Expanding CollectionToSingleton operations into optimizable parts.
Oct 13, 2021 12:46:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-13T12:46:54.753Z: Expanding CoGroupByKey operations into optimizable parts.
Oct 13, 2021 12:46:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-13T12:46:54.810Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Oct 13, 2021 12:46:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-13T12:46:54.840Z: Expanding GroupByKey operations into streaming Read/Write steps
Oct 13, 2021 12:46:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-13T12:46:54.902Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Oct 13, 2021 12:46:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-13T12:46:54.992Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Oct 13, 2021 12:46:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-13T12:46:55.030Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Oct 13, 2021 12:46:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-13T12:46:55.063Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Oct 13, 2021 12:46:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-13T12:46:55.095Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Oct 13, 2021 12:46:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-13T12:46:55.116Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Oct 13, 2021 12:46:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-13T12:46:55.150Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Oct 13, 2021 12:46:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-13T12:46:55.182Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Oct 13, 2021 12:46:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-13T12:46:55.216Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Oct 13, 2021 12:46:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-13T12:46:55.239Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Oct 13, 2021 12:46:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-13T12:46:55.268Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Oct 13, 2021 12:46:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-13T12:46:55.302Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Oct 13, 2021 12:46:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-13T12:46:55.348Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Oct 13, 2021 12:46:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-13T12:46:55.382Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Oct 13, 2021 12:46:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-13T12:46:55.414Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Oct 13, 2021 12:46:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-13T12:46:55.444Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Oct 13, 2021 12:46:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-13T12:46:55.477Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Oct 13, 2021 12:46:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-13T12:46:55.510Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Oct 13, 2021 12:46:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-13T12:46:55.533Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Oct 13, 2021 12:46:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-13T12:46:55.567Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Oct 13, 2021 12:46:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-13T12:46:55.600Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Oct 13, 2021 12:46:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-13T12:46:55.624Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Oct 13, 2021 12:46:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-13T12:46:55.658Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Oct 13, 2021 12:46:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-13T12:46:55.691Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Oct 13, 2021 12:46:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-13T12:46:56.061Z: Starting 5 ****s in us-central1-a...
Oct 13, 2021 12:47:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-13T12:47:23.543Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Oct 13, 2021 12:47:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-13T12:47:37.187Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Oct 13, 2021 12:48:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-13T12:48:24.993Z: Workers have started successfully.
Oct 13, 2021 12:48:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-13T12:48:25.029Z: Workers have started successfully.
Oct 13, 2021 4:00:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-13T16:00:38.767Z: Cancel request is committed for workflow job: 2021-10-13_05_46_33-9020637892299962970.
Oct 13, 2021 4:00:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-13T16:00:38.895Z: Cleaning up.
Oct 13, 2021 4:00:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-13T16:00:38.992Z: Stopping **** pool...
Oct 13, 2021 4:00:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-13T16:00:39.058Z: Stopping **** pool...
Oct 13, 2021 4:03:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-13T16:03:03.417Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Oct 13, 2021 4:03:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-13T16:03:03.462Z: Worker pool stopped.
Oct 13, 2021 4:03:11 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-10-13_05_46_33-9020637892299962970 finished with status CANCELLED.
Load test results for test (ID): 31b7261a-0453-43da-8705-e569b2005f40 and timestamp: 2021-10-13T12:46:27.241000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11449.456
dataflow_v2_java11_total_bytes_count             3.20521575E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211013124333
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:40f67b11010e01baf38404e6a41e5f9346ec0a2575d61edf1310938d4ed891ad
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211013124333]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:40f67b11010e01baf38404e6a41e5f9346ec0a2575d61edf1310938d4ed891ad]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211013124333] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:40f67b11010e01baf38404e6a41e5f9346ec0a2575d61edf1310938d4ed891ad])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:40f67b11010e01baf38404e6a41e5f9346ec0a2575d61edf1310938d4ed891ad
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:40f67b11010e01baf38404e6a41e5f9346ec0a2575d61edf1310938d4ed891ad
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:40f67b11010e01baf38404e6a41e5f9346ec0a2575d61edf1310938d4ed891ad].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 56s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/kaj6iehb6j4xq

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #117

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/117/display/redirect?page=changes>

Changes:

[rogan.o.morrow] [BEAM-12875] Register file systems in SparkExecutableStageFunction

[rogelio.hernandez] [BEAM-12371][BEAM-12373] Fixed page navigation

[Kyle Weaver] [BEAM-12694] Include datetime in dicom test dataset name.

[noreply] [BEAM-12960] Add extra context for structural DoFn ProcessElement error

[noreply] Release 2.33 documentation update (#15543)

[noreply] Merge pull request #15441 from [BEAM-8823] Make FnApiRunner work by


------------------------------------------
[...truncated 49.76 KB...]
d35dc7f4c79e: Waiting
d30f6a731c80: Waiting
3054497613e6: Waiting
1056964ed652: Waiting
c52aab93b4d8: Waiting
9e2d103a3ffa: Waiting
6aac433ceedb: Pushed
49b480f412bb: Pushed
0ddd9ba0266b: Pushed
455bc4cc0c24: Pushed
de3891f9f853: Pushed
edda9796484a: Pushed
9e2d103a3ffa: Pushed
1056964ed652: Pushed
874ad65f91ea: Layer already exists
0fc2498b65e5: Layer already exists
d08e6b97bf21: Layer already exists
3054497613e6: Layer already exists
d35dc7f4c79e: Layer already exists
dabfe5b2ea81: Layer already exists
5e6a409f30b6: Layer already exists
a98514a91652: Pushed
c52aab93b4d8: Pushed
d30f6a731c80: Pushed
86cde4e619ba: Pushed
20211012124337: digest: sha256:bffcaa579e742b49fc16a642742154b49af8008e7571cc861d3a56f171aaaee0 size: 4311

> Task :sdks:java:testing:load-tests:run
Oct 12, 2021 12:45:48 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Oct 12, 2021 12:45:48 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 195 files. Enable logging at DEBUG level to see which files will be staged.
Oct 12, 2021 12:45:49 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Oct 12, 2021 12:45:49 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Oct 12, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 195 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Oct 12, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 195 files cached, 0 files newly uploaded in 0 seconds
Oct 12, 2021 12:45:52 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Oct 12, 2021 12:45:52 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <108949 bytes, hash 3c766c7843f2aa70123fd97f34aa48ff61eceb34ca46ec3d5bf4eb1392fe52df> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-PHZseEPyqnASP9l_NKpI_2Hs6zTKRuw9W_TrE5L-Ut8.pb
Oct 12, 2021 12:45:53 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Oct 12, 2021 12:45:53 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@46aa712c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6ada9c0c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7412ed6b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6e7c351d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b4a0aef, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@45cec376, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26d41711, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@298f0a0b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2b960a7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@31dfc6f5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@37b52340, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@663bb8ef, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2f4e40d7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@60e9c3a5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7e5843db, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@459f703f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@188ac8a3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3650d4fc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@aaa0f76, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@340a8894]
Oct 12, 2021 12:45:53 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Oct 12, 2021 12:45:53 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Oct 12, 2021 12:45:53 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Oct 12, 2021 12:45:53 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Oct 12, 2021 12:45:53 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5349b246, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@32b0876c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2aaf152b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@39bbd9e0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@27fe9713, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@11c3ff67, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4397a639, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@24841372, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77114efe, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@79a7bfbc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77f68df, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3e4e4c1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7e7f3cfd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ae126d1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@46a488c2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6242ae3b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@65ddee5a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@56399b9e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@34b9eb03, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43fda8d9]
Oct 12, 2021 12:45:53 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Oct 12, 2021 12:45:53 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Oct 12, 2021 12:45:53 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Oct 12, 2021 12:45:53 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Oct 12, 2021 12:45:53 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Oct 12, 2021 12:45:53 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Oct 12, 2021 12:45:53 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Oct 12, 2021 12:45:53 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Oct 12, 2021 12:45:53 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Oct 12, 2021 12:45:53 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Oct 12, 2021 12:45:53 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Oct 12, 2021 12:45:54 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.35.0-SNAPSHOT
Oct 12, 2021 12:45:54 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-10-12_05_45_54-13152940544920821177?project=apache-beam-testing
Oct 12, 2021 12:45:54 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-10-12_05_45_54-13152940544920821177
Oct 12, 2021 12:45:54 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-10-12_05_45_54-13152940544920821177
Oct 12, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-10-12T12:45:59.780Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-10-5iov. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Oct 12, 2021 12:46:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-12T12:46:03.238Z: Worker configuration: e2-standard-2 in us-central1-a.
Oct 12, 2021 12:46:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-12T12:46:03.949Z: Expanding SplittableParDo operations into optimizable parts.
Oct 12, 2021 12:46:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-12T12:46:03.986Z: Expanding CollectionToSingleton operations into optimizable parts.
Oct 12, 2021 12:46:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-12T12:46:04.080Z: Expanding CoGroupByKey operations into optimizable parts.
Oct 12, 2021 12:46:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-12T12:46:04.145Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Oct 12, 2021 12:46:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-12T12:46:04.167Z: Expanding GroupByKey operations into streaming Read/Write steps
Oct 12, 2021 12:46:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-12T12:46:04.222Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Oct 12, 2021 12:46:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-12T12:46:04.325Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Oct 12, 2021 12:46:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-12T12:46:04.359Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Oct 12, 2021 12:46:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-12T12:46:04.383Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Oct 12, 2021 12:46:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-12T12:46:04.412Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Oct 12, 2021 12:46:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-12T12:46:04.437Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Oct 12, 2021 12:46:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-12T12:46:04.474Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Oct 12, 2021 12:46:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-12T12:46:04.497Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Oct 12, 2021 12:46:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-12T12:46:04.534Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Oct 12, 2021 12:46:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-12T12:46:04.572Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Oct 12, 2021 12:46:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-12T12:46:04.607Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Oct 12, 2021 12:46:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-12T12:46:04.633Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Oct 12, 2021 12:46:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-12T12:46:04.675Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Oct 12, 2021 12:46:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-12T12:46:04.707Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Oct 12, 2021 12:46:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-12T12:46:04.732Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Oct 12, 2021 12:46:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-12T12:46:04.754Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Oct 12, 2021 12:46:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-12T12:46:04.778Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Oct 12, 2021 12:46:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-12T12:46:04.805Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Oct 12, 2021 12:46:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-12T12:46:04.833Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Oct 12, 2021 12:46:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-12T12:46:04.867Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Oct 12, 2021 12:46:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-12T12:46:04.890Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Oct 12, 2021 12:46:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-12T12:46:04.924Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Oct 12, 2021 12:46:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-12T12:46:04.951Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Oct 12, 2021 12:46:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-12T12:46:04.973Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Oct 12, 2021 12:46:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-12T12:46:05.343Z: Starting 5 ****s in us-central1-a...
Oct 12, 2021 12:46:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-12T12:46:33.042Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Oct 12, 2021 12:46:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-12T12:46:42.512Z: Autoscaling: Raised the number of ****s to 3 so that the pipeline can catch up with its backlog and keep up with its input rate.
Oct 12, 2021 12:46:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-12T12:46:42.547Z: Resized **** pool to 3, though goal was 5.  This could be a quota issue.
Oct 12, 2021 12:46:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-12T12:46:52.851Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Oct 12, 2021 12:47:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-12T12:47:46.501Z: Workers have started successfully.
Oct 12, 2021 12:47:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-12T12:47:46.530Z: Workers have started successfully.
Oct 12, 2021 4:00:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-12T16:00:26.504Z: Cancel request is committed for workflow job: 2021-10-12_05_45_54-13152940544920821177.
Oct 12, 2021 4:00:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-12T16:00:26.571Z: Cleaning up.
Oct 12, 2021 4:00:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-12T16:00:26.645Z: Stopping **** pool...
Oct 12, 2021 4:00:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-12T16:00:26.701Z: Stopping **** pool...
Oct 12, 2021 4:02:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-12T16:02:46.645Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Oct 12, 2021 4:02:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-12T16:02:46.685Z: Worker pool stopped.
Oct 12, 2021 4:02:52 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-10-12_05_45_54-13152940544920821177 finished with status CANCELLED.
Load test results for test (ID): 0c98951e-4da4-46c4-b560-3e6e568486ff and timestamp: 2021-10-12T12:45:48.850000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11533.222
dataflow_v2_java11_total_bytes_count             2.52451079E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211012124337
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:bffcaa579e742b49fc16a642742154b49af8008e7571cc861d3a56f171aaaee0
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211012124337]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:bffcaa579e742b49fc16a642742154b49af8008e7571cc861d3a56f171aaaee0]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211012124337] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:bffcaa579e742b49fc16a642742154b49af8008e7571cc861d3a56f171aaaee0])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:bffcaa579e742b49fc16a642742154b49af8008e7571cc861d3a56f171aaaee0
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:bffcaa579e742b49fc16a642742154b49af8008e7571cc861d3a56f171aaaee0
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:bffcaa579e742b49fc16a642742154b49af8008e7571cc861d3a56f171aaaee0].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:45752f197dc2b623f8d2bfd3a4768e03f4f32a7cedfc9820a4a184bbf86542a7
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:45752f197dc2b623f8d2bfd3a4768e03f4f32a7cedfc9820a4a184bbf86542a7
ERROR: (gcloud.container.images.delete) Not found: response: {'status': '404', 'content-length': '168', 'x-xss-protection': '0', 'transfer-encoding': 'chunked', 'server': 'Docker Registry', '-content-encoding': 'gzip', 'docker-distribution-api-version': 'registry/2.0', 'cache-control': 'private', 'date': 'Tue, 12 Oct 2021 16:02:56 GMT', 'x-frame-options': 'SAMEORIGIN', 'content-type': 'application/json'}
Failed to compute blob liveness for manifest: 'sha256:45752f197dc2b623f8d2bfd3a4768e03f4f32a7cedfc9820a4a184bbf86542a7': None

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages FAILED

FAILURE: Build completed with 2 failures.

1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

2: Task failed with an exception.
-----------
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src/runners/google-cloud-dataflow-java/build.gradle'> line: 282

* What went wrong:
Execution failed for task ':runners:google-cloud-dataflow-java:cleanUpDockerImages'.
> Process 'command './scripts/cleanup_untagged_gcr_images.sh'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 36s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/lwhfh4bgae6cy

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #116

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/116/display/redirect>

Changes:


------------------------------------------
[...truncated 49.34 KB...]
f9b9b01b9d02: Preparing
3190cf4007eb: Preparing
c405272d6d12: Preparing
874ad65f91ea: Preparing
0fc2498b65e5: Preparing
d08e6b97bf21: Preparing
3054497613e6: Preparing
d35dc7f4c79e: Preparing
dabfe5b2ea81: Preparing
5e6a409f30b6: Preparing
c405272d6d12: Waiting
3054497613e6: Waiting
d35dc7f4c79e: Waiting
dabfe5b2ea81: Waiting
a1b58d8c8a0e: Waiting
dceee12a4edb: Waiting
f9b9b01b9d02: Waiting
7a742a74631e: Waiting
874ad65f91ea: Waiting
5e6a409f30b6: Waiting
a3014c377ac0: Waiting
0fc2498b65e5: Waiting
3190cf4007eb: Waiting
cebda604bcad: Pushed
001242b60b94: Pushed
4ccfeddcc41f: Pushed
a3014c377ac0: Pushed
b4346c49963e: Pushed
a1b58d8c8a0e: Pushed
ad430934355c: Pushed
7a742a74631e: Pushed
874ad65f91ea: Layer already exists
0fc2498b65e5: Layer already exists
d08e6b97bf21: Layer already exists
3190cf4007eb: Pushed
c405272d6d12: Pushed
3054497613e6: Layer already exists
d35dc7f4c79e: Layer already exists
dceee12a4edb: Pushed
5e6a409f30b6: Layer already exists
dabfe5b2ea81: Layer already exists
f9b9b01b9d02: Pushed
20211011124333: digest: sha256:58ec7115f0d0dc5d6c434e51effde90b875c3e160e65f28b2a217b9a4dbf7eb7 size: 4311

> Task :sdks:java:testing:load-tests:run
Oct 11, 2021 12:45:35 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Oct 11, 2021 12:45:36 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 195 files. Enable logging at DEBUG level to see which files will be staged.
Oct 11, 2021 12:45:36 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Oct 11, 2021 12:45:36 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Oct 11, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 195 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Oct 11, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 195 files cached, 0 files newly uploaded in 0 seconds
Oct 11, 2021 12:45:39 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Oct 11, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <108949 bytes, hash d964e1819c1ba92903d867b2e840ada497dac5244767451a89033a8f391fc3ec> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-2WThgZwbqSkD2Gey6ECtpJfaxSRHZ0UaiQM6jzkfw-w.pb
Oct 11, 2021 12:45:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Oct 11, 2021 12:45:41 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2a869a16, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@ae202c6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@46aa712c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6ada9c0c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7412ed6b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6e7c351d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b4a0aef, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@45cec376, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26d41711, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@298f0a0b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2b960a7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@31dfc6f5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@37b52340, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@663bb8ef, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2f4e40d7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@60e9c3a5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7e5843db, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@459f703f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@188ac8a3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3650d4fc]
Oct 11, 2021 12:45:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Oct 11, 2021 12:45:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Oct 11, 2021 12:45:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Oct 11, 2021 12:45:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Oct 11, 2021 12:45:41 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@441b8382, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1df1ced0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5349b246, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@32b0876c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2aaf152b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@39bbd9e0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@27fe9713, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@11c3ff67, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4397a639, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@24841372, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77114efe, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@79a7bfbc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77f68df, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3e4e4c1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7e7f3cfd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ae126d1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@46a488c2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6242ae3b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@65ddee5a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@56399b9e]
Oct 11, 2021 12:45:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Oct 11, 2021 12:45:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Oct 11, 2021 12:45:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Oct 11, 2021 12:45:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Oct 11, 2021 12:45:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Oct 11, 2021 12:45:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Oct 11, 2021 12:45:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Oct 11, 2021 12:45:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Oct 11, 2021 12:45:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Oct 11, 2021 12:45:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Oct 11, 2021 12:45:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Oct 11, 2021 12:45:41 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.35.0-SNAPSHOT
Oct 11, 2021 12:45:42 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-10-11_05_45_41-7093437645724640072?project=apache-beam-testing
Oct 11, 2021 12:45:42 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-10-11_05_45_41-7093437645724640072
Oct 11, 2021 12:45:42 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-10-11_05_45_41-7093437645724640072
Oct 11, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-10-11T12:45:47.694Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-10-edm4. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Oct 11, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-11T12:45:51.856Z: Worker configuration: e2-standard-2 in us-central1-a.
Oct 11, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-11T12:45:52.550Z: Expanding SplittableParDo operations into optimizable parts.
Oct 11, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-11T12:45:52.597Z: Expanding CollectionToSingleton operations into optimizable parts.
Oct 11, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-11T12:45:52.667Z: Expanding CoGroupByKey operations into optimizable parts.
Oct 11, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-11T12:45:52.737Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Oct 11, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-11T12:45:52.757Z: Expanding GroupByKey operations into streaming Read/Write steps
Oct 11, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-11T12:45:52.814Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Oct 11, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-11T12:45:52.893Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Oct 11, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-11T12:45:52.928Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Oct 11, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-11T12:45:52.960Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Oct 11, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-11T12:45:52.982Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Oct 11, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-11T12:45:53.006Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Oct 11, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-11T12:45:53.038Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Oct 11, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-11T12:45:53.061Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Oct 11, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-11T12:45:53.094Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Oct 11, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-11T12:45:53.127Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Oct 11, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-11T12:45:53.160Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Oct 11, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-11T12:45:53.195Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Oct 11, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-11T12:45:53.228Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Oct 11, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-11T12:45:53.259Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Oct 11, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-11T12:45:53.297Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Oct 11, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-11T12:45:53.324Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Oct 11, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-11T12:45:53.344Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Oct 11, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-11T12:45:53.369Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Oct 11, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-11T12:45:53.402Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Oct 11, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-11T12:45:53.435Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Oct 11, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-11T12:45:53.458Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Oct 11, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-11T12:45:53.490Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Oct 11, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-11T12:45:53.523Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Oct 11, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-11T12:45:53.549Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Oct 11, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-11T12:45:53.891Z: Starting 5 ****s in us-central1-a...
Oct 11, 2021 12:46:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-11T12:46:11.481Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Oct 11, 2021 12:46:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-11T12:46:24.584Z: Autoscaling: Raised the number of ****s to 1 so that the pipeline can catch up with its backlog and keep up with its input rate.
Oct 11, 2021 12:46:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-11T12:46:24.620Z: Resized **** pool to 1, though goal was 5.  This could be a quota issue.
Oct 11, 2021 12:46:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-11T12:46:34.857Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Oct 11, 2021 12:47:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-11T12:47:27.903Z: Workers have started successfully.
Oct 11, 2021 12:47:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-11T12:47:27.933Z: Workers have started successfully.
Oct 11, 2021 3:00:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-11T15:00:56.753Z: Staged package kafka-clients-2.4.1-or4Vrm01S3aXE64MuzGE8iWqFvyYIFseskr9n8KjYNk.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/kafka-clients-2.4.1-or4Vrm01S3aXE64MuzGE8iWqFvyYIFseskr9n8KjYNk.jar' is inaccessible.
Oct 11, 2021 3:00:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-11T15:00:56.918Z: Staged package lz4-java-1.6.0-0ilUWqKx1SA8h2YUvbz_yswwNpf0-PJvdk4dbB7S5BY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/lz4-java-1.6.0-0ilUWqKx1SA8h2YUvbz_yswwNpf0-PJvdk4dbB7S5BY.jar' is inaccessible.
Oct 11, 2021 3:00:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-10-11T15:00:57.764Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Oct 11, 2021 4:00:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-11T16:00:30.452Z: Cancel request is committed for workflow job: 2021-10-11_05_45_41-7093437645724640072.
Oct 11, 2021 4:00:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-11T16:00:30.492Z: Cleaning up.
Oct 11, 2021 4:00:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-11T16:00:30.584Z: Stopping **** pool...
Oct 11, 2021 4:00:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-11T16:00:30.647Z: Stopping **** pool...
Oct 11, 2021 4:02:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-11T16:02:52.031Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Oct 11, 2021 4:02:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-11T16:02:52.060Z: Worker pool stopped.
Oct 11, 2021 4:02:59 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-10-11_05_45_41-7093437645724640072 finished with status CANCELLED.
Load test results for test (ID): 19df744a-310b-41da-94ae-ba4b3de0b85a and timestamp: 2021-10-11T12:45:36.598000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11534.295
dataflow_v2_java11_total_bytes_count             2.80176771E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211011124333
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:58ec7115f0d0dc5d6c434e51effde90b875c3e160e65f28b2a217b9a4dbf7eb7
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211011124333]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:58ec7115f0d0dc5d6c434e51effde90b875c3e160e65f28b2a217b9a4dbf7eb7]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211011124333] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:58ec7115f0d0dc5d6c434e51effde90b875c3e160e65f28b2a217b9a4dbf7eb7])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:58ec7115f0d0dc5d6c434e51effde90b875c3e160e65f28b2a217b9a4dbf7eb7
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:58ec7115f0d0dc5d6c434e51effde90b875c3e160e65f28b2a217b9a4dbf7eb7
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:58ec7115f0d0dc5d6c434e51effde90b875c3e160e65f28b2a217b9a4dbf7eb7].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 43s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/nygtfqxxlkqco

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #115

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/115/display/redirect>

Changes:


------------------------------------------
[...truncated 49.05 KB...]
fdccd057115e: Preparing
a1dfa02f6031: Preparing
26c28ed988c8: Preparing
c702fe3df9da: Preparing
c44e21fb849a: Preparing
592385105b40: Preparing
155ac4444f11: Preparing
da53b36b3009: Preparing
6f14d5c08ef2: Preparing
5d4f43e30d09: Preparing
0efe004fdc66: Preparing
b6fc84e1bca0: Preparing
874ad65f91ea: Preparing
0fc2498b65e5: Preparing
d08e6b97bf21: Preparing
3054497613e6: Preparing
d35dc7f4c79e: Preparing
dabfe5b2ea81: Preparing
5e6a409f30b6: Preparing
b6fc84e1bca0: Waiting
874ad65f91ea: Waiting
0fc2498b65e5: Waiting
592385105b40: Waiting
d08e6b97bf21: Waiting
155ac4444f11: Waiting
3054497613e6: Waiting
da53b36b3009: Waiting
d35dc7f4c79e: Waiting
6f14d5c08ef2: Waiting
dabfe5b2ea81: Waiting
5d4f43e30d09: Waiting
5e6a409f30b6: Waiting
0efe004fdc66: Waiting
c44e21fb849a: Pushed
a1dfa02f6031: Pushed
26c28ed988c8: Pushed
592385105b40: Pushed
fdccd057115e: Pushed
da53b36b3009: Pushed
c702fe3df9da: Pushed
6f14d5c08ef2: Pushed
874ad65f91ea: Layer already exists
0fc2498b65e5: Layer already exists
d08e6b97bf21: Layer already exists
3054497613e6: Layer already exists
d35dc7f4c79e: Layer already exists
dabfe5b2ea81: Layer already exists
5e6a409f30b6: Layer already exists
0efe004fdc66: Pushed
b6fc84e1bca0: Pushed
155ac4444f11: Pushed
5d4f43e30d09: Pushed
20211010124345: digest: sha256:2923c19b9c9fcb013a5ff0e6a011fca8c844b6510756f03a55b5c8979fa8172d size: 4311

> Task :sdks:java:testing:load-tests:run
Oct 10, 2021 12:46:26 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Oct 10, 2021 12:46:26 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 195 files. Enable logging at DEBUG level to see which files will be staged.
Oct 10, 2021 12:46:27 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Oct 10, 2021 12:46:27 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Oct 10, 2021 12:46:30 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 195 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Oct 10, 2021 12:46:31 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 195 files cached, 0 files newly uploaded in 0 seconds
Oct 10, 2021 12:46:31 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Oct 10, 2021 12:46:31 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <108949 bytes, hash 2eb408e091d7f1c72e90669c412d5fc95bc62023f5a989a2a15536a4a8f13ca0> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-LrQI4JHX8ccukGacQS1fyVvGICP1qYmioVU2pKjxPKA.pb
Oct 10, 2021 12:46:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Oct 10, 2021 12:46:33 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2a869a16, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@ae202c6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@46aa712c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6ada9c0c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7412ed6b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6e7c351d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b4a0aef, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@45cec376, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26d41711, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@298f0a0b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2b960a7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@31dfc6f5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@37b52340, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@663bb8ef, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2f4e40d7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@60e9c3a5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7e5843db, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@459f703f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@188ac8a3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3650d4fc]
Oct 10, 2021 12:46:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Oct 10, 2021 12:46:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Oct 10, 2021 12:46:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Oct 10, 2021 12:46:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Oct 10, 2021 12:46:33 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@441b8382, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1df1ced0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5349b246, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@32b0876c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2aaf152b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@39bbd9e0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@27fe9713, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@11c3ff67, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4397a639, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@24841372, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77114efe, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@79a7bfbc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77f68df, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3e4e4c1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7e7f3cfd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ae126d1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@46a488c2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6242ae3b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@65ddee5a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@56399b9e]
Oct 10, 2021 12:46:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Oct 10, 2021 12:46:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Oct 10, 2021 12:46:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Oct 10, 2021 12:46:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Oct 10, 2021 12:46:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Oct 10, 2021 12:46:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Oct 10, 2021 12:46:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Oct 10, 2021 12:46:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Oct 10, 2021 12:46:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Oct 10, 2021 12:46:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Oct 10, 2021 12:46:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Oct 10, 2021 12:46:33 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.35.0-SNAPSHOT
Oct 10, 2021 12:46:34 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-10-10_05_46_33-8151387498521069836?project=apache-beam-testing
Oct 10, 2021 12:46:34 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-10-10_05_46_33-8151387498521069836
Oct 10, 2021 12:46:34 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-10-10_05_46_33-8151387498521069836
Oct 10, 2021 12:46:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-10-10T12:46:40.829Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-10-ldct. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Oct 10, 2021 12:46:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-10T12:46:45.091Z: Worker configuration: e2-standard-2 in us-central1-a.
Oct 10, 2021 12:46:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-10T12:46:45.823Z: Expanding SplittableParDo operations into optimizable parts.
Oct 10, 2021 12:46:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-10T12:46:45.851Z: Expanding CollectionToSingleton operations into optimizable parts.
Oct 10, 2021 12:46:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-10T12:46:45.920Z: Expanding CoGroupByKey operations into optimizable parts.
Oct 10, 2021 12:46:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-10T12:46:45.979Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Oct 10, 2021 12:46:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-10T12:46:46.014Z: Expanding GroupByKey operations into streaming Read/Write steps
Oct 10, 2021 12:46:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-10T12:46:46.085Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Oct 10, 2021 12:46:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-10T12:46:46.210Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Oct 10, 2021 12:46:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-10T12:46:46.242Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Oct 10, 2021 12:46:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-10T12:46:46.299Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Oct 10, 2021 12:46:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-10T12:46:46.333Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Oct 10, 2021 12:46:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-10T12:46:46.377Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Oct 10, 2021 12:46:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-10T12:46:46.413Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Oct 10, 2021 12:46:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-10T12:46:46.449Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Oct 10, 2021 12:46:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-10T12:46:46.481Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Oct 10, 2021 12:46:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-10T12:46:46.522Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Oct 10, 2021 12:46:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-10T12:46:46.546Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Oct 10, 2021 12:46:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-10T12:46:46.575Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Oct 10, 2021 12:46:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-10T12:46:46.613Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Oct 10, 2021 12:46:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-10T12:46:46.638Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Oct 10, 2021 12:46:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-10T12:46:46.679Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Oct 10, 2021 12:46:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-10T12:46:46.712Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Oct 10, 2021 12:46:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-10T12:46:46.742Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Oct 10, 2021 12:46:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-10T12:46:46.779Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Oct 10, 2021 12:46:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-10T12:46:46.810Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Oct 10, 2021 12:46:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-10T12:46:46.844Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Oct 10, 2021 12:46:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-10T12:46:46.881Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Oct 10, 2021 12:46:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-10T12:46:46.915Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Oct 10, 2021 12:46:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-10T12:46:46.939Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Oct 10, 2021 12:46:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-10T12:46:46.971Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Oct 10, 2021 12:46:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-10T12:46:47.330Z: Starting 5 ****s in us-central1-a...
Oct 10, 2021 12:47:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-10T12:47:20.296Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Oct 10, 2021 12:47:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-10T12:47:32.073Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Oct 10, 2021 12:48:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-10T12:48:26.139Z: Workers have started successfully.
Oct 10, 2021 12:48:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-10T12:48:26.163Z: Workers have started successfully.
Oct 10, 2021 4:00:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-10T16:00:27.990Z: Cancel request is committed for workflow job: 2021-10-10_05_46_33-8151387498521069836.
Oct 10, 2021 4:00:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-10T16:00:28.078Z: Cleaning up.
Oct 10, 2021 4:00:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-10T16:00:28.205Z: Stopping **** pool...
Oct 10, 2021 4:00:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-10T16:00:28.275Z: Stopping **** pool...
Oct 10, 2021 4:02:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-10T16:02:46.730Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Oct 10, 2021 4:02:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-10T16:02:46.767Z: Worker pool stopped.
Oct 10, 2021 4:02:53 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-10-10_05_46_33-8151387498521069836 finished with status CANCELLED.
Load test results for test (ID): a8ed30b6-9ca7-4597-90ac-99aeb97b7bd0 and timestamp: 2021-10-10T12:46:27.062000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11479.373
dataflow_v2_java11_total_bytes_count             2.61106055E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211010124345
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:2923c19b9c9fcb013a5ff0e6a011fca8c844b6510756f03a55b5c8979fa8172d
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211010124345]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:2923c19b9c9fcb013a5ff0e6a011fca8c844b6510756f03a55b5c8979fa8172d]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211010124345] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:2923c19b9c9fcb013a5ff0e6a011fca8c844b6510756f03a55b5c8979fa8172d])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:2923c19b9c9fcb013a5ff0e6a011fca8c844b6510756f03a55b5c8979fa8172d
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:2923c19b9c9fcb013a5ff0e6a011fca8c844b6510756f03a55b5c8979fa8172d
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:2923c19b9c9fcb013a5ff0e6a011fca8c844b6510756f03a55b5c8979fa8172d].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 31s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/jhy47c5oha5ow

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #114

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/114/display/redirect?page=changes>

Changes:

[noreply] fixing test link

[Brian Hulette] Add DataFrame changes to CHANGES.md

[stranniknm] [BEAM-12964]: add code editor component

[stranniknm] [BEAM-12966]: load initial example on page load

[stranniknm] [BEAM-12964]: fix editor scrolling

[stranniknm] [BEAM-12964]: add dark theme support

[stranniknm] [BEAM-12964]: add licence to example repository file

[stranniknm] [BEAM-12964]: extract playground page providers

[stranniknm] [BEAM-12964]: refactor styles

[noreply] fix linter issues

[Brian Hulette] Don't use run_pytest.sh for pyarrow tests

[Brian Hulette] Fail in run_pytest.sh if -m is specified

[noreply] [BEAM-10913] - Forcing update of YAML file by running kubectl apply

[noreply] [BEAM-10955] Flink Java Runner test flake: Could not find Flink job

[noreply] [BEAM-10114] Bump Pub/Sub Lite version (#15640)


------------------------------------------
[...truncated 49.40 KB...]
dee979226441: Preparing
1916c76e2494: Preparing
efa2acb5fb41: Preparing
2c95d17f6062: Preparing
42561a8a8281: Preparing
d73fc204f59e: Preparing
3f892ce166a8: Preparing
9e2f628bd2c2: Preparing
9b4008169b79: Preparing
4e60e3c97e6a: Preparing
73a376432bd2: Preparing
6008dee27547: Preparing
874ad65f91ea: Preparing
0fc2498b65e5: Preparing
d08e6b97bf21: Preparing
3054497613e6: Preparing
d35dc7f4c79e: Preparing
dabfe5b2ea81: Preparing
5e6a409f30b6: Preparing
3f892ce166a8: Waiting
9b4008169b79: Waiting
d73fc204f59e: Waiting
874ad65f91ea: Waiting
5e6a409f30b6: Waiting
73a376432bd2: Waiting
3054497613e6: Waiting
9e2f628bd2c2: Waiting
d08e6b97bf21: Waiting
6008dee27547: Waiting
4e60e3c97e6a: Waiting
d35dc7f4c79e: Waiting
1916c76e2494: Pushed
42561a8a8281: Pushed
efa2acb5fb41: Pushed
dee979226441: Pushed
d73fc204f59e: Pushed
2c95d17f6062: Pushed
9e2f628bd2c2: Pushed
9b4008169b79: Pushed
874ad65f91ea: Layer already exists
0fc2498b65e5: Layer already exists
d08e6b97bf21: Layer already exists
3054497613e6: Layer already exists
d35dc7f4c79e: Layer already exists
73a376432bd2: Pushed
3f892ce166a8: Pushed
dabfe5b2ea81: Layer already exists
5e6a409f30b6: Layer already exists
6008dee27547: Pushed
4e60e3c97e6a: Pushed
20211009124332: digest: sha256:2f81eade5b657d80bb1f34c9933206fff45543a424d81dd97f52667de5b350d4 size: 4311

> Task :sdks:java:testing:load-tests:run
Oct 09, 2021 12:45:35 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Oct 09, 2021 12:45:35 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 195 files. Enable logging at DEBUG level to see which files will be staged.
Oct 09, 2021 12:45:36 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Oct 09, 2021 12:45:36 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Oct 09, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 195 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Oct 09, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 195 files cached, 0 files newly uploaded in 0 seconds
Oct 09, 2021 12:45:39 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Oct 09, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <108949 bytes, hash e399f8c535506db5c907b1ed5dc6d661cc648e01c91ff19338c5963e13bca986> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-45n4xTVQbbXJB7HtXcbWYcxkjgHJH_GTOMWWPhO8qYY.pb
Oct 09, 2021 12:45:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Oct 09, 2021 12:45:41 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2a869a16, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@ae202c6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@46aa712c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6ada9c0c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7412ed6b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6e7c351d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b4a0aef, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@45cec376, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26d41711, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@298f0a0b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2b960a7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@31dfc6f5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@37b52340, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@663bb8ef, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2f4e40d7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@60e9c3a5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7e5843db, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@459f703f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@188ac8a3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3650d4fc]
Oct 09, 2021 12:45:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Oct 09, 2021 12:45:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Oct 09, 2021 12:45:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Oct 09, 2021 12:45:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Oct 09, 2021 12:45:41 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@441b8382, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1df1ced0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5349b246, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@32b0876c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2aaf152b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@39bbd9e0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@27fe9713, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@11c3ff67, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4397a639, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@24841372, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77114efe, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@79a7bfbc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77f68df, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3e4e4c1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7e7f3cfd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ae126d1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@46a488c2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6242ae3b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@65ddee5a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@56399b9e]
Oct 09, 2021 12:45:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Oct 09, 2021 12:45:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Oct 09, 2021 12:45:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Oct 09, 2021 12:45:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Oct 09, 2021 12:45:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Oct 09, 2021 12:45:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Oct 09, 2021 12:45:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Oct 09, 2021 12:45:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Oct 09, 2021 12:45:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Oct 09, 2021 12:45:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Oct 09, 2021 12:45:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Oct 09, 2021 12:45:41 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.35.0-SNAPSHOT
Oct 09, 2021 12:45:42 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-10-09_05_45_41-5228465986545050931?project=apache-beam-testing
Oct 09, 2021 12:45:42 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-10-09_05_45_41-5228465986545050931
Oct 09, 2021 12:45:42 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-10-09_05_45_41-5228465986545050931
Oct 09, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-10-09T12:45:47.493Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-10-g77q. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Oct 09, 2021 12:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-09T12:45:51.663Z: Worker configuration: e2-standard-2 in us-central1-a.
Oct 09, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-09T12:45:52.405Z: Expanding SplittableParDo operations into optimizable parts.
Oct 09, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-09T12:45:52.456Z: Expanding CollectionToSingleton operations into optimizable parts.
Oct 09, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-09T12:45:52.520Z: Expanding CoGroupByKey operations into optimizable parts.
Oct 09, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-09T12:45:52.587Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Oct 09, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-09T12:45:52.627Z: Expanding GroupByKey operations into streaming Read/Write steps
Oct 09, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-09T12:45:52.689Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Oct 09, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-09T12:45:52.808Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Oct 09, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-09T12:45:52.836Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Oct 09, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-09T12:45:52.869Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Oct 09, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-09T12:45:52.890Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Oct 09, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-09T12:45:52.922Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Oct 09, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-09T12:45:52.958Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Oct 09, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-09T12:45:52.984Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Oct 09, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-09T12:45:53.012Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Oct 09, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-09T12:45:53.040Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Oct 09, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-09T12:45:53.072Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Oct 09, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-09T12:45:53.119Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Oct 09, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-09T12:45:53.152Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Oct 09, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-09T12:45:53.185Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Oct 09, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-09T12:45:53.208Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Oct 09, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-09T12:45:53.243Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Oct 09, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-09T12:45:53.272Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Oct 09, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-09T12:45:53.299Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Oct 09, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-09T12:45:53.331Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Oct 09, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-09T12:45:53.356Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Oct 09, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-09T12:45:53.382Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Oct 09, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-09T12:45:53.405Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Oct 09, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-09T12:45:53.462Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Oct 09, 2021 12:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-09T12:45:53.498Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Oct 09, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-09T12:45:53.828Z: Starting 5 ****s in us-central1-a...
Oct 09, 2021 12:46:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-09T12:46:14.986Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Oct 09, 2021 12:46:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-09T12:46:34.769Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Oct 09, 2021 12:47:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-09T12:47:32.218Z: Workers have started successfully.
Oct 09, 2021 12:47:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-09T12:47:32.253Z: Workers have started successfully.
Oct 09, 2021 1:01:05 PM org.apache.beam.sdk.metrics.MetricsEnvironment getCurrentContainer
WARNING: Reporting metrics are not supported in the current execution environment.
Oct 09, 2021 4:00:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-09T16:00:25.436Z: Cancel request is committed for workflow job: 2021-10-09_05_45_41-5228465986545050931.
Oct 09, 2021 4:00:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-09T16:00:25.470Z: Cleaning up.
Oct 09, 2021 4:00:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-09T16:00:25.541Z: Stopping **** pool...
Oct 09, 2021 4:00:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-09T16:00:25.607Z: Stopping **** pool...
Oct 09, 2021 4:02:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-09T16:02:46.299Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Oct 09, 2021 4:02:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-09T16:02:46.347Z: Worker pool stopped.
Oct 09, 2021 4:02:52 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-10-09_05_45_41-5228465986545050931 finished with status CANCELLED.
Load test results for test (ID): 6d47d85d-014b-4bb2-a548-347584ac4163 and timestamp: 2021-10-09T12:45:36.194000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11519.927
dataflow_v2_java11_total_bytes_count             2.47245451E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211009124332
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:2f81eade5b657d80bb1f34c9933206fff45543a424d81dd97f52667de5b350d4
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211009124332]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:2f81eade5b657d80bb1f34c9933206fff45543a424d81dd97f52667de5b350d4]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211009124332] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:2f81eade5b657d80bb1f34c9933206fff45543a424d81dd97f52667de5b350d4])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:2f81eade5b657d80bb1f34c9933206fff45543a424d81dd97f52667de5b350d4
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:2f81eade5b657d80bb1f34c9933206fff45543a424d81dd97f52667de5b350d4
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:2f81eade5b657d80bb1f34c9933206fff45543a424d81dd97f52667de5b350d4].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 38s
101 actionable tasks: 72 executed, 27 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/vakcw2lsbpcw4

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #113

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/113/display/redirect?page=changes>

Changes:

[Udi Meiri] Release script fixes

[noreply] [BEAM-13015] Remove the overhead of SpecMonitoringInfoValidator

[noreply] Minor: Replace generic external.py links in multi-language documentation

[noreply] Revert "[BEAM-12993] Update to Debezium 1.7.0.Final (#15636)"

[kawaigin] Updated screendiff integration test golden screenshots.

[noreply] [BEAM-12769] Few fixes related to Java Class Lookup based cross-language


------------------------------------------
[...truncated 49.01 KB...]
The push refers to repository [us.gcr.io/apache-beam-testing/java-postcommit-it/java]
cfb0e330e2cd: Preparing
350a52240e0a: Preparing
00e9ae2e2d18: Preparing
df97d3495d61: Preparing
62a494936b66: Preparing
89ac3b3d8563: Preparing
9aa27e885a7a: Preparing
99924550047b: Preparing
b40a81c38a7c: Preparing
337ad35acb76: Preparing
e885f8d0ca73: Preparing
0269687aeff1: Preparing
874ad65f91ea: Preparing
0fc2498b65e5: Preparing
d08e6b97bf21: Preparing
89ac3b3d8563: Waiting
3054497613e6: Preparing
d35dc7f4c79e: Preparing
dabfe5b2ea81: Preparing
5e6a409f30b6: Preparing
3054497613e6: Waiting
e885f8d0ca73: Waiting
99924550047b: Waiting
dabfe5b2ea81: Waiting
5e6a409f30b6: Waiting
0269687aeff1: Waiting
9aa27e885a7a: Waiting
d08e6b97bf21: Waiting
874ad65f91ea: Waiting
d35dc7f4c79e: Waiting
350a52240e0a: Pushed
62a494936b66: Pushed
00e9ae2e2d18: Pushed
89ac3b3d8563: Pushed
cfb0e330e2cd: Pushed
df97d3495d61: Pushed
99924550047b: Pushed
b40a81c38a7c: Pushed
874ad65f91ea: Layer already exists
0fc2498b65e5: Layer already exists
d08e6b97bf21: Layer already exists
3054497613e6: Layer already exists
e885f8d0ca73: Pushed
d35dc7f4c79e: Layer already exists
0269687aeff1: Pushed
9aa27e885a7a: Pushed
5e6a409f30b6: Layer already exists
dabfe5b2ea81: Layer already exists
337ad35acb76: Pushed
20211008124332: digest: sha256:1447960b06deec03f1c40156ae68f611464231872252e290a78a8ca8b5fbc7c4 size: 4311

> Task :sdks:java:testing:load-tests:run
Oct 08, 2021 12:45:24 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Oct 08, 2021 12:45:24 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 195 files. Enable logging at DEBUG level to see which files will be staged.
Oct 08, 2021 12:45:25 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Oct 08, 2021 12:45:25 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Oct 08, 2021 12:45:27 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 195 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Oct 08, 2021 12:45:27 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 195 files cached, 0 files newly uploaded in 0 seconds
Oct 08, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Oct 08, 2021 12:45:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <108949 bytes, hash 21965ab3c94e3331365372cf25a920dffaf948608673c1d0b0ecb2f42428204e> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-IZZas8lOMzE2U3LPJakg3_r5SGCGc8HQsOyy9CQoIE4.pb
Oct 08, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Oct 08, 2021 12:45:29 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2a869a16, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@ae202c6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@46aa712c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6ada9c0c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7412ed6b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6e7c351d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b4a0aef, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@45cec376, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26d41711, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@298f0a0b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2b960a7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@31dfc6f5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@37b52340, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@663bb8ef, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2f4e40d7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@60e9c3a5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7e5843db, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@459f703f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@188ac8a3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3650d4fc]
Oct 08, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Oct 08, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Oct 08, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Oct 08, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Oct 08, 2021 12:45:29 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@441b8382, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1df1ced0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5349b246, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@32b0876c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2aaf152b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@39bbd9e0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@27fe9713, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@11c3ff67, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4397a639, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@24841372, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77114efe, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@79a7bfbc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77f68df, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3e4e4c1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7e7f3cfd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ae126d1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@46a488c2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6242ae3b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@65ddee5a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@56399b9e]
Oct 08, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Oct 08, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Oct 08, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Oct 08, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Oct 08, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Oct 08, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Oct 08, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Oct 08, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Oct 08, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Oct 08, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Oct 08, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Oct 08, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.35.0-SNAPSHOT
Oct 08, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-10-08_05_45_29-12309857362774759606?project=apache-beam-testing
Oct 08, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-10-08_05_45_29-12309857362774759606
Oct 08, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-10-08_05_45_29-12309857362774759606
Oct 08, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-10-08T12:45:36.429Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-10-krt3. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Oct 08, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-08T12:45:40.659Z: Worker configuration: e2-standard-2 in us-central1-a.
Oct 08, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-08T12:45:41.664Z: Expanding SplittableParDo operations into optimizable parts.
Oct 08, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-08T12:45:41.695Z: Expanding CollectionToSingleton operations into optimizable parts.
Oct 08, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-08T12:45:41.761Z: Expanding CoGroupByKey operations into optimizable parts.
Oct 08, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-08T12:45:41.820Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Oct 08, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-08T12:45:41.849Z: Expanding GroupByKey operations into streaming Read/Write steps
Oct 08, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-08T12:45:41.913Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Oct 08, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-08T12:45:42.012Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Oct 08, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-08T12:45:42.048Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Oct 08, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-08T12:45:42.082Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Oct 08, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-08T12:45:42.115Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Oct 08, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-08T12:45:42.151Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Oct 08, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-08T12:45:42.185Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Oct 08, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-08T12:45:42.227Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Oct 08, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-08T12:45:42.255Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Oct 08, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-08T12:45:42.289Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Oct 08, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-08T12:45:42.318Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Oct 08, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-08T12:45:42.345Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Oct 08, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-08T12:45:42.376Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Oct 08, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-08T12:45:42.422Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Oct 08, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-08T12:45:42.455Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Oct 08, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-08T12:45:42.491Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Oct 08, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-08T12:45:42.520Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Oct 08, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-08T12:45:42.547Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Oct 08, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-08T12:45:42.576Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Oct 08, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-08T12:45:42.603Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Oct 08, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-08T12:45:42.633Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Oct 08, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-08T12:45:42.664Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Oct 08, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-08T12:45:42.688Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Oct 08, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-08T12:45:42.713Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Oct 08, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-08T12:45:43.051Z: Starting 5 ****s in us-central1-a...
Oct 08, 2021 12:46:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-08T12:46:13.573Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Oct 08, 2021 12:46:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-08T12:46:23.365Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Oct 08, 2021 12:47:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-08T12:47:25.664Z: Workers have started successfully.
Oct 08, 2021 12:47:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-08T12:47:25.698Z: Workers have started successfully.
Oct 08, 2021 4:00:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-08T16:00:38.056Z: Cancel request is committed for workflow job: 2021-10-08_05_45_29-12309857362774759606.
Oct 08, 2021 4:00:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-08T16:00:38.283Z: Cleaning up.
Oct 08, 2021 4:00:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-08T16:00:38.366Z: Stopping **** pool...
Oct 08, 2021 4:00:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-08T16:00:38.439Z: Stopping **** pool...
Oct 08, 2021 4:03:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-08T16:03:02.106Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Oct 08, 2021 4:03:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-08T16:03:02.149Z: Worker pool stopped.
Oct 08, 2021 4:03:09 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-10-08_05_45_29-12309857362774759606 finished with status CANCELLED.
Oct 08, 2021 4:03:09 PM org.apache.beam.sdk.testutils.metrics.MetricsReader getCounterMetric
SEVERE: Failed to get metric totalBytes.count, from namespace co_gbk
Load test results for test (ID): 1f1e0994-1f49-4adc-9278-e1a2b31c5e51 and timestamp: 2021-10-08T12:45:24.872000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                       0.0
dataflow_v2_java11_total_bytes_count                      -1.0
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211008124332
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:1447960b06deec03f1c40156ae68f611464231872252e290a78a8ca8b5fbc7c4
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211008124332]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:1447960b06deec03f1c40156ae68f611464231872252e290a78a8ca8b5fbc7c4]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211008124332] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:1447960b06deec03f1c40156ae68f611464231872252e290a78a8ca8b5fbc7c4])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:1447960b06deec03f1c40156ae68f611464231872252e290a78a8ca8b5fbc7c4
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:1447960b06deec03f1c40156ae68f611464231872252e290a78a8ca8b5fbc7c4
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:1447960b06deec03f1c40156ae68f611464231872252e290a78a8ca8b5fbc7c4].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 54s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/x42dkfqynqtlo

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #112

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/112/display/redirect?page=changes>

Changes:

[Robert Bradshaw] Restore "Default to Runner v2 for Python Streaming jobs. (#15140)"

[stefan.istrate] Fix "too many pings" errors.

[stefan.istrate] Increase keepalive timeout to 5 minutes.

[david.prieto] [BEAM-12950] Not delete orphaned files to avoid missing events

[david.prieto] [BEAM-12950] Add Bug fix description to CHANGES.md

[david.prieto] [BEAM-12950] fix linter issues

[Robert Bradshaw] Dead letter option.

[Robert Bradshaw] Guard setup.py logic with __main__ condition.

[stefan.istrate] Fix yapf complaints.

[Robert Bradshaw] Avoid incompatible setting.

[aydar.zaynutdinov] [BEAM-12969] [Playground]

[david.prieto] [BEAN-12950] Skip unit test

[noreply] [BEAM-12909][BEAM-12849]  Add support for running spark3 nexmark queries

[Robert Bradshaw] Add the ability to use subprocesses with the dead letter queue.

[Robert Bradshaw] Support multi-output DoFns.

[noreply] Merge pull request #15510 from [BEAM-12883] Add coder for

[Robert Bradshaw] multi-output fix

[Robert Bradshaw] Add thresholding to dead letter pattern.

[Robert Bradshaw] treshold test fixes

[Robert Bradshaw] Better naming, documentation.

[noreply] [BEAM-12482] Ensure that we ignore schema update options when loading

[Kyle Weaver] Moving to 2.35.0-SNAPSHOT on master branch.

[Kyle Weaver] Add 2.35.0 section to changelog.

[noreply] Fix email links in the contact page


------------------------------------------
[...truncated 48.66 KB...]
160c9ee67560: Preparing
0e3d0b3abb6a: Preparing
7b536ea6ab11: Preparing
420dd72927f0: Preparing
c3f410888c3c: Preparing
1b9057003931: Preparing
c73056b16765: Preparing
874ad65f91ea: Preparing
0fc2498b65e5: Preparing
d08e6b97bf21: Preparing
3054497613e6: Preparing
0e3d0b3abb6a: Waiting
d35dc7f4c79e: Preparing
dabfe5b2ea81: Preparing
5e6a409f30b6: Preparing
7b536ea6ab11: Waiting
160c9ee67560: Waiting
3054497613e6: Waiting
420dd72927f0: Waiting
1b9057003931: Waiting
d35dc7f4c79e: Waiting
dabfe5b2ea81: Waiting
0fc2498b65e5: Waiting
5e6a409f30b6: Waiting
c73056b16765: Waiting
874ad65f91ea: Waiting
d08e6b97bf21: Waiting
e35b5129678d: Pushed
6e384c2b1809: Pushed
6ecbd3a9033f: Pushed
160c9ee67560: Pushed
f48d33aaf847: Pushed
0a9c7e99229b: Pushed
7b536ea6ab11: Pushed
420dd72927f0: Pushed
c73056b16765: Pushed
874ad65f91ea: Layer already exists
1b9057003931: Pushed
d08e6b97bf21: Layer already exists
0fc2498b65e5: Layer already exists
3054497613e6: Layer already exists
0e3d0b3abb6a: Pushed
d35dc7f4c79e: Layer already exists
dabfe5b2ea81: Layer already exists
5e6a409f30b6: Layer already exists
c3f410888c3c: Pushed
20211007124337: digest: sha256:11c0503cf08d39e4b66d2656f17cc15473fde8fe87c87f5344ef98c37c52b182 size: 4311

> Task :sdks:java:testing:load-tests:run
Oct 07, 2021 12:45:51 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Oct 07, 2021 12:45:51 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 195 files. Enable logging at DEBUG level to see which files will be staged.
Oct 07, 2021 12:45:52 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Oct 07, 2021 12:45:52 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Oct 07, 2021 12:45:54 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 195 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Oct 07, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 195 files cached, 0 files newly uploaded in 0 seconds
Oct 07, 2021 12:45:55 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Oct 07, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <108949 bytes, hash eb1d8cd27ace7e0e65013f0eced8e52f70fbf23f069c491660490d7d5b97df53> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-6x2M0nrOfg5lAT8OztjlL3D78j8GnEkWYEkNfVuX31M.pb
Oct 07, 2021 12:45:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Oct 07, 2021 12:45:56 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2a869a16, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@ae202c6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@46aa712c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6ada9c0c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7412ed6b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6e7c351d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b4a0aef, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@45cec376, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26d41711, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@298f0a0b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2b960a7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@31dfc6f5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@37b52340, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@663bb8ef, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2f4e40d7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@60e9c3a5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7e5843db, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@459f703f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@188ac8a3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3650d4fc]
Oct 07, 2021 12:45:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Oct 07, 2021 12:45:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Oct 07, 2021 12:45:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Oct 07, 2021 12:45:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Oct 07, 2021 12:45:56 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@441b8382, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1df1ced0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5349b246, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@32b0876c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2aaf152b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@39bbd9e0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@27fe9713, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@11c3ff67, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4397a639, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@24841372, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77114efe, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@79a7bfbc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77f68df, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3e4e4c1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7e7f3cfd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ae126d1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@46a488c2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6242ae3b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@65ddee5a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@56399b9e]
Oct 07, 2021 12:45:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Oct 07, 2021 12:45:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Oct 07, 2021 12:45:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Oct 07, 2021 12:45:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Oct 07, 2021 12:45:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Oct 07, 2021 12:45:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Oct 07, 2021 12:45:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Oct 07, 2021 12:45:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Oct 07, 2021 12:45:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Oct 07, 2021 12:45:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Oct 07, 2021 12:45:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Oct 07, 2021 12:45:56 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.35.0-SNAPSHOT
Oct 07, 2021 12:45:57 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-10-07_05_45_57-9477186185672990765?project=apache-beam-testing
Oct 07, 2021 12:45:57 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-10-07_05_45_57-9477186185672990765
Oct 07, 2021 12:45:57 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-10-07_05_45_57-9477186185672990765
Oct 07, 2021 12:46:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-10-07T12:46:04.506Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-10-adcz. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Oct 07, 2021 12:46:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-07T12:46:10.714Z: Worker configuration: e2-standard-2 in us-central1-a.
Oct 07, 2021 12:46:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-07T12:46:11.476Z: Expanding SplittableParDo operations into optimizable parts.
Oct 07, 2021 12:46:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-07T12:46:11.524Z: Expanding CollectionToSingleton operations into optimizable parts.
Oct 07, 2021 12:46:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-07T12:46:11.571Z: Expanding CoGroupByKey operations into optimizable parts.
Oct 07, 2021 12:46:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-07T12:46:11.620Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Oct 07, 2021 12:46:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-07T12:46:11.639Z: Expanding GroupByKey operations into streaming Read/Write steps
Oct 07, 2021 12:46:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-07T12:46:11.701Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Oct 07, 2021 12:46:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-07T12:46:11.803Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Oct 07, 2021 12:46:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-07T12:46:11.836Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Oct 07, 2021 12:46:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-07T12:46:11.875Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Oct 07, 2021 12:46:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-07T12:46:11.904Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Oct 07, 2021 12:46:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-07T12:46:11.938Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Oct 07, 2021 12:46:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-07T12:46:11.970Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Oct 07, 2021 12:46:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-07T12:46:12.005Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Oct 07, 2021 12:46:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-07T12:46:12.030Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Oct 07, 2021 12:46:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-07T12:46:12.049Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Oct 07, 2021 12:46:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-07T12:46:12.078Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Oct 07, 2021 12:46:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-07T12:46:12.107Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Oct 07, 2021 12:46:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-07T12:46:12.129Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Oct 07, 2021 12:46:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-07T12:46:12.164Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Oct 07, 2021 12:46:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-07T12:46:12.189Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Oct 07, 2021 12:46:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-07T12:46:12.241Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Oct 07, 2021 12:46:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-07T12:46:12.277Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Oct 07, 2021 12:46:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-07T12:46:12.314Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Oct 07, 2021 12:46:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-07T12:46:12.374Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Oct 07, 2021 12:46:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-07T12:46:12.596Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Oct 07, 2021 12:46:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-07T12:46:12.671Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Oct 07, 2021 12:46:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-07T12:46:12.717Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Oct 07, 2021 12:46:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-07T12:46:12.754Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Oct 07, 2021 12:46:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-07T12:46:12.793Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Oct 07, 2021 12:46:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-07T12:46:13.175Z: Starting 5 ****s in us-central1-a...
Oct 07, 2021 12:46:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-07T12:46:17.671Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Oct 07, 2021 12:46:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-07T12:46:54.095Z: Autoscaling: Raised the number of ****s to 3 so that the pipeline can catch up with its backlog and keep up with its input rate.
Oct 07, 2021 12:46:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-07T12:46:54.131Z: Resized **** pool to 3, though goal was 5.  This could be a quota issue.
Oct 07, 2021 12:47:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-07T12:47:04.550Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Oct 07, 2021 12:47:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-07T12:47:57.232Z: Workers have started successfully.
Oct 07, 2021 12:47:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-07T12:47:57.267Z: Workers have started successfully.
Oct 07, 2021 4:00:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-07T16:00:35.338Z: Cancel request is committed for workflow job: 2021-10-07_05_45_57-9477186185672990765.
Oct 07, 2021 4:00:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-07T16:00:35.411Z: Cleaning up.
Oct 07, 2021 4:00:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-07T16:00:35.503Z: Stopping **** pool...
Oct 07, 2021 4:00:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-07T16:00:35.559Z: Stopping **** pool...
Oct 07, 2021 4:02:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-07T16:02:56.441Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Oct 07, 2021 4:02:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-07T16:02:56.476Z: Worker pool stopped.
Oct 07, 2021 4:03:02 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-10-07_05_45_57-9477186185672990765 finished with status CANCELLED.
Oct 07, 2021 4:03:02 PM org.apache.beam.sdk.testutils.metrics.MetricsReader getCounterMetric
SEVERE: Failed to get metric totalBytes.count, from namespace co_gbk
Load test results for test (ID): 91d881a9-db67-46b6-87eb-f7bc4b2362f4 and timestamp: 2021-10-07T12:45:51.937000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                       0.0
dataflow_v2_java11_total_bytes_count                      -1.0
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211007124337
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:11c0503cf08d39e4b66d2656f17cc15473fde8fe87c87f5344ef98c37c52b182
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211007124337]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:11c0503cf08d39e4b66d2656f17cc15473fde8fe87c87f5344ef98c37c52b182]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211007124337] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:11c0503cf08d39e4b66d2656f17cc15473fde8fe87c87f5344ef98c37c52b182])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:11c0503cf08d39e4b66d2656f17cc15473fde8fe87c87f5344ef98c37c52b182
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:11c0503cf08d39e4b66d2656f17cc15473fde8fe87c87f5344ef98c37c52b182
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:11c0503cf08d39e4b66d2656f17cc15473fde8fe87c87f5344ef98c37c52b182].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 47s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/xqmtc2upalt3o

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #111

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/111/display/redirect?page=changes>

Changes:

[noreply] Only stage the single (fat) jar when auto-starting expansion service.

[noreply] Disable samza counters (#15659)

[noreply] Merge pull request #15614 from [BEAM-12953] [Playground] Create protobuf

[noreply] [BEAM-11831] Parially Revert "[BEAM-11805] Replace user-agent for

[kawaigin] [BEAM-10708] Enable submit beam_sql built jobs to Dataflow

[noreply] Merge pull request #15602 from [BEAM-10917] Add support for BigQuery


------------------------------------------
[...truncated 50.07 KB...]
71f5d4a7c0ce: Waiting
5e81acc91598: Pushed
094dbb8ccf85: Pushed
4c21d10adb6c: Pushed
cf18ee068f1f: Pushed
714381da582b: Pushed
7eb1bf43d45c: Pushed
bbe3d4d07eac: Pushed
61152048081e: Pushed
874ad65f91ea: Layer already exists
0fc2498b65e5: Layer already exists
d08e6b97bf21: Layer already exists
3054497613e6: Layer already exists
f9a8961724da: Pushed
d35dc7f4c79e: Layer already exists
8bd8b261dd95: Pushed
dabfe5b2ea81: Layer already exists
5e6a409f30b6: Layer already exists
71f5d4a7c0ce: Pushed
0f672dd9254e: Pushed
20211006124334: digest: sha256:e628016bfaefed79c9cff4756a17dbfc5c8f0a02b812613169cf37166ac31d39 size: 4311

> Task :sdks:java:testing:load-tests:run
Oct 06, 2021 12:45:43 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Oct 06, 2021 12:45:43 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 195 files. Enable logging at DEBUG level to see which files will be staged.
Oct 06, 2021 12:45:44 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Oct 06, 2021 12:45:44 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Oct 06, 2021 12:45:46 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 195 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Oct 06, 2021 12:45:47 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 195 files cached, 0 files newly uploaded in 0 seconds
Oct 06, 2021 12:45:47 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Oct 06, 2021 12:45:47 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <108949 bytes, hash ddbfa83e790ba31b88c31ac68c22385d9dcdf84d1287bca449e8067d47787b60> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-3b-oPnkLoxuIwxrGjCI4XZ3N-E0Sh7ykSegGfUd4e2A.pb
Oct 06, 2021 12:45:49 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Oct 06, 2021 12:45:49 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2a869a16, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@ae202c6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@46aa712c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6ada9c0c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7412ed6b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6e7c351d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b4a0aef, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@45cec376, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26d41711, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@298f0a0b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2b960a7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@31dfc6f5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@37b52340, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@663bb8ef, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2f4e40d7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@60e9c3a5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7e5843db, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@459f703f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@188ac8a3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3650d4fc]
Oct 06, 2021 12:45:49 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Oct 06, 2021 12:45:49 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Oct 06, 2021 12:45:49 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Oct 06, 2021 12:45:49 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Oct 06, 2021 12:45:49 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@441b8382, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1df1ced0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5349b246, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@32b0876c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2aaf152b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@39bbd9e0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@27fe9713, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@11c3ff67, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4397a639, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@24841372, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77114efe, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@79a7bfbc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77f68df, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3e4e4c1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7e7f3cfd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ae126d1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@46a488c2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6242ae3b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@65ddee5a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@56399b9e]
Oct 06, 2021 12:45:49 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Oct 06, 2021 12:45:49 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Oct 06, 2021 12:45:49 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Oct 06, 2021 12:45:49 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Oct 06, 2021 12:45:49 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Oct 06, 2021 12:45:49 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Oct 06, 2021 12:45:49 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Oct 06, 2021 12:45:49 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Oct 06, 2021 12:45:49 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Oct 06, 2021 12:45:49 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Oct 06, 2021 12:45:49 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Oct 06, 2021 12:45:49 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.34.0-SNAPSHOT
Oct 06, 2021 12:45:50 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-10-06_05_45_49-7294463732958114634?project=apache-beam-testing
Oct 06, 2021 12:45:50 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-10-06_05_45_49-7294463732958114634
Oct 06, 2021 12:45:50 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-10-06_05_45_49-7294463732958114634
Oct 06, 2021 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-10-06T12:45:56.672Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-10-556h. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Oct 06, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-06T12:46:01.350Z: Worker configuration: e2-standard-2 in us-central1-a.
Oct 06, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-06T12:46:02.010Z: Expanding SplittableParDo operations into optimizable parts.
Oct 06, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-06T12:46:02.055Z: Expanding CollectionToSingleton operations into optimizable parts.
Oct 06, 2021 12:46:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-06T12:46:02.124Z: Expanding CoGroupByKey operations into optimizable parts.
Oct 06, 2021 12:46:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-06T12:46:02.205Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Oct 06, 2021 12:46:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-06T12:46:02.223Z: Expanding GroupByKey operations into streaming Read/Write steps
Oct 06, 2021 12:46:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-06T12:46:02.278Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Oct 06, 2021 12:46:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-06T12:46:02.384Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Oct 06, 2021 12:46:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-06T12:46:02.415Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Oct 06, 2021 12:46:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-06T12:46:02.446Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Oct 06, 2021 12:46:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-06T12:46:02.478Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Oct 06, 2021 12:46:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-06T12:46:02.504Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Oct 06, 2021 12:46:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-06T12:46:02.530Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Oct 06, 2021 12:46:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-06T12:46:02.559Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Oct 06, 2021 12:46:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-06T12:46:02.596Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Oct 06, 2021 12:46:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-06T12:46:02.628Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Oct 06, 2021 12:46:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-06T12:46:02.661Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Oct 06, 2021 12:46:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-06T12:46:02.690Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Oct 06, 2021 12:46:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-06T12:46:02.716Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Oct 06, 2021 12:46:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-06T12:46:02.745Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Oct 06, 2021 12:46:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-06T12:46:02.775Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Oct 06, 2021 12:46:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-06T12:46:02.807Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Oct 06, 2021 12:46:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-06T12:46:02.858Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Oct 06, 2021 12:46:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-06T12:46:02.888Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Oct 06, 2021 12:46:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-06T12:46:02.912Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Oct 06, 2021 12:46:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-06T12:46:02.940Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Oct 06, 2021 12:46:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-06T12:46:02.978Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Oct 06, 2021 12:46:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-06T12:46:03.004Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Oct 06, 2021 12:46:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-06T12:46:03.055Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Oct 06, 2021 12:46:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-06T12:46:03.082Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Oct 06, 2021 12:46:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-06T12:46:03.428Z: Starting 5 ****s in us-central1-a...
Oct 06, 2021 12:46:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-06T12:46:20.669Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Oct 06, 2021 12:46:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-06T12:46:47.561Z: Autoscaling: Raised the number of ****s to 4 so that the pipeline can catch up with its backlog and keep up with its input rate.
Oct 06, 2021 12:46:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-06T12:46:47.597Z: Resized **** pool to 4, though goal was 5.  This could be a quota issue.
Oct 06, 2021 12:47:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-06T12:46:57.987Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Oct 06, 2021 12:47:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-06T12:47:39.882Z: Workers have started successfully.
Oct 06, 2021 12:47:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-06T12:47:39.906Z: Workers have started successfully.
Oct 06, 2021 4:00:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-06T16:00:26.283Z: Cancel request is committed for workflow job: 2021-10-06_05_45_49-7294463732958114634.
Oct 06, 2021 4:00:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-06T16:00:26.344Z: Cleaning up.
Oct 06, 2021 4:00:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-06T16:00:26.423Z: Stopping **** pool...
Oct 06, 2021 4:00:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-06T16:00:26.475Z: Stopping **** pool...
Oct 06, 2021 4:03:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-06T16:02:58.434Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Oct 06, 2021 4:03:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-06T16:02:58.478Z: Worker pool stopped.
Oct 06, 2021 4:03:03 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-10-06_05_45_49-7294463732958114634 finished with status CANCELLED.
Load test results for test (ID): 3318cbb5-e9bd-4b3d-91a2-e9d4b550ae4f and timestamp: 2021-10-06T12:45:43.844000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11527.271
dataflow_v2_java11_total_bytes_count             2.63679296E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211006124334
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:e628016bfaefed79c9cff4756a17dbfc5c8f0a02b812613169cf37166ac31d39
Deleted: sha256:7bbf3285144113335b0065fc3ef1ec7edf73bd6f80ee93017ea17caf970dd188
Deleted: sha256:76505d323d88ad58093e61d857381448040b965316c6723d9b3325588dbe3218
Deleted: sha256:afa140d8cc6b486910b2888e64f7d79203690319b57a933122c09e25caa29810
Deleted: sha256:5303ccee4b211750d4697609d783a189cf29570274f23ae43c9f8123def30f23
Deleted: sha256:171ffe0152ee103c3c35c63fc180f991d36cd44af9b64e031673b9d44a405a30
Deleted: sha256:60b43119adb48034e4add73a695523c3dbf2f27651c0687a02fa023ab6f3f38b
Deleted: sha256:cde541cd0dc247fe4fb433fa29ac641d13b8fd4efd4fdbc8fff105d8d9d957ec
Deleted: sha256:11414c1abb8f3de6e5c0062908897c982f699c208650f36795f28010afc4f51f
Deleted: sha256:67efe9104c53640dda844ab8ec04c55b398bce4b981df267161e9eeedad2089b
Deleted: sha256:efc26505aecb7a8c53f2e28018dea9c5566ba259c25b4d1b032e3251a495d909
Deleted: sha256:ae3fbe8b708a452d5f2b22d4e98d89ab3751973c8e2359102864fad0555eaf0b
Deleted: sha256:b66da3f19303aa9c7803a4cb004f5fa30028828b68c097540639d69944c8aa73
Deleted: sha256:2ca58c3ef0f2f4b600c12e00a425813a954aaa1cfaf0e7381f93c356794b0bbc
Deleted: sha256:5496f82e318bd89065476e2f26399a910b54a0f573271937c037d3eb5c5c7e8c
Deleted: sha256:da4fe823d6811761a2df825e4703d6d984e7ac400520156eba4bfe11014df8c2
Deleted: sha256:aff94b42c584140951d41f832c3265adc2ecdce2d1cdc74de0483c774d986dda
Deleted: sha256:50c847249b22f605c39871352cf8a51f044f9c56a79453580c93c1ef3d2eea10
Deleted: sha256:85cc57f55fe877d4cf64226494d3c4931c0674f31d2e8b2f225b6fe12ab468aa
Deleted: sha256:1c12885fdc712d6f6fb8beab01622935019acc90f15c9446dd8089d957d5fc84
Deleted: sha256:6a3e88143635807e52ef3f715941e0da9dd67915c48a50a8ee01f8f314d50669
Deleted: sha256:7ddea6e09e44b81f97cd7d2a55f49cfd4a36eed2049f1ea6c0261d54dccea484
Deleted: sha256:373e96462f2ff4cd979f30ae7c36cce65ae749877989edb8a5c8c6fe95e05768
Deleted: sha256:acfcdce21a67e9a3aaa9d49a40d33df17df74a78f5cdddc85c607fc6c274f5bd
Deleted: sha256:b7a76f6eb431d41b8e9620a4dc49a29367e2545c662d54b52b159d784c9cbfea
Deleted: sha256:05f764117388c05707eb4ec41f86e86ec92607b8e4ce23fadcb9407d1c04bcc9
Deleted: sha256:d06b76b5a9511376df0350fa36e8d839c800b4b6aba4db0cba427609dfe35ef3
Deleted: sha256:be578b9fd9685c97b096078673a86ef21301d373d91eacaf7e7307e79e963eae
Deleted: sha256:9d27526a3324ec0604f28abca0494c642329938e037fec88dd75b42c05cda26b
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211006124334]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:e628016bfaefed79c9cff4756a17dbfc5c8f0a02b812613169cf37166ac31d39]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211006124334] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:e628016bfaefed79c9cff4756a17dbfc5c8f0a02b812613169cf37166ac31d39])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:e628016bfaefed79c9cff4756a17dbfc5c8f0a02b812613169cf37166ac31d39
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:e628016bfaefed79c9cff4756a17dbfc5c8f0a02b812613169cf37166ac31d39
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:e628016bfaefed79c9cff4756a17dbfc5c8f0a02b812613169cf37166ac31d39].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 47s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/lmrharnyeqwl6

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #110

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/110/display/redirect?page=changes>

Changes:

[danthev] Switch hintNumWorkers to ValueProvider, switch firstInstant to side

[noreply] [BEAM-11516] Upgrade to pylint 2.11.1, fix warnings (#15612)

[noreply] [BEAM-12979, BEAM-11097] Change cache to store ReStreams, clean up to…

[noreply] [BEAM-3304, BEAM-12513] Trigger changes and Windowing. (#15644)

[noreply] Merge pull request #15618 from [BEAM-12968] [Playground] Create README

[noreply] Merge pull request #15626 from [BEAM-12963] [Playground] Create base

[noreply] Merge pull request #15566 from [BEAM-12925] Correct behavior retrieving

[noreply] [BEAM-12911] Update schema translation (Java, Python) to log more

[noreply] Update CHANGES.md for JdbcIO breaking change (#15651)

[noreply] [BEAM-12513] Add Go SDK metrics content to BPG. (#15650)

[noreply] [BEAM-12996] Improve Error Logging in ConfigBuilder (#15646)

[noreply] [BEAM-13000] Disable Reshuffle Translation in Samza Portable Mode


------------------------------------------
[...truncated 50.54 KB...]
d35dc7f4c79e: Layer already exists
dabfe5b2ea81: Layer already exists
5e6a409f30b6: Layer already exists
2a9ded77b8c8: Pushed
20211005124338: digest: sha256:ab9613b9dc166dd247d63962af738ee43bb2af319c8c3c6671ebebe920f88cd3 size: 4311

> Task :sdks:java:testing:load-tests:run
Oct 05, 2021 12:45:52 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Oct 05, 2021 12:45:52 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 195 files. Enable logging at DEBUG level to see which files will be staged.
Oct 05, 2021 12:45:53 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Oct 05, 2021 12:45:53 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Oct 05, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 195 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Oct 05, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 195 files cached, 0 files newly uploaded in 0 seconds
Oct 05, 2021 12:45:56 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Oct 05, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <108949 bytes, hash c2c0a836df596cfe1b9287aa8bd003a2f851522c1c94950c98f05c765b52bc9d> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-wsCoNt9ZbP4bkoeqi9ADovhRUiwclJUMmPBcdltSvJ0.pb
Oct 05, 2021 12:45:58 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Oct 05, 2021 12:45:58 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2a869a16, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@ae202c6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@46aa712c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6ada9c0c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7412ed6b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6e7c351d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b4a0aef, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@45cec376, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26d41711, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@298f0a0b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2b960a7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@31dfc6f5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@37b52340, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@663bb8ef, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2f4e40d7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@60e9c3a5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7e5843db, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@459f703f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@188ac8a3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3650d4fc]
Oct 05, 2021 12:45:58 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Oct 05, 2021 12:45:58 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Oct 05, 2021 12:45:58 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Oct 05, 2021 12:45:58 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Oct 05, 2021 12:45:58 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7da39774, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@441b8382, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1df1ced0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5349b246, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@32b0876c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2aaf152b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@39bbd9e0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@27fe9713, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@11c3ff67, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4397a639, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@24841372, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77114efe, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@79a7bfbc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77f68df, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3e4e4c1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7e7f3cfd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ae126d1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@46a488c2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6242ae3b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@65ddee5a]
Oct 05, 2021 12:45:58 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Oct 05, 2021 12:45:58 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Oct 05, 2021 12:45:58 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Oct 05, 2021 12:45:58 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Oct 05, 2021 12:45:58 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Oct 05, 2021 12:45:58 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Oct 05, 2021 12:45:58 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Oct 05, 2021 12:45:58 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Oct 05, 2021 12:45:58 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Oct 05, 2021 12:45:58 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Oct 05, 2021 12:45:58 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Oct 05, 2021 12:45:58 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.34.0-SNAPSHOT
Oct 05, 2021 12:46:00 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-10-05_05_45_58-6157338180392792576?project=apache-beam-testing
Oct 05, 2021 12:46:00 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-10-05_05_45_58-6157338180392792576
Oct 05, 2021 12:46:00 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-10-05_05_45_58-6157338180392792576
Oct 05, 2021 12:46:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-10-05T12:46:07.146Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-10-gspg. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Oct 05, 2021 12:46:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-05T12:46:11.314Z: Worker configuration: e2-standard-2 in us-central1-a.
Oct 05, 2021 12:46:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-05T12:46:12.065Z: Expanding SplittableParDo operations into optimizable parts.
Oct 05, 2021 12:46:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-05T12:46:12.098Z: Expanding CollectionToSingleton operations into optimizable parts.
Oct 05, 2021 12:46:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-05T12:46:12.154Z: Expanding CoGroupByKey operations into optimizable parts.
Oct 05, 2021 12:46:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-05T12:46:12.219Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Oct 05, 2021 12:46:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-05T12:46:12.255Z: Expanding GroupByKey operations into streaming Read/Write steps
Oct 05, 2021 12:46:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-05T12:46:12.334Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Oct 05, 2021 12:46:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-05T12:46:12.425Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Oct 05, 2021 12:46:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-05T12:46:12.455Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Oct 05, 2021 12:46:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-05T12:46:12.490Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Oct 05, 2021 12:46:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-05T12:46:12.538Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Oct 05, 2021 12:46:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-05T12:46:12.572Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Oct 05, 2021 12:46:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-05T12:46:12.602Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Oct 05, 2021 12:46:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-05T12:46:12.642Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Oct 05, 2021 12:46:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-05T12:46:12.675Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Oct 05, 2021 12:46:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-05T12:46:12.708Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Oct 05, 2021 12:46:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-05T12:46:12.736Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Oct 05, 2021 12:46:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-05T12:46:12.761Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Oct 05, 2021 12:46:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-05T12:46:12.783Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Oct 05, 2021 12:46:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-05T12:46:12.810Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Oct 05, 2021 12:46:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-05T12:46:12.845Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Oct 05, 2021 12:46:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-05T12:46:12.879Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Oct 05, 2021 12:46:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-05T12:46:12.911Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Oct 05, 2021 12:46:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-05T12:46:12.944Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Oct 05, 2021 12:46:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-05T12:46:12.979Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Oct 05, 2021 12:46:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-05T12:46:13.012Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Oct 05, 2021 12:46:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-05T12:46:13.041Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Oct 05, 2021 12:46:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-05T12:46:13.080Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Oct 05, 2021 12:46:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-05T12:46:13.107Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Oct 05, 2021 12:46:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-05T12:46:13.138Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Oct 05, 2021 12:46:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-05T12:46:13.546Z: Starting 5 ****s in us-central1-a...
Oct 05, 2021 12:46:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-05T12:46:34.885Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Oct 05, 2021 12:46:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-05T12:46:48.957Z: Autoscaling: Raised the number of ****s to 1 so that the pipeline can catch up with its backlog and keep up with its input rate.
Oct 05, 2021 12:46:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-05T12:46:48.989Z: Resized **** pool to 1, though goal was 5.  This could be a quota issue.
Oct 05, 2021 12:47:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-05T12:46:59.336Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Oct 05, 2021 12:47:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-05T12:47:52.464Z: Workers have started successfully.
Oct 05, 2021 12:47:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-05T12:47:52.490Z: Workers have started successfully.
Oct 05, 2021 2:53:15 PM org.apache.beam.sdk.metrics.MetricsEnvironment getCurrentContainer
WARNING: Reporting metrics are not supported in the current execution environment.
Oct 05, 2021 3:19:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-05T15:19:14.210Z: Staged package beam-vendor-grpc-1_36_0-0.2-FOu94RegK8zFq2op7b1CeJprMEONKvFs2JFoeFWpqFU.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-grpc-1_36_0-0.2-FOu94RegK8zFq2op7b1CeJprMEONKvFs2JFoeFWpqFU.jar' is inaccessible.
Oct 05, 2021 3:19:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-10-05T15:19:16.481Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Oct 05, 2021 3:22:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-10-05T15:22:16.970Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Oct 05, 2021 3:25:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-05T15:25:14.242Z: Staged package beam-vendor-grpc-1_36_0-0.2-FOu94RegK8zFq2op7b1CeJprMEONKvFs2JFoeFWpqFU.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-grpc-1_36_0-0.2-FOu94RegK8zFq2op7b1CeJprMEONKvFs2JFoeFWpqFU.jar' is inaccessible.
Oct 05, 2021 3:25:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-10-05T15:25:16.929Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Oct 05, 2021 3:28:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-10-05T15:28:17.058Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Oct 05, 2021 3:31:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-05T15:31:14.323Z: Staged package beam-vendor-grpc-1_36_0-0.2-FOu94RegK8zFq2op7b1CeJprMEONKvFs2JFoeFWpqFU.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-grpc-1_36_0-0.2-FOu94RegK8zFq2op7b1CeJprMEONKvFs2JFoeFWpqFU.jar' is inaccessible.
Oct 05, 2021 3:31:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-10-05T15:31:17.127Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Oct 05, 2021 3:34:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-10-05T15:34:17.065Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Oct 05, 2021 3:37:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-05T15:37:14.371Z: Staged package beam-vendor-grpc-1_36_0-0.2-FOu94RegK8zFq2op7b1CeJprMEONKvFs2JFoeFWpqFU.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-grpc-1_36_0-0.2-FOu94RegK8zFq2op7b1CeJprMEONKvFs2JFoeFWpqFU.jar' is inaccessible.
Oct 05, 2021 3:37:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-10-05T15:37:16.856Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Oct 05, 2021 3:40:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-10-05T15:40:16.876Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Oct 05, 2021 3:43:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-05T15:43:14.293Z: Staged package beam-vendor-grpc-1_36_0-0.2-FOu94RegK8zFq2op7b1CeJprMEONKvFs2JFoeFWpqFU.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-grpc-1_36_0-0.2-FOu94RegK8zFq2op7b1CeJprMEONKvFs2JFoeFWpqFU.jar' is inaccessible.
Oct 05, 2021 3:43:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-10-05T15:43:16.866Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Oct 05, 2021 3:46:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-10-05T15:46:17.045Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Oct 05, 2021 3:49:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-05T15:49:14.824Z: Staged package beam-vendor-grpc-1_36_0-0.2-FOu94RegK8zFq2op7b1CeJprMEONKvFs2JFoeFWpqFU.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-grpc-1_36_0-0.2-FOu94RegK8zFq2op7b1CeJprMEONKvFs2JFoeFWpqFU.jar' is inaccessible.
Oct 05, 2021 3:49:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-10-05T15:49:17.493Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Oct 05, 2021 3:52:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-10-05T15:52:16.921Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Oct 05, 2021 3:55:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-05T15:55:14.257Z: Staged package beam-vendor-grpc-1_36_0-0.2-FOu94RegK8zFq2op7b1CeJprMEONKvFs2JFoeFWpqFU.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-grpc-1_36_0-0.2-FOu94RegK8zFq2op7b1CeJprMEONKvFs2JFoeFWpqFU.jar' is inaccessible.
Oct 05, 2021 3:55:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-10-05T15:55:16.777Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Oct 05, 2021 3:58:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-10-05T15:58:16.868Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Oct 05, 2021 4:00:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-05T16:00:30.725Z: Cancel request is committed for workflow job: 2021-10-05_05_45_58-6157338180392792576.
Oct 05, 2021 4:00:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-05T16:00:30.751Z: Cleaning up.
Oct 05, 2021 4:00:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-05T16:00:30.801Z: Stopping **** pool...
Oct 05, 2021 4:00:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-05T16:00:30.848Z: Stopping **** pool...
Oct 05, 2021 4:02:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-05T16:02:54.028Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Oct 05, 2021 4:02:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-05T16:02:54.061Z: Worker pool stopped.
Oct 05, 2021 4:02:59 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-10-05_05_45_58-6157338180392792576 finished with status CANCELLED.
Load test results for test (ID): 3fe74c8b-ffc5-49ab-8bc8-d97a737df786 and timestamp: 2021-10-05T12:45:52.765000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11515.863
dataflow_v2_java11_total_bytes_count             2.16827679E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211005124338
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ab9613b9dc166dd247d63962af738ee43bb2af319c8c3c6671ebebe920f88cd3
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211005124338]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ab9613b9dc166dd247d63962af738ee43bb2af319c8c3c6671ebebe920f88cd3]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211005124338] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ab9613b9dc166dd247d63962af738ee43bb2af319c8c3c6671ebebe920f88cd3])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ab9613b9dc166dd247d63962af738ee43bb2af319c8c3c6671ebebe920f88cd3
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ab9613b9dc166dd247d63962af738ee43bb2af319c8c3c6671ebebe920f88cd3
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ab9613b9dc166dd247d63962af738ee43bb2af319c8c3c6671ebebe920f88cd3].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 41s
101 actionable tasks: 71 executed, 28 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/adw7revy7emsm

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #109

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/109/display/redirect>

Changes:


------------------------------------------
[...truncated 49.58 KB...]
9f76f57b7c6b: Waiting
5e6a409f30b6: Waiting
8319f73d7f90: Waiting
5c54222c71d2: Waiting
3054497613e6: Waiting
dabfe5b2ea81: Waiting
d35dc7f4c79e: Waiting
0fc2498b65e5: Waiting
3a50f81907ae: Waiting
d08e6b97bf21: Waiting
874ad65f91ea: Waiting
6835ae92c92b: Pushed
b251633a88ca: Pushed
15cc6117fc3d: Pushed
73459cccf4dd: Pushed
ba9f799a23f0: Pushed
9f76f57b7c6b: Pushed
2f286d92c874: Pushed
5c54222c71d2: Pushed
874ad65f91ea: Layer already exists
0fc2498b65e5: Layer already exists
d08e6b97bf21: Layer already exists
3054497613e6: Layer already exists
d35dc7f4c79e: Layer already exists
dabfe5b2ea81: Layer already exists
5e6a409f30b6: Layer already exists
8319f73d7f90: Pushed
fc040c8215d7: Pushed
3a50f81907ae: Pushed
2b025f87242a: Pushed
20211004124351: digest: sha256:c9882c75be6c4d85295e8745a86d18134e0050f38a39bd621eab3a8f776d629d size: 4311

> Task :sdks:java:testing:load-tests:run
Oct 04, 2021 12:46:29 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Oct 04, 2021 12:46:29 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 195 files. Enable logging at DEBUG level to see which files will be staged.
Oct 04, 2021 12:46:30 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Oct 04, 2021 12:46:30 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Oct 04, 2021 12:46:33 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 195 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Oct 04, 2021 12:46:36 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 195 files cached, 0 files newly uploaded in 2 seconds
Oct 04, 2021 12:46:36 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Oct 04, 2021 12:46:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <108949 bytes, hash adadd895c2b7f1f2147519d5f0b7a947cf150e113435eb0c116f8138417db950> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-ra3YlcK38fIUdRnV8LepR88VDhE0NesMEW-BOEF9uVA.pb
Oct 04, 2021 12:46:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Oct 04, 2021 12:46:38 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@60e9c3a5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7e5843db, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@459f703f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@188ac8a3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3650d4fc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@aaa0f76, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@340a8894, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7a8b9166, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4acc5dff, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@10c72a6f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@70e94ecb, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@56cfe111, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7e446d92, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@57f9b467, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6d5c2745, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@44b29496, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1947596f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3078cac, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@f6de586, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5f2bd6d9]
Oct 04, 2021 12:46:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Oct 04, 2021 12:46:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Oct 04, 2021 12:46:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Oct 04, 2021 12:46:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Oct 04, 2021 12:46:38 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ae126d1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@46a488c2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6242ae3b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@65ddee5a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@56399b9e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@34b9eb03, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43fda8d9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@49d831c2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@53a7a60c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@28bdbe88, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6a87026, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@ef60710, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@600f5704, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2503ec73, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@606f81b5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5e1fc42f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@44b21f9f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7ee8130e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6296474f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4288d98e]
Oct 04, 2021 12:46:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Oct 04, 2021 12:46:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Oct 04, 2021 12:46:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Oct 04, 2021 12:46:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Oct 04, 2021 12:46:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Oct 04, 2021 12:46:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Oct 04, 2021 12:46:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Oct 04, 2021 12:46:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Oct 04, 2021 12:46:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Oct 04, 2021 12:46:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Oct 04, 2021 12:46:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Oct 04, 2021 12:46:38 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.34.0-SNAPSHOT
Oct 04, 2021 12:46:39 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-10-04_05_46_38-593367212600458282?project=apache-beam-testing
Oct 04, 2021 12:46:39 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-10-04_05_46_38-593367212600458282
Oct 04, 2021 12:46:39 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-10-04_05_46_38-593367212600458282
Oct 04, 2021 12:46:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-10-04T12:46:45.865Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-10-orar. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Oct 04, 2021 12:46:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-04T12:46:50.275Z: Worker configuration: e2-standard-2 in us-central1-a.
Oct 04, 2021 12:46:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-04T12:46:51.120Z: Expanding SplittableParDo operations into optimizable parts.
Oct 04, 2021 12:46:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-04T12:46:51.159Z: Expanding CollectionToSingleton operations into optimizable parts.
Oct 04, 2021 12:46:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-04T12:46:51.237Z: Expanding CoGroupByKey operations into optimizable parts.
Oct 04, 2021 12:46:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-04T12:46:51.322Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Oct 04, 2021 12:46:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-04T12:46:51.353Z: Expanding GroupByKey operations into streaming Read/Write steps
Oct 04, 2021 12:46:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-04T12:46:51.402Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Oct 04, 2021 12:46:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-04T12:46:51.559Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Oct 04, 2021 12:46:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-04T12:46:51.612Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Oct 04, 2021 12:46:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-04T12:46:51.655Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Oct 04, 2021 12:46:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-04T12:46:51.680Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Oct 04, 2021 12:46:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-04T12:46:51.705Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Oct 04, 2021 12:46:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-04T12:46:51.737Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Oct 04, 2021 12:46:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-04T12:46:51.760Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Oct 04, 2021 12:46:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-04T12:46:51.792Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Oct 04, 2021 12:46:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-04T12:46:51.826Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Oct 04, 2021 12:46:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-04T12:46:51.857Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Oct 04, 2021 12:46:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-04T12:46:51.890Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Oct 04, 2021 12:46:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-04T12:46:51.926Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Oct 04, 2021 12:46:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-04T12:46:51.957Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Oct 04, 2021 12:46:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-04T12:46:51.990Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Oct 04, 2021 12:46:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-04T12:46:52.040Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Oct 04, 2021 12:46:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-04T12:46:52.069Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Oct 04, 2021 12:46:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-04T12:46:52.103Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Oct 04, 2021 12:46:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-04T12:46:52.132Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Oct 04, 2021 12:46:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-04T12:46:52.166Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Oct 04, 2021 12:46:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-04T12:46:52.200Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Oct 04, 2021 12:46:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-04T12:46:52.235Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Oct 04, 2021 12:46:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-04T12:46:52.268Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Oct 04, 2021 12:46:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-04T12:46:52.298Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Oct 04, 2021 12:46:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-04T12:46:52.636Z: Starting 5 ****s in us-central1-a...
Oct 04, 2021 12:47:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-04T12:47:07.823Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Oct 04, 2021 12:47:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-04T12:47:35.894Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Oct 04, 2021 12:48:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-04T12:48:31.445Z: Workers have started successfully.
Oct 04, 2021 12:48:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-04T12:48:31.479Z: Workers have started successfully.
Oct 04, 2021 1:31:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-04T13:31:53.734Z: Staged package checker-qual-3.10.0-pNyILKaqxJbTM4Hl5esGBMRUg7AEvD6sk2jxuxfLJRI.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/checker-qual-3.10.0-pNyILKaqxJbTM4Hl5esGBMRUg7AEvD6sk2jxuxfLJRI.jar' is inaccessible.
Oct 04, 2021 1:31:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-04T13:31:54.666Z: Staged package google-http-client-1.39.2-arSWeGCSfwUd1_K1deS3qO8182fcLjQIBzqk3uMo4ic.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-http-client-1.39.2-arSWeGCSfwUd1_K1deS3qO8182fcLjQIBzqk3uMo4ic.jar' is inaccessible.
Oct 04, 2021 1:31:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-04T13:31:54.706Z: Staged package google-http-client-apache-v2-1.39.2-UtOSCfZav29XfhyrRmrbmfEtHNK6TpjYblAfddf4UM8.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-http-client-apache-v2-1.39.2-UtOSCfZav29XfhyrRmrbmfEtHNK6TpjYblAfddf4UM8.jar' is inaccessible.
Oct 04, 2021 1:31:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-04T13:31:54.752Z: Staged package google-http-client-appengine-1.39.2-9jINwNsDWv9uqy4odIVg2nGKtGX7w0gbZT6j8nqlysk.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-http-client-appengine-1.39.2-9jINwNsDWv9uqy4odIVg2nGKtGX7w0gbZT6j8nqlysk.jar' is inaccessible.
Oct 04, 2021 1:31:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-04T13:31:54.794Z: Staged package google-http-client-gson-1.39.2-z4GUT9Qf4NqoabkTvQqq7w5Nj-Z3ShlVQPRU7NPPULY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-http-client-gson-1.39.2-z4GUT9Qf4NqoabkTvQqq7w5Nj-Z3ShlVQPRU7NPPULY.jar' is inaccessible.
Oct 04, 2021 1:31:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-04T13:31:54.855Z: Staged package google-http-client-jackson2-1.39.2-dKSTp7ol4d-z06Td9xMwbVo3htsAo0idWQYdyM_zIbE.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-http-client-jackson2-1.39.2-dKSTp7ol4d-z06Td9xMwbVo3htsAo0idWQYdyM_zIbE.jar' is inaccessible.
Oct 04, 2021 1:31:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-04T13:31:54.901Z: Staged package google-http-client-protobuf-1.39.2-48CRMH_IJ9gY8fZ3-5ANC0qF7rqbijaXD90xuQTMr5s.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-http-client-protobuf-1.39.2-48CRMH_IJ9gY8fZ3-5ANC0qF7rqbijaXD90xuQTMr5s.jar' is inaccessible.
Oct 04, 2021 1:31:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-04T13:31:56.325Z: Staged package opencensus-contrib-grpc-util-0.28.0-uRaDRuavZZMwChvCfvdCVKofJAGYhZON2PuFK4d9VfA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/opencensus-contrib-grpc-util-0.28.0-uRaDRuavZZMwChvCfvdCVKofJAGYhZON2PuFK4d9VfA.jar' is inaccessible.
Oct 04, 2021 1:31:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-04T13:31:56.888Z: Staged package zstd-jni-1.4.5-2-lzQxwUtNCahuI7cYQRb8rC2FUB60p0MPfRhczhr0YFA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/zstd-jni-1.4.5-2-lzQxwUtNCahuI7cYQRb8rC2FUB60p0MPfRhczhr0YFA.jar' is inaccessible.
Oct 04, 2021 1:31:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-10-04T13:31:56.915Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Oct 04, 2021 1:34:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-10-04T13:34:56.452Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Oct 04, 2021 4:00:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-04T16:00:25.031Z: Cancel request is committed for workflow job: 2021-10-04_05_46_38-593367212600458282.
Oct 04, 2021 4:00:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-04T16:00:25.101Z: Cleaning up.
Oct 04, 2021 4:00:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-04T16:00:25.190Z: Stopping **** pool...
Oct 04, 2021 4:00:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-04T16:00:25.265Z: Stopping **** pool...
Oct 04, 2021 4:02:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-04T16:02:55.598Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Oct 04, 2021 4:02:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-04T16:02:55.640Z: Worker pool stopped.
Oct 04, 2021 4:03:02 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-10-04_05_46_38-593367212600458282 finished with status CANCELLED.
Load test results for test (ID): 27e6074e-5b81-4ace-ace8-dc66d233bb9a and timestamp: 2021-10-04T12:46:29.845000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11465.354
dataflow_v2_java11_total_bytes_count             2.61286546E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211004124351
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:c9882c75be6c4d85295e8745a86d18134e0050f38a39bd621eab3a8f776d629d
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211004124351]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:c9882c75be6c4d85295e8745a86d18134e0050f38a39bd621eab3a8f776d629d]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211004124351] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:c9882c75be6c4d85295e8745a86d18134e0050f38a39bd621eab3a8f776d629d])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:c9882c75be6c4d85295e8745a86d18134e0050f38a39bd621eab3a8f776d629d
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:c9882c75be6c4d85295e8745a86d18134e0050f38a39bd621eab3a8f776d629d
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:c9882c75be6c4d85295e8745a86d18134e0050f38a39bd621eab3a8f776d629d].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 39s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/5iw7uoimzj6b2

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #108

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/108/display/redirect>

Changes:


------------------------------------------
[...truncated 53.94 KB...]
INFO: Adding Read input/StripIds as step s2
Oct 03, 2021 12:45:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Oct 03, 2021 12:45:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Oct 03, 2021 12:45:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Oct 03, 2021 12:45:42 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2aaf152b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@39bbd9e0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@27fe9713, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@11c3ff67, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4397a639, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@24841372, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77114efe, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@79a7bfbc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77f68df, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3e4e4c1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7e7f3cfd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ae126d1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@46a488c2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6242ae3b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@65ddee5a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@56399b9e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@34b9eb03, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43fda8d9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@49d831c2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@53a7a60c]
Oct 03, 2021 12:45:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Oct 03, 2021 12:45:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Oct 03, 2021 12:45:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Oct 03, 2021 12:45:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Oct 03, 2021 12:45:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Oct 03, 2021 12:45:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Oct 03, 2021 12:45:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Oct 03, 2021 12:45:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Oct 03, 2021 12:45:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Oct 03, 2021 12:45:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Oct 03, 2021 12:45:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Oct 03, 2021 12:45:42 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.34.0-SNAPSHOT
Oct 03, 2021 12:45:43 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-10-03_05_45_42-3666392489918784440?project=apache-beam-testing
Oct 03, 2021 12:45:43 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-10-03_05_45_42-3666392489918784440
Oct 03, 2021 12:45:43 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-10-03_05_45_42-3666392489918784440
Oct 03, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-10-03T12:45:49.889Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-10-3613. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Oct 03, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-03T12:45:54.375Z: Worker configuration: e2-standard-2 in us-central1-a.
Oct 03, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-03T12:45:55.061Z: Expanding SplittableParDo operations into optimizable parts.
Oct 03, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-03T12:45:55.093Z: Expanding CollectionToSingleton operations into optimizable parts.
Oct 03, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-03T12:45:55.158Z: Expanding CoGroupByKey operations into optimizable parts.
Oct 03, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-03T12:45:55.211Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Oct 03, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-03T12:45:55.234Z: Expanding GroupByKey operations into streaming Read/Write steps
Oct 03, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-03T12:45:55.303Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Oct 03, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-03T12:45:55.413Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Oct 03, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-03T12:45:55.442Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Oct 03, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-03T12:45:55.472Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Oct 03, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-03T12:45:55.508Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Oct 03, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-03T12:45:55.543Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Oct 03, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-03T12:45:55.575Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Oct 03, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-03T12:45:55.624Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Oct 03, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-03T12:45:55.656Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Oct 03, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-03T12:45:55.694Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Oct 03, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-03T12:45:55.717Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Oct 03, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-03T12:45:55.744Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Oct 03, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-03T12:45:55.778Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Oct 03, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-03T12:45:55.813Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Oct 03, 2021 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-03T12:45:55.849Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Oct 03, 2021 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-03T12:45:55.886Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Oct 03, 2021 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-03T12:45:55.920Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Oct 03, 2021 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-03T12:45:55.953Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Oct 03, 2021 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-03T12:45:55.988Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Oct 03, 2021 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-03T12:45:56.032Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Oct 03, 2021 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-03T12:45:56.085Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Oct 03, 2021 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-03T12:45:56.119Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Oct 03, 2021 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-03T12:45:56.157Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Oct 03, 2021 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-03T12:45:56.191Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Oct 03, 2021 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-03T12:45:56.593Z: Starting 5 ****s in us-central1-a...
Oct 03, 2021 12:46:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-03T12:46:08.850Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Oct 03, 2021 12:46:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-03T12:46:28.917Z: Autoscaling: Raised the number of ****s to 1 so that the pipeline can catch up with its backlog and keep up with its input rate.
Oct 03, 2021 12:46:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-03T12:46:28.949Z: Resized **** pool to 1, though goal was 5.  This could be a quota issue.
Oct 03, 2021 12:46:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-03T12:46:39.276Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Oct 03, 2021 12:47:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-03T12:47:28.039Z: Workers have started successfully.
Oct 03, 2021 12:47:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-03T12:47:28.072Z: Workers have started successfully.
Oct 03, 2021 1:42:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-03T13:42:57.719Z: Staged package common-config-5.3.2-B_HQvj7POWJFHkM1OEwdiS2Y6zStZi7OXoiw-BJN7Nc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/common-config-5.3.2-B_HQvj7POWJFHkM1OEwdiS2Y6zStZi7OXoiw-BJN7Nc.jar' is inaccessible.
Oct 03, 2021 1:42:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-03T13:42:57.798Z: Staged package common-utils-5.3.2-4Rm-xdfQw0Yp9EH6ym0CZyPloNIGMAJ4b5DO_bByg9c.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/common-utils-5.3.2-4Rm-xdfQw0Yp9EH6ym0CZyPloNIGMAJ4b5DO_bByg9c.jar' is inaccessible.
Oct 03, 2021 1:43:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-03T13:42:59.813Z: Staged package kafka-avro-serializer-5.3.2-7FAoV2nngjZLbCXD3IPBOus_R_2KrM_zffAaNlVOls4.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/kafka-avro-serializer-5.3.2-7FAoV2nngjZLbCXD3IPBOus_R_2KrM_zffAaNlVOls4.jar' is inaccessible.
Oct 03, 2021 1:43:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-03T13:42:59.871Z: Staged package kafka-schema-registry-client-5.3.2-0mGQmXEF1E2hwRsR-O-mddgC_DwcNIU4Ba9SQcw2hKY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/kafka-schema-registry-client-5.3.2-0mGQmXEF1E2hwRsR-O-mddgC_DwcNIU4Ba9SQcw2hKY.jar' is inaccessible.
Oct 03, 2021 1:43:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-03T13:43:00.666Z: Staged package spring-core-4.3.18.RELEASE-qd-hKocv03STfOrJiihBIfzwVyBqE8q9UnPLPqutu2w.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/spring-core-4.3.18.RELEASE-qd-hKocv03STfOrJiihBIfzwVyBqE8q9UnPLPqutu2w.jar' is inaccessible.
Oct 03, 2021 1:43:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-03T13:43:00.708Z: Staged package spring-expression-4.3.18.RELEASE-aB3PS8pe5rtvJwR2BF3WrB6nO_rdEY2KpkFz7bel1_A.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/spring-expression-4.3.18.RELEASE-aB3PS8pe5rtvJwR2BF3WrB6nO_rdEY2KpkFz7bel1_A.jar' is inaccessible.
Oct 03, 2021 1:43:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-03T13:43:00.807Z: Staged package zkclient-0.10-JumIuLuoOMck_YNQszHui1_8WcOpwHTfEVxMOmyEOHg.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/zkclient-0.10-JumIuLuoOMck_YNQszHui1_8WcOpwHTfEVxMOmyEOHg.jar' is inaccessible.
Oct 03, 2021 1:43:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-10-03T13:43:00.854Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Oct 03, 2021 1:46:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-10-03T13:46:00.372Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Oct 03, 2021 1:48:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-03T13:48:57.810Z: Staged package common-config-5.3.2-B_HQvj7POWJFHkM1OEwdiS2Y6zStZi7OXoiw-BJN7Nc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/common-config-5.3.2-B_HQvj7POWJFHkM1OEwdiS2Y6zStZi7OXoiw-BJN7Nc.jar' is inaccessible.
Oct 03, 2021 1:48:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-03T13:48:57.855Z: Staged package common-utils-5.3.2-4Rm-xdfQw0Yp9EH6ym0CZyPloNIGMAJ4b5DO_bByg9c.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/common-utils-5.3.2-4Rm-xdfQw0Yp9EH6ym0CZyPloNIGMAJ4b5DO_bByg9c.jar' is inaccessible.
Oct 03, 2021 1:49:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-03T13:49:00.108Z: Staged package kafka-avro-serializer-5.3.2-7FAoV2nngjZLbCXD3IPBOus_R_2KrM_zffAaNlVOls4.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/kafka-avro-serializer-5.3.2-7FAoV2nngjZLbCXD3IPBOus_R_2KrM_zffAaNlVOls4.jar' is inaccessible.
Oct 03, 2021 1:49:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-03T13:49:00.179Z: Staged package kafka-schema-registry-client-5.3.2-0mGQmXEF1E2hwRsR-O-mddgC_DwcNIU4Ba9SQcw2hKY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/kafka-schema-registry-client-5.3.2-0mGQmXEF1E2hwRsR-O-mddgC_DwcNIU4Ba9SQcw2hKY.jar' is inaccessible.
Oct 03, 2021 1:49:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-03T13:49:01.130Z: Staged package spring-core-4.3.18.RELEASE-qd-hKocv03STfOrJiihBIfzwVyBqE8q9UnPLPqutu2w.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/spring-core-4.3.18.RELEASE-qd-hKocv03STfOrJiihBIfzwVyBqE8q9UnPLPqutu2w.jar' is inaccessible.
Oct 03, 2021 1:49:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-03T13:49:01.176Z: Staged package spring-expression-4.3.18.RELEASE-aB3PS8pe5rtvJwR2BF3WrB6nO_rdEY2KpkFz7bel1_A.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/spring-expression-4.3.18.RELEASE-aB3PS8pe5rtvJwR2BF3WrB6nO_rdEY2KpkFz7bel1_A.jar' is inaccessible.
Oct 03, 2021 1:49:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-03T13:49:01.296Z: Staged package zkclient-0.10-JumIuLuoOMck_YNQszHui1_8WcOpwHTfEVxMOmyEOHg.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/zkclient-0.10-JumIuLuoOMck_YNQszHui1_8WcOpwHTfEVxMOmyEOHg.jar' is inaccessible.
Oct 03, 2021 1:49:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-10-03T13:49:01.341Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Oct 03, 2021 1:52:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-10-03T13:52:00.876Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Oct 03, 2021 1:54:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-03T13:54:57.703Z: Staged package common-config-5.3.2-B_HQvj7POWJFHkM1OEwdiS2Y6zStZi7OXoiw-BJN7Nc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/common-config-5.3.2-B_HQvj7POWJFHkM1OEwdiS2Y6zStZi7OXoiw-BJN7Nc.jar' is inaccessible.
Oct 03, 2021 1:54:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-03T13:54:57.799Z: Staged package common-utils-5.3.2-4Rm-xdfQw0Yp9EH6ym0CZyPloNIGMAJ4b5DO_bByg9c.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/common-utils-5.3.2-4Rm-xdfQw0Yp9EH6ym0CZyPloNIGMAJ4b5DO_bByg9c.jar' is inaccessible.
Oct 03, 2021 1:55:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-03T13:54:59.842Z: Staged package kafka-avro-serializer-5.3.2-7FAoV2nngjZLbCXD3IPBOus_R_2KrM_zffAaNlVOls4.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/kafka-avro-serializer-5.3.2-7FAoV2nngjZLbCXD3IPBOus_R_2KrM_zffAaNlVOls4.jar' is inaccessible.
Oct 03, 2021 1:55:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-03T13:54:59.916Z: Staged package kafka-schema-registry-client-5.3.2-0mGQmXEF1E2hwRsR-O-mddgC_DwcNIU4Ba9SQcw2hKY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/kafka-schema-registry-client-5.3.2-0mGQmXEF1E2hwRsR-O-mddgC_DwcNIU4Ba9SQcw2hKY.jar' is inaccessible.
Oct 03, 2021 1:55:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-03T13:55:00.781Z: Staged package spring-core-4.3.18.RELEASE-qd-hKocv03STfOrJiihBIfzwVyBqE8q9UnPLPqutu2w.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/spring-core-4.3.18.RELEASE-qd-hKocv03STfOrJiihBIfzwVyBqE8q9UnPLPqutu2w.jar' is inaccessible.
Oct 03, 2021 1:55:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-03T13:55:00.836Z: Staged package spring-expression-4.3.18.RELEASE-aB3PS8pe5rtvJwR2BF3WrB6nO_rdEY2KpkFz7bel1_A.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/spring-expression-4.3.18.RELEASE-aB3PS8pe5rtvJwR2BF3WrB6nO_rdEY2KpkFz7bel1_A.jar' is inaccessible.
Oct 03, 2021 1:55:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-03T13:55:00.936Z: Staged package zkclient-0.10-JumIuLuoOMck_YNQszHui1_8WcOpwHTfEVxMOmyEOHg.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/zkclient-0.10-JumIuLuoOMck_YNQszHui1_8WcOpwHTfEVxMOmyEOHg.jar' is inaccessible.
Oct 03, 2021 1:55:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-10-03T13:55:01.006Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Oct 03, 2021 1:58:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-10-03T13:58:00.798Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Oct 03, 2021 2:00:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-03T14:00:57.810Z: Staged package common-config-5.3.2-B_HQvj7POWJFHkM1OEwdiS2Y6zStZi7OXoiw-BJN7Nc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/common-config-5.3.2-B_HQvj7POWJFHkM1OEwdiS2Y6zStZi7OXoiw-BJN7Nc.jar' is inaccessible.
Oct 03, 2021 2:00:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-03T14:00:57.865Z: Staged package common-utils-5.3.2-4Rm-xdfQw0Yp9EH6ym0CZyPloNIGMAJ4b5DO_bByg9c.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/common-utils-5.3.2-4Rm-xdfQw0Yp9EH6ym0CZyPloNIGMAJ4b5DO_bByg9c.jar' is inaccessible.
Oct 03, 2021 2:01:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-03T14:00:59.666Z: Staged package kafka-avro-serializer-5.3.2-7FAoV2nngjZLbCXD3IPBOus_R_2KrM_zffAaNlVOls4.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/kafka-avro-serializer-5.3.2-7FAoV2nngjZLbCXD3IPBOus_R_2KrM_zffAaNlVOls4.jar' is inaccessible.
Oct 03, 2021 2:01:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-03T14:00:59.775Z: Staged package kafka-schema-registry-client-5.3.2-0mGQmXEF1E2hwRsR-O-mddgC_DwcNIU4Ba9SQcw2hKY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/kafka-schema-registry-client-5.3.2-0mGQmXEF1E2hwRsR-O-mddgC_DwcNIU4Ba9SQcw2hKY.jar' is inaccessible.
Oct 03, 2021 2:01:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-03T14:01:00.616Z: Staged package spring-core-4.3.18.RELEASE-qd-hKocv03STfOrJiihBIfzwVyBqE8q9UnPLPqutu2w.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/spring-core-4.3.18.RELEASE-qd-hKocv03STfOrJiihBIfzwVyBqE8q9UnPLPqutu2w.jar' is inaccessible.
Oct 03, 2021 2:01:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-03T14:01:00.663Z: Staged package spring-expression-4.3.18.RELEASE-aB3PS8pe5rtvJwR2BF3WrB6nO_rdEY2KpkFz7bel1_A.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/spring-expression-4.3.18.RELEASE-aB3PS8pe5rtvJwR2BF3WrB6nO_rdEY2KpkFz7bel1_A.jar' is inaccessible.
Oct 03, 2021 2:01:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-10-03T14:01:00.792Z: Staged package zkclient-0.10-JumIuLuoOMck_YNQszHui1_8WcOpwHTfEVxMOmyEOHg.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/zkclient-0.10-JumIuLuoOMck_YNQszHui1_8WcOpwHTfEVxMOmyEOHg.jar' is inaccessible.
Oct 03, 2021 2:01:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-10-03T14:01:00.846Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Oct 03, 2021 2:04:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-10-03T14:04:00.658Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Oct 03, 2021 4:00:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-03T16:00:29.798Z: Cancel request is committed for workflow job: 2021-10-03_05_45_42-3666392489918784440.
Oct 03, 2021 4:00:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-03T16:00:29.839Z: Cleaning up.
Oct 03, 2021 4:00:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-03T16:00:29.929Z: Stopping **** pool...
Oct 03, 2021 4:00:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-03T16:00:29.996Z: Stopping **** pool...
Oct 03, 2021 4:02:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-03T16:02:58.331Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Oct 03, 2021 4:02:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-03T16:02:58.373Z: Worker pool stopped.
Oct 03, 2021 4:03:05 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-10-03_05_45_42-3666392489918784440 finished with status CANCELLED.
Load test results for test (ID): 49c562dd-2153-4d7c-937f-90954f10e085 and timestamp: 2021-10-03T12:45:37.177000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11531.379
dataflow_v2_java11_total_bytes_count             2.87697566E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211003124333
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:51d2c1149def1955f76024ebb3750826a7513251b5b43c88e75e59ddb9c580dd
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211003124333]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:51d2c1149def1955f76024ebb3750826a7513251b5b43c88e75e59ddb9c580dd]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211003124333] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:51d2c1149def1955f76024ebb3750826a7513251b5b43c88e75e59ddb9c580dd])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:51d2c1149def1955f76024ebb3750826a7513251b5b43c88e75e59ddb9c580dd
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:51d2c1149def1955f76024ebb3750826a7513251b5b43c88e75e59ddb9c580dd
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:51d2c1149def1955f76024ebb3750826a7513251b5b43c88e75e59ddb9c580dd].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 50s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/kqhftznrjmsny

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #107

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/107/display/redirect?page=changes>

Changes:

[noreply] Switching to innerText and removing alerts

[noreply] [BEAM-12993] Update to Debezium 1.7.0.Final (#15636)

[noreply] Minor: Fix `Iterable[SplitResultResidual]` type errors (#15634)

[noreply] [BEAM-11217] Implemented metrics filtering (#15482)

[noreply] [BEAM-12513] Schemas and Coders (#15632)


------------------------------------------
[...truncated 49.58 KB...]
640645f1878b: Preparing
29ce3742c9d5: Preparing
52840aa4aa92: Preparing
b28e321914c8: Preparing
791896ea0949: Preparing
d222b75e7c8c: Preparing
874ad65f91ea: Preparing
0fc2498b65e5: Preparing
d08e6b97bf21: Preparing
3054497613e6: Preparing
d35dc7f4c79e: Preparing
dabfe5b2ea81: Preparing
29ce3742c9d5: Waiting
52840aa4aa92: Waiting
5e6a409f30b6: Preparing
b28e321914c8: Waiting
874ad65f91ea: Waiting
0fc2498b65e5: Waiting
791896ea0949: Waiting
d222b75e7c8c: Waiting
d08e6b97bf21: Waiting
3054497613e6: Waiting
d35dc7f4c79e: Waiting
ab6a9c92964e: Waiting
640645f1878b: Waiting
dabfe5b2ea81: Waiting
5e6a409f30b6: Waiting
2c2a8799badc: Pushed
53c97e82a729: Pushed
fadf7501ed55: Pushed
3fde5d19938d: Pushed
ab6a9c92964e: Pushed
dfd8c9ec8f3f: Pushed
29ce3742c9d5: Pushed
52840aa4aa92: Pushed
874ad65f91ea: Layer already exists
0fc2498b65e5: Layer already exists
d08e6b97bf21: Layer already exists
791896ea0949: Pushed
640645f1878b: Pushed
3054497613e6: Layer already exists
d35dc7f4c79e: Layer already exists
dabfe5b2ea81: Layer already exists
5e6a409f30b6: Layer already exists
d222b75e7c8c: Pushed
b28e321914c8: Pushed
20211002124334: digest: sha256:ed13aef81494bd85134a92db6119708c986b99a1f942cff69996c73fd69161d1 size: 4311

> Task :sdks:java:testing:load-tests:run
Oct 02, 2021 12:45:35 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Oct 02, 2021 12:45:35 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 195 files. Enable logging at DEBUG level to see which files will be staged.
Oct 02, 2021 12:45:36 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Oct 02, 2021 12:45:36 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Oct 02, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 195 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Oct 02, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.m2/repository/org/apache/beam/beam-vendor-grpc-1_36_0/0.2/beam-vendor-grpc-1_36_0-0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-grpc-1_36_0-0.2-cBdYf-JPXEZ1XnZqk0m4-VlBsCKU_yOoV0qLHUFCl2c.jar
Oct 02, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 194 files cached, 1 files newly uploaded in 1 seconds
Oct 02, 2021 12:45:40 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Oct 02, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <108949 bytes, hash ce3df63d83166cf44021866553c29fac922b02d6e869a1bf66db1c0754bb456d> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-zj32PYMWbPRAIYZlU8KfrJIrAtboaaG_ZtscB1S7RW0.pb
Oct 02, 2021 12:45:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Oct 02, 2021 12:45:42 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@31dfc6f5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@37b52340, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@663bb8ef, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2f4e40d7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@60e9c3a5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7e5843db, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@459f703f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@188ac8a3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3650d4fc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@aaa0f76, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@340a8894, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7a8b9166, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4acc5dff, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@10c72a6f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@70e94ecb, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@56cfe111, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7e446d92, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@57f9b467, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6d5c2745, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@44b29496]
Oct 02, 2021 12:45:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Oct 02, 2021 12:45:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Oct 02, 2021 12:45:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Oct 02, 2021 12:45:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Oct 02, 2021 12:45:42 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@32b0876c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2aaf152b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@39bbd9e0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@27fe9713, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@11c3ff67, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4397a639, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@24841372, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77114efe, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@79a7bfbc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77f68df, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3e4e4c1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7e7f3cfd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ae126d1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@46a488c2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6242ae3b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@65ddee5a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@56399b9e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@34b9eb03, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43fda8d9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@49d831c2]
Oct 02, 2021 12:45:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Oct 02, 2021 12:45:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Oct 02, 2021 12:45:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Oct 02, 2021 12:45:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Oct 02, 2021 12:45:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Oct 02, 2021 12:45:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Oct 02, 2021 12:45:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Oct 02, 2021 12:45:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Oct 02, 2021 12:45:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Oct 02, 2021 12:45:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Oct 02, 2021 12:45:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Oct 02, 2021 12:45:42 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.34.0-SNAPSHOT
Oct 02, 2021 12:45:43 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-10-02_05_45_42-13751100958647493540?project=apache-beam-testing
Oct 02, 2021 12:45:43 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-10-02_05_45_42-13751100958647493540
Oct 02, 2021 12:45:43 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-10-02_05_45_42-13751100958647493540
Oct 02, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-10-02T12:45:49.687Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-10-cx6n. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Oct 02, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-02T12:45:54.585Z: Worker configuration: e2-standard-2 in us-central1-a.
Oct 02, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-02T12:45:55.331Z: Expanding SplittableParDo operations into optimizable parts.
Oct 02, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-02T12:45:55.358Z: Expanding CollectionToSingleton operations into optimizable parts.
Oct 02, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-02T12:45:55.411Z: Expanding CoGroupByKey operations into optimizable parts.
Oct 02, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-02T12:45:55.478Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Oct 02, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-02T12:45:55.500Z: Expanding GroupByKey operations into streaming Read/Write steps
Oct 02, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-02T12:45:55.550Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Oct 02, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-02T12:45:55.651Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Oct 02, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-02T12:45:55.686Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Oct 02, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-02T12:45:55.714Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Oct 02, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-02T12:45:55.741Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Oct 02, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-02T12:45:55.772Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Oct 02, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-02T12:45:55.804Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Oct 02, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-02T12:45:55.835Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Oct 02, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-02T12:45:55.858Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Oct 02, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-02T12:45:55.883Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Oct 02, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-02T12:45:55.916Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Oct 02, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-02T12:45:55.944Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Oct 02, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-02T12:45:55.970Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Oct 02, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-02T12:45:55.992Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Oct 02, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-02T12:45:56.015Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Oct 02, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-02T12:45:56.048Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Oct 02, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-02T12:45:56.089Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Oct 02, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-02T12:45:56.116Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Oct 02, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-02T12:45:56.145Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Oct 02, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-02T12:45:56.177Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Oct 02, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-02T12:45:56.212Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Oct 02, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-02T12:45:56.237Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Oct 02, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-02T12:45:56.261Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Oct 02, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-02T12:45:56.289Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Oct 02, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-02T12:45:56.621Z: Starting 5 ****s in us-central1-a...
Oct 02, 2021 12:46:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-02T12:46:28.183Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Oct 02, 2021 12:46:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-02T12:46:37.891Z: Autoscaling: Raised the number of ****s to 1 so that the pipeline can catch up with its backlog and keep up with its input rate.
Oct 02, 2021 12:46:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-02T12:46:37.935Z: Resized **** pool to 1, though goal was 5.  This could be a quota issue.
Oct 02, 2021 12:46:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-02T12:46:48.264Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Oct 02, 2021 12:47:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-02T12:47:40.912Z: Workers have started successfully.
Oct 02, 2021 12:47:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-02T12:47:40.939Z: Workers have started successfully.
Oct 02, 2021 4:00:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-02T16:00:24.861Z: Cancel request is committed for workflow job: 2021-10-02_05_45_42-13751100958647493540.
Oct 02, 2021 4:00:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-02T16:00:24.924Z: Cleaning up.
Oct 02, 2021 4:00:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-02T16:00:24.996Z: Stopping **** pool...
Oct 02, 2021 4:00:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-02T16:00:25.060Z: Stopping **** pool...
Oct 02, 2021 4:02:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-02T16:02:50.544Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Oct 02, 2021 4:02:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-02T16:02:50.575Z: Worker pool stopped.
Oct 02, 2021 4:02:57 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-10-02_05_45_42-13751100958647493540 finished with status CANCELLED.
Load test results for test (ID): d6cdfdb6-008c-4f4f-9cfa-5f603b82dbc6 and timestamp: 2021-10-02T12:45:36.151000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11129.238
dataflow_v2_java11_total_bytes_count             2.34195967E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211002124334
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ed13aef81494bd85134a92db6119708c986b99a1f942cff69996c73fd69161d1
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211002124334]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ed13aef81494bd85134a92db6119708c986b99a1f942cff69996c73fd69161d1]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211002124334] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ed13aef81494bd85134a92db6119708c986b99a1f942cff69996c73fd69161d1])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ed13aef81494bd85134a92db6119708c986b99a1f942cff69996c73fd69161d1
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ed13aef81494bd85134a92db6119708c986b99a1f942cff69996c73fd69161d1
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ed13aef81494bd85134a92db6119708c986b99a1f942cff69996c73fd69161d1].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 40s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/43klybgxzcpt2

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #106

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/106/display/redirect?page=changes>

Changes:

[stranniknm] [BEAM-12951]: Create initial structure for Playground application

[dpcollins] [BEAM-12908] Change to use PubsubSignal for information propagation so

[Robert Bradshaw] Preserve more types in transform replacement.

[rohde.samuel] Fix BEAM-12984

[Robert Bradshaw] Update test to reflect preserved type hint.

[noreply] Update Beam glossary (#15619)

[noreply] [BEAM-11985] Python Bigtable - Implement IO Request Count metrics

[noreply] [BEAM-9918] Make TryCrossLanguage match non Try API (#15633)

[noreply] [BEAM-12957] Add support for pyarrow 5.x (#15588)


------------------------------------------
[...truncated 50.14 KB...]
3d2dc6a1e0cc: Pushed
6e6702e72581: Pushed
747315a15ae9: Pushed
874ad65f91ea: Layer already exists
0fc2498b65e5: Layer already exists
d08e6b97bf21: Layer already exists
3054497613e6: Layer already exists
d35dc7f4c79e: Layer already exists
dabfe5b2ea81: Layer already exists
2292b797a958: Pushed
3eda0e4baae4: Pushed
5e6a409f30b6: Layer already exists
806581f04ff7: Pushed
85a429cff7e2: Pushed
20211001124333: digest: sha256:dd477c33c5beb826333aa466e18538363e2c3fe7f2cf4a85ac8e44997f97253b size: 4311

> Task :sdks:java:testing:load-tests:run
Oct 01, 2021 12:45:23 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Oct 01, 2021 12:45:24 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 195 files. Enable logging at DEBUG level to see which files will be staged.
Oct 01, 2021 12:45:24 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Oct 01, 2021 12:45:24 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Oct 01, 2021 12:45:27 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 195 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Oct 01, 2021 12:45:28 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 195 files cached, 0 files newly uploaded in 0 seconds
Oct 01, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Oct 01, 2021 12:45:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <108949 bytes, hash 1b4010c853ec5001e5a693f58392bd22dd98accd9e44d7b7959cfc9a7a297b01> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-G0AQyFPsUAHlppP1g5K9It2YrM2eRNe3lZz8mnopewE.pb
Oct 01, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Oct 01, 2021 12:45:29 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2a869a16, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@ae202c6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@46aa712c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6ada9c0c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7412ed6b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6e7c351d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b4a0aef, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@45cec376, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26d41711, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@298f0a0b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2b960a7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@31dfc6f5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@37b52340, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@663bb8ef, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2f4e40d7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@60e9c3a5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7e5843db, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@459f703f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@188ac8a3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3650d4fc]
Oct 01, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Oct 01, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Oct 01, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Oct 01, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Oct 01, 2021 12:45:29 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@441b8382, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1df1ced0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5349b246, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@32b0876c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2aaf152b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@39bbd9e0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@27fe9713, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@11c3ff67, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4397a639, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@24841372, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77114efe, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@79a7bfbc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77f68df, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3e4e4c1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7e7f3cfd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ae126d1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@46a488c2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6242ae3b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@65ddee5a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@56399b9e]
Oct 01, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Oct 01, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Oct 01, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Oct 01, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Oct 01, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Oct 01, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Oct 01, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Oct 01, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Oct 01, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Oct 01, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Oct 01, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Oct 01, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.34.0-SNAPSHOT
Oct 01, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-10-01_05_45_30-5041997696408205737?project=apache-beam-testing
Oct 01, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-10-01_05_45_30-5041997696408205737
Oct 01, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-10-01_05_45_30-5041997696408205737
Oct 01, 2021 12:45:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-10-01T12:45:36.699Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-10-772t. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Oct 01, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-01T12:45:41.362Z: Worker configuration: e2-standard-2 in us-central1-a.
Oct 01, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-01T12:45:42.196Z: Expanding SplittableParDo operations into optimizable parts.
Oct 01, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-01T12:45:42.231Z: Expanding CollectionToSingleton operations into optimizable parts.
Oct 01, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-01T12:45:42.299Z: Expanding CoGroupByKey operations into optimizable parts.
Oct 01, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-01T12:45:42.364Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Oct 01, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-01T12:45:42.402Z: Expanding GroupByKey operations into streaming Read/Write steps
Oct 01, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-01T12:45:42.460Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Oct 01, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-01T12:45:42.551Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Oct 01, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-01T12:45:42.584Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Oct 01, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-01T12:45:42.612Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Oct 01, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-01T12:45:42.636Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Oct 01, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-01T12:45:42.676Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Oct 01, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-01T12:45:42.707Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Oct 01, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-01T12:45:42.729Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Oct 01, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-01T12:45:42.752Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Oct 01, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-01T12:45:42.779Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Oct 01, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-01T12:45:42.817Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Oct 01, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-01T12:45:42.869Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Oct 01, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-01T12:45:42.903Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Oct 01, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-01T12:45:42.930Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Oct 01, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-01T12:45:42.966Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Oct 01, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-01T12:45:42.992Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Oct 01, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-01T12:45:43.023Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Oct 01, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-01T12:45:43.056Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Oct 01, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-01T12:45:43.086Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Oct 01, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-01T12:45:43.153Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Oct 01, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-01T12:45:43.193Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Oct 01, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-01T12:45:43.226Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Oct 01, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-01T12:45:43.252Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Oct 01, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-01T12:45:43.287Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Oct 01, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-01T12:45:43.698Z: Starting 5 ****s in us-central1-a...
Oct 01, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-01T12:45:49.563Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Oct 01, 2021 12:46:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-01T12:46:24.841Z: Autoscaling: Raised the number of ****s to 2 so that the pipeline can catch up with its backlog and keep up with its input rate.
Oct 01, 2021 12:46:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-01T12:46:24.900Z: Resized **** pool to 2, though goal was 5.  This could be a quota issue.
Oct 01, 2021 12:46:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-01T12:46:35.247Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Oct 01, 2021 12:47:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-01T12:47:26.897Z: Workers have started successfully.
Oct 01, 2021 12:47:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-01T12:47:26.935Z: Workers have started successfully.
Oct 01, 2021 1:59:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-01T13:59:13.320Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Oct 01, 2021 1:59:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-01T13:59:17.037Z: Worker configuration: e2-standard-2 in us-central1-a.
Oct 01, 2021 1:59:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-01T13:59:21.992Z: Workers have started successfully.
Oct 01, 2021 4:00:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-01T16:00:32.460Z: Cancel request is committed for workflow job: 2021-10-01_05_45_30-5041997696408205737.
Oct 01, 2021 4:00:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-01T16:00:32.529Z: Cleaning up.
Oct 01, 2021 4:00:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-01T16:00:32.672Z: Stopping **** pool...
Oct 01, 2021 4:00:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-01T16:00:32.718Z: Stopping **** pool...
Oct 01, 2021 4:02:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-01T16:02:51.220Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Oct 01, 2021 4:02:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-10-01T16:02:51.266Z: Worker pool stopped.
Oct 01, 2021 4:02:57 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-10-01_05_45_30-5041997696408205737 finished with status CANCELLED.
Load test results for test (ID): fb59a485-203f-4f1e-9098-c28e1c02c18f and timestamp: 2021-10-01T12:45:24.398000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11531.837
dataflow_v2_java11_total_bytes_count             2.43221006E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211001124333
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:dd477c33c5beb826333aa466e18538363e2c3fe7f2cf4a85ac8e44997f97253b
Deleted: sha256:c01e8807dba0aab5a842cbf698b778c444509fcd0dec2b2a119ca3add85024bd
Deleted: sha256:31dfd48fe9b93e69db6ea66c559a9b21d73a2f725187c45bda42f1a43fbc8e0d
Deleted: sha256:f431a51fde4110ef9eb0558f4033d8d0c629c8d3379ac5ecc74771c7bae11666
Deleted: sha256:0fe799e990d18863c8c3f8005b89c6a2fcc034205fcc41d08d58836ae9b735c2
Deleted: sha256:5507d971e203d2665cbbed5aa625501a340d9b7b2f430b7d0586afad3ddda6ea
Deleted: sha256:d17ecca3fe50e37de306278d3e0015691cc182fc06947e50a4f210f567753996
Deleted: sha256:42682df17fb6d59c71a2a771fa9d5dbb8a1a4dccdb7b3f1ad183cc2ae4162544
Deleted: sha256:a67af73edaab39b6681540f60ced171487a5819ae79d97d9b354e2d4cbe2e5cb
Deleted: sha256:11591c7607b82920587f37c2c7ffc783b3ae5291e6a9ae7635e345484db9c966
Deleted: sha256:87a161edbb103585e5d0d343db07c80193c8efeb4c50f0feb7a052c4817901ad
Deleted: sha256:77c9bc2c908ea28df6c216590ae706b8f984752cceaddb9369c240269310a123
Deleted: sha256:9b5a76fc3eb6551048994037c0ceefb28a1c0cf5e1156af37a286c1c3a7b7082
Deleted: sha256:4f66acc753892fbeac4a54e7c974e476788f9876b87c5d5094d670d99b9e551e
Deleted: sha256:1bea7a1dc3b875a77fb3cf3af62a3fb60756fda31895a0e7b8c5387c17f50694
Deleted: sha256:a860b823499c0bb1181bbf4c9668eeb03655778b5c7a491231b67efe37f04d26
Deleted: sha256:342e0bc8c4f7bba121a3fedc221e43ff54e66e9189167c2d0f486dccb7b278ca
Deleted: sha256:04b34b54af1820ef757b047ec9ae096ebbb697718c5177f7c9dc56fe5428e5ba
Deleted: sha256:d3f85266edb2d64348af2a7b0bd960d8a9787994986521d03dd7c490600996df
Deleted: sha256:43b8cb7f9a927f016e291300173b6076a4c7af606e3af951e3ede2ce31e5622f
Deleted: sha256:9355f50c015c060021d2f4ba8a97685aa982b83d6995365a87a3924dff3a0a2e
Deleted: sha256:cd7f70c4017ea618dedaf045bf5f9c74d7a5608e99baf91d4ef4b35e030800ae
Deleted: sha256:3e27d3af5e7a0a467e94d34c5687ab3f678d03c40bef575f16aae1b6dc2707bb
Deleted: sha256:d93a6e9dd72cd6957e527137bcc6661826942f253efaefc568cc3a373edc81cd
Deleted: sha256:5f8b9633bfbbafb3084c1db94e0361c0042e02ca12d73f2e5353b969ca475e85
Deleted: sha256:0de755e6293192f8c30be83e8031207668e7ad0200d7d66db6e8cf710d5b72bd
Deleted: sha256:b4ed11189636cc3cfeb2fbb9c12a5a908fd3c9ef12f37ef777749bcf33de7c93
Deleted: sha256:2f984c5689db1c797bc425eb8f3289934dfd45b1b503bde5555dabc2b2dc9297
Deleted: sha256:ed4e6008df64cf5464d9421fd9dce7b47f257e8ffdc8a8149f911628563ab1ac
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211001124333]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:dd477c33c5beb826333aa466e18538363e2c3fe7f2cf4a85ac8e44997f97253b]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211001124333] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:dd477c33c5beb826333aa466e18538363e2c3fe7f2cf4a85ac8e44997f97253b])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:dd477c33c5beb826333aa466e18538363e2c3fe7f2cf4a85ac8e44997f97253b
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:dd477c33c5beb826333aa466e18538363e2c3fe7f2cf4a85ac8e44997f97253b
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:dd477c33c5beb826333aa466e18538363e2c3fe7f2cf4a85ac8e44997f97253b].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 42s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/m6kjyg6hv5jv4

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #105

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/105/display/redirect?page=changes>

Changes:

[kileysok] Update windmill state map to resolve *IfAbsent methods immediately

[samuelw] [BEAM-12942] Validate pubsub messages before they are published in

[clairem] [BEAM-12628] make useReflectApi default to true

[noreply] [BEAM-12856] Change hard-coded limits for reading from a UnboundedReader

[piotr.szczepanik] [BEAM-12356] Fixed last non-cached usage of DatasetService in BigQuery

[noreply] [BEAM-12977] Translates Reshuffle in Portable Mode with Samza native

[noreply] [BEAM-12982] Help users debug which JvmInitializer is running and when.


------------------------------------------
[...truncated 53.44 KB...]
INFO: Adding Read input/StripIds as step s2
Sep 30, 2021 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Sep 30, 2021 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Sep 30, 2021 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Sep 30, 2021 12:45:24 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@32b0876c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2aaf152b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@39bbd9e0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@27fe9713, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@11c3ff67, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4397a639, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@24841372, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77114efe, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@79a7bfbc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77f68df, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3e4e4c1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7e7f3cfd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ae126d1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@46a488c2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6242ae3b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@65ddee5a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@56399b9e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@34b9eb03, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43fda8d9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@49d831c2]
Sep 30, 2021 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Sep 30, 2021 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Sep 30, 2021 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Sep 30, 2021 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Sep 30, 2021 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Sep 30, 2021 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Sep 30, 2021 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Sep 30, 2021 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Sep 30, 2021 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Sep 30, 2021 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Sep 30, 2021 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Sep 30, 2021 12:45:24 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.34.0-SNAPSHOT
Sep 30, 2021 12:45:25 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-09-30_05_45_25-10486700576588734041?project=apache-beam-testing
Sep 30, 2021 12:45:25 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-09-30_05_45_25-10486700576588734041
Sep 30, 2021 12:45:25 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-09-30_05_45_25-10486700576588734041
Sep 30, 2021 12:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-09-30T12:45:33.878Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-09-qp0w. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Sep 30, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-30T12:45:38.763Z: Worker configuration: e2-standard-2 in us-central1-a.
Sep 30, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-30T12:45:39.569Z: Expanding SplittableParDo operations into optimizable parts.
Sep 30, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-30T12:45:39.611Z: Expanding CollectionToSingleton operations into optimizable parts.
Sep 30, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-30T12:45:39.684Z: Expanding CoGroupByKey operations into optimizable parts.
Sep 30, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-30T12:45:39.758Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Sep 30, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-30T12:45:39.786Z: Expanding GroupByKey operations into streaming Read/Write steps
Sep 30, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-30T12:45:39.833Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Sep 30, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-30T12:45:39.942Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Sep 30, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-30T12:45:39.969Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Sep 30, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-30T12:45:40.000Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Sep 30, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-30T12:45:40.026Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Sep 30, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-30T12:45:40.061Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Sep 30, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-30T12:45:40.085Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Sep 30, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-30T12:45:40.107Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Sep 30, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-30T12:45:40.144Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Sep 30, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-30T12:45:40.177Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Sep 30, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-30T12:45:40.221Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Sep 30, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-30T12:45:40.257Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Sep 30, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-30T12:45:40.291Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Sep 30, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-30T12:45:40.330Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Sep 30, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-30T12:45:40.360Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Sep 30, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-30T12:45:40.390Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Sep 30, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-30T12:45:40.445Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Sep 30, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-30T12:45:40.585Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Sep 30, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-30T12:45:40.723Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Sep 30, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-30T12:45:40.778Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Sep 30, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-30T12:45:40.821Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Sep 30, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-30T12:45:40.849Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Sep 30, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-30T12:45:40.883Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Sep 30, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-30T12:45:40.913Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Sep 30, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-30T12:45:41.270Z: Starting 5 ****s in us-central1-a...
Sep 30, 2021 12:46:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-30T12:46:06.036Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Sep 30, 2021 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-30T12:46:17.937Z: Autoscaling: Raised the number of ****s to 1 so that the pipeline can catch up with its backlog and keep up with its input rate.
Sep 30, 2021 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-30T12:46:17.963Z: Resized **** pool to 1, though goal was 5.  This could be a quota issue.
Sep 30, 2021 12:46:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-30T12:46:28.302Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Sep 30, 2021 12:47:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-30T12:47:26.885Z: Workers have started successfully.
Sep 30, 2021 12:47:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-30T12:47:26.919Z: Workers have started successfully.
Sep 30, 2021 2:51:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-30T14:51:41.426Z: Staged package amazon-kinesis-client-1.14.2-qtBh3OuKuLE7S26ITTJzYsoYfG-mWjqbGmv3VJPhec8.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/amazon-kinesis-client-1.14.2-qtBh3OuKuLE7S26ITTJzYsoYfG-mWjqbGmv3VJPhec8.jar' is inaccessible.
Sep 30, 2021 2:51:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-30T14:51:41.579Z: Staged package annotations-4.1.1.4-unNOHoTAnWFa9qCdMwNLTwRC-Hct7BIO-zdthqVlrhU.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/annotations-4.1.1.4-unNOHoTAnWFa9qCdMwNLTwRC-Hct7BIO-zdthqVlrhU.jar' is inaccessible.
Sep 30, 2021 2:51:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-30T14:51:41.803Z: Staged package avro-1.8.2-91SggwzmelqfpnpU7BXRA-8V4chQ17Jvr3tkfu3cgtM.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/avro-1.8.2-91SggwzmelqfpnpU7BXRA-8V4chQ17Jvr3tkfu3cgtM.jar' is inaccessible.
Sep 30, 2021 2:51:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-30T14:51:42.460Z: Staged package beam-vendor-bytebuddy-1_11_0-0.1-SneHjzQgCpxxLkvyN8dugqO_sTzwXJ2995kqFu4l2i4.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-bytebuddy-1_11_0-0.1-SneHjzQgCpxxLkvyN8dugqO_sTzwXJ2995kqFu4l2i4.jar' is inaccessible.
Sep 30, 2021 2:51:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-30T14:51:42.545Z: Staged package beam-vendor-guava-26_0-jre-0.1-8quizfXTrmcvJyOXi3zWDFtP5EofgqjksY5sIbsBqws.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-guava-26_0-jre-0.1-8quizfXTrmcvJyOXi3zWDFtP5EofgqjksY5sIbsBqws.jar' is inaccessible.
Sep 30, 2021 2:51:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-30T14:51:42.635Z: Staged package checker-compat-qual-2.5.5-EdE0skXpysxHRRTS1mtbhhj4A5oUZc3FW7wLNOAAi3o.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/checker-compat-qual-2.5.5-EdE0skXpysxHRRTS1mtbhhj4A5oUZc3FW7wLNOAAi3o.jar' is inaccessible.
Sep 30, 2021 2:51:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-30T14:51:42.840Z: Staged package commons-lang3-3.9-3i4dzc8--ReozoWGYaBnJqmpRPKOM61_ngi-pE3DwjA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/commons-lang3-3.9-3i4dzc8--ReozoWGYaBnJqmpRPKOM61_ngi-pE3DwjA.jar' is inaccessible.
Sep 30, 2021 2:51:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-30T14:51:42.939Z: Staged package commons-logging-1.2-2t3qHqC-D1aXirMAa4rJKDSv7vvZt-TmMW_KV98PpjY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/commons-logging-1.2-2t3qHqC-D1aXirMAa4rJKDSv7vvZt-TmMW_KV98PpjY.jar' is inaccessible.
Sep 30, 2021 2:51:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-30T14:51:43.094Z: Staged package commons-math3-3.6.1-HlbXsFjSi2Wr0la4RY44hbZ0wdWI-kPNfRy7nH7yswg.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/commons-math3-3.6.1-HlbXsFjSi2Wr0la4RY44hbZ0wdWI-kPNfRy7nH7yswg.jar' is inaccessible.
Sep 30, 2021 2:51:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-30T14:51:43.205Z: Staged package datastore-v1-proto-client-1.6.3-8GhVpKiAAK6wztlpNJ48AH4jyyB1KYo-5qJLXiBC6cw.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/datastore-v1-proto-client-1.6.3-8GhVpKiAAK6wztlpNJ48AH4jyyB1KYo-5qJLXiBC6cw.jar' is inaccessible.
Sep 30, 2021 2:51:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-30T14:51:44.370Z: Staged package gson-2.8.6-yPtIOQVNKAswM_gA0fWpfeLwKOuLoutFitKH5Tbz8l8.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/gson-2.8.6-yPtIOQVNKAswM_gA0fWpfeLwKOuLoutFitKH5Tbz8l8.jar' is inaccessible.
Sep 30, 2021 2:51:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-30T14:51:44.493Z: Staged package hamcrest-2.1-upOy46ViMiukMvChtTrdzFXLGIJTMZoCDtd_gk5pIFA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/hamcrest-2.1-upOy46ViMiukMvChtTrdzFXLGIJTMZoCDtd_gk5pIFA.jar' is inaccessible.
Sep 30, 2021 2:51:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-30T14:51:44.545Z: Staged package hamcrest-core-2.1-4JEJ5UoonYhQa5v-yYfd0Zn0IXyUZBMmaDUbmk8Avuk.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/hamcrest-core-2.1-4JEJ5UoonYhQa5v-yYfd0Zn0IXyUZBMmaDUbmk8Avuk.jar' is inaccessible.
Sep 30, 2021 2:51:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-30T14:51:44.665Z: Staged package j2objc-annotations-1.3-Ia8wySJnvWEiwOC00gzMtmQaN-r5VsZUDsRx1YTmSns.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/j2objc-annotations-1.3-Ia8wySJnvWEiwOC00gzMtmQaN-r5VsZUDsRx1YTmSns.jar' is inaccessible.
Sep 30, 2021 2:51:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-30T14:51:44.748Z: Staged package jackson-core-asl-1.9.13-RAqctcqVshX5U9OiCmsaENofCbUpqd3qX4pJBd2rT1o.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/jackson-core-asl-1.9.13-RAqctcqVshX5U9OiCmsaENofCbUpqd3qX4pJBd2rT1o.jar' is inaccessible.
Sep 30, 2021 2:51:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-30T14:51:44.839Z: Staged package jackson-mapper-asl-1.9.13-dOegenby7breKTEqWi68z6AZEovAIezjhW12GX6b4MI.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/jackson-mapper-asl-1.9.13-dOegenby7breKTEqWi68z6AZEovAIezjhW12GX6b4MI.jar' is inaccessible.
Sep 30, 2021 2:51:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-30T14:51:44.897Z: Staged package jakarta.activation-api-1.2.2-oYepORA671hJp6-EvX4nvi0SDEEK8pFDc3X_4GH08J0.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/jakarta.activation-api-1.2.2-oYepORA671hJp6-EvX4nvi0SDEEK8pFDc3X_4GH08J0.jar' is inaccessible.
Sep 30, 2021 2:51:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-30T14:51:44.951Z: Staged package jakarta.xml.bind-api-2.3.3-wEU59HLppt0MdoXqgtZ3KCJpq457rKLhRQDjgeDGzsU.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/jakarta.xml.bind-api-2.3.3-wEU59HLppt0MdoXqgtZ3KCJpq457rKLhRQDjgeDGzsU.jar' is inaccessible.
Sep 30, 2021 2:51:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-30T14:51:45.006Z: Staged package javax.annotation-api-1.3.2-4EulGVvNVV3JVlD3zGFNFR5LzVLSmhC4qiGX86uJq5s.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/javax.annotation-api-1.3.2-4EulGVvNVV3JVlD3zGFNFR5LzVLSmhC4qiGX86uJq5s.jar' is inaccessible.
Sep 30, 2021 2:51:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-30T14:51:45.076Z: Staged package json-20200518-D_MObagvFa-vDna9bplqerVWaFEWaoP5c8ZKmg-UbS0.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/json-20200518-D_MObagvFa-vDna9bplqerVWaFEWaoP5c8ZKmg-UbS0.jar' is inaccessible.
Sep 30, 2021 2:51:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-30T14:51:45.122Z: Staged package jsr305-3.0.2-dmrSoHg_JoeWLIrXTO7MOKKLn3Ki0IXuQ4t4E-ko0Mc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/jsr305-3.0.2-dmrSoHg_JoeWLIrXTO7MOKKLn3Ki0IXuQ4t4E-ko0Mc.jar' is inaccessible.
Sep 30, 2021 2:51:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-09-30T14:51:45.986Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Sep 30, 2021 4:00:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-30T16:00:35.199Z: Cancel request is committed for workflow job: 2021-09-30_05_45_25-10486700576588734041.
Sep 30, 2021 4:00:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-30T16:00:35.415Z: Cleaning up.
Sep 30, 2021 4:00:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-30T16:00:35.501Z: Stopping **** pool...
Sep 30, 2021 4:00:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-30T16:00:35.559Z: Stopping **** pool...
Sep 30, 2021 4:03:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-30T16:03:04.190Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Sep 30, 2021 4:03:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-30T16:03:04.230Z: Worker pool stopped.
Sep 30, 2021 4:03:11 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-09-30_05_45_25-10486700576588734041 finished with status CANCELLED.
Load test results for test (ID): c0aa9c2c-2d92-4cee-882b-2e297f9c60aa and timestamp: 2021-09-30T12:45:20.300000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11556.755
dataflow_v2_java11_total_bytes_count             2.99148595E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210930124333
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:4557d1d81f235f0b2d1d85ff19eeb6e31b86e1e220d14d2d4d1b97b0484a4fc7
Deleted: sha256:4a921f3ddf13e4dde2bf38841c800f227027888eec37f41d35793217b0d911b6
Deleted: sha256:1483bac0782df00e251f4a7ac101080f704f38831282d1bf3a082e4267b6a863
Deleted: sha256:88fe30db21d041f8a1ab60dd0b4d4e8018f335cb3236afd0735437a02c18770e
Deleted: sha256:bb0393cfcb74572a3aa4a152cd1a0d1251b7a4510c26359b1407babf77645b1d
Deleted: sha256:77e421a2e2054c1c73c6e0272d0cde0edfe748ac03063c5382d7071edcb9d184
Deleted: sha256:b03df5f20755da811b68c034e3b920590d5afd4ff76db0397b085ade4b6297ea
Deleted: sha256:ec44ead991e8a26087a06d0460bb94a055593c9a6830b737b1f9a7b3d4923fbb
Deleted: sha256:d1b7a8a07febb4057f92dc8951e5583accb2792e793fecfab0a41afad2b53b61
Deleted: sha256:a06eaa286ccf1b79502e7377c41dae9ec0914ff2f8566de4dc6d235d55c1e540
Deleted: sha256:aa19a9a178ad436341d3fad631ca1de8e60f87cd9def0adebdaf342e767bfcf5
Deleted: sha256:d3affdddaf41ae8ae76b88fea030ee028eea135a6fb10e5f107cff13f2e97335
Deleted: sha256:70a89a6ac16cdfcd9de9a74b6f83b8aaa57d6d518b4f500a327321f70b29bc99
Deleted: sha256:e57de2b7f10f7ffb5a09929446b4f1a6c719683f58b22785bf810cf358cfeca3
Deleted: sha256:96c48d3376f31852f54245221807c751c35f6e7454c5b06ec4888e7945a4a1d4
Deleted: sha256:649c4d442a244a7907ee8d1a9302a52b645a3a8ff290238d023eeeed3517aa8a
Deleted: sha256:1c2304e0befd96984e05865cb4e650e1d036eadfc1de279edcf1c0a339de1a05
Deleted: sha256:c028a26b3d101b1bf005f819338f2dbac406704988c496aeb8bebf564a3fb36e
Deleted: sha256:c4301378493c8c4a7590ed9c01979a18eb41eeb72b1226bce1c98658a7c6192d
Deleted: sha256:62e6a0d8c80089c5d27bed21b1a63872f1a98d679ad72ac23aa32bab77f73a5a
Deleted: sha256:dc5c455a042545bac84356892998f30f257649a5de2e8f5af6330e94a45198dd
Deleted: sha256:25b5859570cbeeace215ba45d6035b802401e86fde4023ce77c781c7475b98e9
Deleted: sha256:71e9d045fcbc4b4e5610e84f6d3fd7566a0f53cf6c3ff67396050991212cf144
Deleted: sha256:0ac16a6a0ef806998fa7fd784f02f140c45251b666bd31e1270d6fac04db8c87
Deleted: sha256:7ddcb572f9f31ece0acebf3f1658b6f6a8be8e96ed56176b6f412caa16d4acc8
Deleted: sha256:80b31ec477f265a13380faf29ab4dda282e8d21cf0830d2610ae244340df1456
Deleted: sha256:0b6b291060d2f285a1c5f03263d18a976171f4267086fd714ababd5bcddd10d7
Deleted: sha256:8addd63e2f780eeb0e61d7ddb9518b17a9a0acb6aea405cb9afd1e2445a66e3d
Deleted: sha256:591ea11b3c177685ef5c13b3a4805a434476f4d6115117d5e14ee9bdb967bc86
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210930124333]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:4557d1d81f235f0b2d1d85ff19eeb6e31b86e1e220d14d2d4d1b97b0484a4fc7]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210930124333] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:4557d1d81f235f0b2d1d85ff19eeb6e31b86e1e220d14d2d4d1b97b0484a4fc7])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:4557d1d81f235f0b2d1d85ff19eeb6e31b86e1e220d14d2d4d1b97b0484a4fc7
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:4557d1d81f235f0b2d1d85ff19eeb6e31b86e1e220d14d2d4d1b97b0484a4fc7
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:4557d1d81f235f0b2d1d85ff19eeb6e31b86e1e220d14d2d4d1b97b0484a4fc7].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 54s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/seu2yzalrbneu

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #104

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/104/display/redirect?page=changes>

Changes:

[chamikaramj] Sets the correct coder in 'ConstantTableDestinations' when using BQ

[Luke Cwik] [BEAM-12974] Migrate off of deprecated package for MockitoJUnitRunner

[noreply] [BEAM-12593] Verify DataFrame API on pandas 1.3 (with container update)

[noreply] [BEAM-12973] Print Go Test and Script info. (#15604)

[noreply] [BEAM-10913] - Installing and persisting Grafana plugin in kubernetes

[noreply] [BEAM-11129] Add namespace and key to portable display data (#15564)

[noreply] [BEAM-12906] Add a `dataframe` extra for installing a pandas version


------------------------------------------
[...truncated 84.97 KB...]
SEVERE: 2021-09-29T15:30:47.200Z: Staged package netty-codec-socks-4.1.52.Final-_sxXfgu3ch1naM_BhG7hcpEdFquA372ZldllelUdTqc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-codec-socks-4.1.52.Final-_sxXfgu3ch1naM_BhG7hcpEdFquA372ZldllelUdTqc.jar' is inaccessible.
Sep 29, 2021 3:30:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-29T15:30:47.237Z: Staged package netty-common-4.1.52.Final-DR8QHk231TDtBNfKvVfZXAdQ8Fm7az-t_tDIDUc0xxc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-common-4.1.52.Final-DR8QHk231TDtBNfKvVfZXAdQ8Fm7az-t_tDIDUc0xxc.jar' is inaccessible.
Sep 29, 2021 3:30:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-29T15:30:47.281Z: Staged package netty-handler-4.1.52.Final-LNwh-_rgSUDIWSkDl3nJP1H2GPXvshYHkxnc_PMqVcE.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-handler-4.1.52.Final-LNwh-_rgSUDIWSkDl3nJP1H2GPXvshYHkxnc_PMqVcE.jar' is inaccessible.
Sep 29, 2021 3:30:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-29T15:30:47.329Z: Staged package netty-handler-proxy-4.1.52.Final-6NB4Y860vUgG4lWh1TLIZQ8POy1NAGrW0hgS6UG0L9w.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-handler-proxy-4.1.52.Final-6NB4Y860vUgG4lWh1TLIZQ8POy1NAGrW0hgS6UG0L9w.jar' is inaccessible.
Sep 29, 2021 3:30:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-29T15:30:47.407Z: Staged package netty-resolver-4.1.52.Final-butyojOXnLRbC01tRsWkEzLFfAz4qAOx78YNu-y6CaY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-resolver-4.1.52.Final-butyojOXnLRbC01tRsWkEzLFfAz4qAOx78YNu-y6CaY.jar' is inaccessible.
Sep 29, 2021 3:30:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-29T15:30:47.464Z: Staged package netty-transport-4.1.52.Final-mj5vjA5V3jY-seoQ_ngXl-yjlOYhht8q4LTrK84LRUE.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-transport-4.1.52.Final-mj5vjA5V3jY-seoQ_ngXl-yjlOYhht8q4LTrK84LRUE.jar' is inaccessible.
Sep 29, 2021 3:30:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-29T15:30:47.507Z: Staged package opencensus-api-0.28.0-DBcj8_bTBhMjhFzouIs1_dpQCBLgp1uOtfzErYyHGpU.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/opencensus-api-0.28.0-DBcj8_bTBhMjhFzouIs1_dpQCBLgp1uOtfzErYyHGpU.jar' is inaccessible.
Sep 29, 2021 3:30:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-29T15:30:47.614Z: Staged package paranamer-2.8-aIyxGKYCHYGROOhVIIyVYDFoi-S0eiS7YVvsxjrO3wc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/paranamer-2.8-aIyxGKYCHYGROOhVIIyVYDFoi-S0eiS7YVvsxjrO3wc.jar' is inaccessible.
Sep 29, 2021 3:30:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-29T15:30:47.661Z: Staged package perfmark-api-0.23.0-xwW1wQwY_zAyuegXQrwvaw5WB_am38DIrQz_ddSRMEI.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/perfmark-api-0.23.0-xwW1wQwY_zAyuegXQrwvaw5WB_am38DIrQz_ddSRMEI.jar' is inaccessible.
Sep 29, 2021 3:30:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-29T15:30:48.078Z: Staged package snappy-java-1.1.8.4-JMTR_B6J4HgzGrj0AamcrWhZm95KLkUWBCy1SMUbHD4.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/snappy-java-1.1.8.4-JMTR_B6J4HgzGrj0AamcrWhZm95KLkUWBCy1SMUbHD4.jar' is inaccessible.
Sep 29, 2021 3:30:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-09-29T15:30:48.269Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Sep 29, 2021 3:33:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-09-29T15:33:47.866Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Sep 29, 2021 3:36:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-29T15:36:44.038Z: Staged package auto-value-annotations-1.8.1-N-wJtH1-01qZ0Tkn21yG_JBx9iD5Q-rV11cURpgxCFI.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/auto-value-annotations-1.8.1-N-wJtH1-01qZ0Tkn21yG_JBx9iD5Q-rV11cURpgxCFI.jar' is inaccessible.
Sep 29, 2021 3:36:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-29T15:36:44.943Z: Staged package commons-compress-1.21-auz9VFlyillWAc-gcljRMZcv_Dm0kutIvdWWV3ovJEo.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/commons-compress-1.21-auz9VFlyillWAc-gcljRMZcv_Dm0kutIvdWWV3ovJEo.jar' is inaccessible.
Sep 29, 2021 3:36:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-29T15:36:45.103Z: Staged package conscrypt-openjdk-uber-2.5.1-AfnHQstZKhUeLmK9U5eomAYoqWcAH82s1KpHRGeGhfM.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/conscrypt-openjdk-uber-2.5.1-AfnHQstZKhUeLmK9U5eomAYoqWcAH82s1KpHRGeGhfM.jar' is inaccessible.
Sep 29, 2021 3:36:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-29T15:36:47.117Z: Staged package netty-buffer-4.1.52.Final-QcoQNa7m4PgXdZfQp2EKWqLG5vrHRKyB_eTl5Y9jlnU.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-buffer-4.1.52.Final-QcoQNa7m4PgXdZfQp2EKWqLG5vrHRKyB_eTl5Y9jlnU.jar' is inaccessible.
Sep 29, 2021 3:36:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-29T15:36:47.154Z: Staged package netty-codec-4.1.52.Final-PH543M5-NT2GrH26LCUatc3dTrhuqoic_kIpeCuCjlI.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-codec-4.1.52.Final-PH543M5-NT2GrH26LCUatc3dTrhuqoic_kIpeCuCjlI.jar' is inaccessible.
Sep 29, 2021 3:36:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-29T15:36:47.194Z: Staged package netty-codec-http-4.1.52.Final-tdoo0OqHiwfTERrPXk-vQSiO8jArJkFBkP3Rp9lzyAw.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-codec-http-4.1.52.Final-tdoo0OqHiwfTERrPXk-vQSiO8jArJkFBkP3Rp9lzyAw.jar' is inaccessible.
Sep 29, 2021 3:36:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-29T15:36:47.242Z: Staged package netty-codec-http2-4.1.52.Final-61J6QtIonRLe5tR1PKJxWXije7tBxxkVzk6VOcSfk6E.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-codec-http2-4.1.52.Final-61J6QtIonRLe5tR1PKJxWXije7tBxxkVzk6VOcSfk6E.jar' is inaccessible.
Sep 29, 2021 3:36:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-29T15:36:47.300Z: Staged package netty-codec-socks-4.1.52.Final-_sxXfgu3ch1naM_BhG7hcpEdFquA372ZldllelUdTqc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-codec-socks-4.1.52.Final-_sxXfgu3ch1naM_BhG7hcpEdFquA372ZldllelUdTqc.jar' is inaccessible.
Sep 29, 2021 3:36:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-29T15:36:47.344Z: Staged package netty-common-4.1.52.Final-DR8QHk231TDtBNfKvVfZXAdQ8Fm7az-t_tDIDUc0xxc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-common-4.1.52.Final-DR8QHk231TDtBNfKvVfZXAdQ8Fm7az-t_tDIDUc0xxc.jar' is inaccessible.
Sep 29, 2021 3:36:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-29T15:36:47.385Z: Staged package netty-handler-4.1.52.Final-LNwh-_rgSUDIWSkDl3nJP1H2GPXvshYHkxnc_PMqVcE.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-handler-4.1.52.Final-LNwh-_rgSUDIWSkDl3nJP1H2GPXvshYHkxnc_PMqVcE.jar' is inaccessible.
Sep 29, 2021 3:36:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-29T15:36:47.474Z: Staged package netty-handler-proxy-4.1.52.Final-6NB4Y860vUgG4lWh1TLIZQ8POy1NAGrW0hgS6UG0L9w.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-handler-proxy-4.1.52.Final-6NB4Y860vUgG4lWh1TLIZQ8POy1NAGrW0hgS6UG0L9w.jar' is inaccessible.
Sep 29, 2021 3:36:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-29T15:36:47.510Z: Staged package netty-resolver-4.1.52.Final-butyojOXnLRbC01tRsWkEzLFfAz4qAOx78YNu-y6CaY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-resolver-4.1.52.Final-butyojOXnLRbC01tRsWkEzLFfAz4qAOx78YNu-y6CaY.jar' is inaccessible.
Sep 29, 2021 3:36:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-29T15:36:47.576Z: Staged package netty-transport-4.1.52.Final-mj5vjA5V3jY-seoQ_ngXl-yjlOYhht8q4LTrK84LRUE.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-transport-4.1.52.Final-mj5vjA5V3jY-seoQ_ngXl-yjlOYhht8q4LTrK84LRUE.jar' is inaccessible.
Sep 29, 2021 3:36:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-29T15:36:47.640Z: Staged package opencensus-api-0.28.0-DBcj8_bTBhMjhFzouIs1_dpQCBLgp1uOtfzErYyHGpU.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/opencensus-api-0.28.0-DBcj8_bTBhMjhFzouIs1_dpQCBLgp1uOtfzErYyHGpU.jar' is inaccessible.
Sep 29, 2021 3:36:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-29T15:36:47.739Z: Staged package paranamer-2.8-aIyxGKYCHYGROOhVIIyVYDFoi-S0eiS7YVvsxjrO3wc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/paranamer-2.8-aIyxGKYCHYGROOhVIIyVYDFoi-S0eiS7YVvsxjrO3wc.jar' is inaccessible.
Sep 29, 2021 3:36:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-29T15:36:47.782Z: Staged package perfmark-api-0.23.0-xwW1wQwY_zAyuegXQrwvaw5WB_am38DIrQz_ddSRMEI.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/perfmark-api-0.23.0-xwW1wQwY_zAyuegXQrwvaw5WB_am38DIrQz_ddSRMEI.jar' is inaccessible.
Sep 29, 2021 3:36:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-29T15:36:48.302Z: Staged package snappy-java-1.1.8.4-JMTR_B6J4HgzGrj0AamcrWhZm95KLkUWBCy1SMUbHD4.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/snappy-java-1.1.8.4-JMTR_B6J4HgzGrj0AamcrWhZm95KLkUWBCy1SMUbHD4.jar' is inaccessible.
Sep 29, 2021 3:36:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-09-29T15:36:48.446Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Sep 29, 2021 3:39:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-09-29T15:39:47.714Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Sep 29, 2021 3:42:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-29T15:42:44.077Z: Staged package auto-value-annotations-1.8.1-N-wJtH1-01qZ0Tkn21yG_JBx9iD5Q-rV11cURpgxCFI.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/auto-value-annotations-1.8.1-N-wJtH1-01qZ0Tkn21yG_JBx9iD5Q-rV11cURpgxCFI.jar' is inaccessible.
Sep 29, 2021 3:42:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-29T15:42:45.088Z: Staged package commons-compress-1.21-auz9VFlyillWAc-gcljRMZcv_Dm0kutIvdWWV3ovJEo.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/commons-compress-1.21-auz9VFlyillWAc-gcljRMZcv_Dm0kutIvdWWV3ovJEo.jar' is inaccessible.
Sep 29, 2021 3:42:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-29T15:42:45.231Z: Staged package conscrypt-openjdk-uber-2.5.1-AfnHQstZKhUeLmK9U5eomAYoqWcAH82s1KpHRGeGhfM.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/conscrypt-openjdk-uber-2.5.1-AfnHQstZKhUeLmK9U5eomAYoqWcAH82s1KpHRGeGhfM.jar' is inaccessible.
Sep 29, 2021 3:42:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-29T15:42:47.166Z: Staged package netty-buffer-4.1.52.Final-QcoQNa7m4PgXdZfQp2EKWqLG5vrHRKyB_eTl5Y9jlnU.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-buffer-4.1.52.Final-QcoQNa7m4PgXdZfQp2EKWqLG5vrHRKyB_eTl5Y9jlnU.jar' is inaccessible.
Sep 29, 2021 3:42:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-29T15:42:47.212Z: Staged package netty-codec-4.1.52.Final-PH543M5-NT2GrH26LCUatc3dTrhuqoic_kIpeCuCjlI.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-codec-4.1.52.Final-PH543M5-NT2GrH26LCUatc3dTrhuqoic_kIpeCuCjlI.jar' is inaccessible.
Sep 29, 2021 3:42:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-29T15:42:47.262Z: Staged package netty-codec-http-4.1.52.Final-tdoo0OqHiwfTERrPXk-vQSiO8jArJkFBkP3Rp9lzyAw.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-codec-http-4.1.52.Final-tdoo0OqHiwfTERrPXk-vQSiO8jArJkFBkP3Rp9lzyAw.jar' is inaccessible.
Sep 29, 2021 3:42:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-29T15:42:47.304Z: Staged package netty-codec-http2-4.1.52.Final-61J6QtIonRLe5tR1PKJxWXije7tBxxkVzk6VOcSfk6E.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-codec-http2-4.1.52.Final-61J6QtIonRLe5tR1PKJxWXije7tBxxkVzk6VOcSfk6E.jar' is inaccessible.
Sep 29, 2021 3:42:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-29T15:42:47.348Z: Staged package netty-codec-socks-4.1.52.Final-_sxXfgu3ch1naM_BhG7hcpEdFquA372ZldllelUdTqc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-codec-socks-4.1.52.Final-_sxXfgu3ch1naM_BhG7hcpEdFquA372ZldllelUdTqc.jar' is inaccessible.
Sep 29, 2021 3:42:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-29T15:42:47.377Z: Staged package netty-common-4.1.52.Final-DR8QHk231TDtBNfKvVfZXAdQ8Fm7az-t_tDIDUc0xxc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-common-4.1.52.Final-DR8QHk231TDtBNfKvVfZXAdQ8Fm7az-t_tDIDUc0xxc.jar' is inaccessible.
Sep 29, 2021 3:42:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-29T15:42:47.418Z: Staged package netty-handler-4.1.52.Final-LNwh-_rgSUDIWSkDl3nJP1H2GPXvshYHkxnc_PMqVcE.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-handler-4.1.52.Final-LNwh-_rgSUDIWSkDl3nJP1H2GPXvshYHkxnc_PMqVcE.jar' is inaccessible.
Sep 29, 2021 3:42:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-29T15:42:47.480Z: Staged package netty-handler-proxy-4.1.52.Final-6NB4Y860vUgG4lWh1TLIZQ8POy1NAGrW0hgS6UG0L9w.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-handler-proxy-4.1.52.Final-6NB4Y860vUgG4lWh1TLIZQ8POy1NAGrW0hgS6UG0L9w.jar' is inaccessible.
Sep 29, 2021 3:42:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-29T15:42:47.525Z: Staged package netty-resolver-4.1.52.Final-butyojOXnLRbC01tRsWkEzLFfAz4qAOx78YNu-y6CaY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-resolver-4.1.52.Final-butyojOXnLRbC01tRsWkEzLFfAz4qAOx78YNu-y6CaY.jar' is inaccessible.
Sep 29, 2021 3:42:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-29T15:42:47.584Z: Staged package netty-transport-4.1.52.Final-mj5vjA5V3jY-seoQ_ngXl-yjlOYhht8q4LTrK84LRUE.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-transport-4.1.52.Final-mj5vjA5V3jY-seoQ_ngXl-yjlOYhht8q4LTrK84LRUE.jar' is inaccessible.
Sep 29, 2021 3:42:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-29T15:42:47.650Z: Staged package opencensus-api-0.28.0-DBcj8_bTBhMjhFzouIs1_dpQCBLgp1uOtfzErYyHGpU.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/opencensus-api-0.28.0-DBcj8_bTBhMjhFzouIs1_dpQCBLgp1uOtfzErYyHGpU.jar' is inaccessible.
Sep 29, 2021 3:42:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-29T15:42:47.751Z: Staged package paranamer-2.8-aIyxGKYCHYGROOhVIIyVYDFoi-S0eiS7YVvsxjrO3wc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/paranamer-2.8-aIyxGKYCHYGROOhVIIyVYDFoi-S0eiS7YVvsxjrO3wc.jar' is inaccessible.
Sep 29, 2021 3:42:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-29T15:42:47.805Z: Staged package perfmark-api-0.23.0-xwW1wQwY_zAyuegXQrwvaw5WB_am38DIrQz_ddSRMEI.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/perfmark-api-0.23.0-xwW1wQwY_zAyuegXQrwvaw5WB_am38DIrQz_ddSRMEI.jar' is inaccessible.
Sep 29, 2021 3:42:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-29T15:42:48.188Z: Staged package snappy-java-1.1.8.4-JMTR_B6J4HgzGrj0AamcrWhZm95KLkUWBCy1SMUbHD4.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/snappy-java-1.1.8.4-JMTR_B6J4HgzGrj0AamcrWhZm95KLkUWBCy1SMUbHD4.jar' is inaccessible.
Sep 29, 2021 3:42:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-09-29T15:42:48.344Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Sep 29, 2021 3:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-09-29T15:45:47.713Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Sep 29, 2021 3:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-29T15:48:44.075Z: Staged package auto-value-annotations-1.8.1-N-wJtH1-01qZ0Tkn21yG_JBx9iD5Q-rV11cURpgxCFI.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/auto-value-annotations-1.8.1-N-wJtH1-01qZ0Tkn21yG_JBx9iD5Q-rV11cURpgxCFI.jar' is inaccessible.
Sep 29, 2021 3:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-29T15:48:44.967Z: Staged package commons-compress-1.21-auz9VFlyillWAc-gcljRMZcv_Dm0kutIvdWWV3ovJEo.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/commons-compress-1.21-auz9VFlyillWAc-gcljRMZcv_Dm0kutIvdWWV3ovJEo.jar' is inaccessible.
Sep 29, 2021 3:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-29T15:48:45.127Z: Staged package conscrypt-openjdk-uber-2.5.1-AfnHQstZKhUeLmK9U5eomAYoqWcAH82s1KpHRGeGhfM.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/conscrypt-openjdk-uber-2.5.1-AfnHQstZKhUeLmK9U5eomAYoqWcAH82s1KpHRGeGhfM.jar' is inaccessible.
Sep 29, 2021 3:48:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-29T15:48:47.199Z: Staged package netty-buffer-4.1.52.Final-QcoQNa7m4PgXdZfQp2EKWqLG5vrHRKyB_eTl5Y9jlnU.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-buffer-4.1.52.Final-QcoQNa7m4PgXdZfQp2EKWqLG5vrHRKyB_eTl5Y9jlnU.jar' is inaccessible.
Sep 29, 2021 3:48:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-29T15:48:47.253Z: Staged package netty-codec-4.1.52.Final-PH543M5-NT2GrH26LCUatc3dTrhuqoic_kIpeCuCjlI.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-codec-4.1.52.Final-PH543M5-NT2GrH26LCUatc3dTrhuqoic_kIpeCuCjlI.jar' is inaccessible.
Sep 29, 2021 3:48:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-29T15:48:47.297Z: Staged package netty-codec-http-4.1.52.Final-tdoo0OqHiwfTERrPXk-vQSiO8jArJkFBkP3Rp9lzyAw.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-codec-http-4.1.52.Final-tdoo0OqHiwfTERrPXk-vQSiO8jArJkFBkP3Rp9lzyAw.jar' is inaccessible.
Sep 29, 2021 3:48:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-29T15:48:47.346Z: Staged package netty-codec-http2-4.1.52.Final-61J6QtIonRLe5tR1PKJxWXije7tBxxkVzk6VOcSfk6E.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-codec-http2-4.1.52.Final-61J6QtIonRLe5tR1PKJxWXije7tBxxkVzk6VOcSfk6E.jar' is inaccessible.
Sep 29, 2021 3:48:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-29T15:48:47.392Z: Staged package netty-codec-socks-4.1.52.Final-_sxXfgu3ch1naM_BhG7hcpEdFquA372ZldllelUdTqc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-codec-socks-4.1.52.Final-_sxXfgu3ch1naM_BhG7hcpEdFquA372ZldllelUdTqc.jar' is inaccessible.
Sep 29, 2021 3:48:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-29T15:48:47.427Z: Staged package netty-common-4.1.52.Final-DR8QHk231TDtBNfKvVfZXAdQ8Fm7az-t_tDIDUc0xxc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-common-4.1.52.Final-DR8QHk231TDtBNfKvVfZXAdQ8Fm7az-t_tDIDUc0xxc.jar' is inaccessible.
Sep 29, 2021 3:48:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-29T15:48:47.463Z: Staged package netty-handler-4.1.52.Final-LNwh-_rgSUDIWSkDl3nJP1H2GPXvshYHkxnc_PMqVcE.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-handler-4.1.52.Final-LNwh-_rgSUDIWSkDl3nJP1H2GPXvshYHkxnc_PMqVcE.jar' is inaccessible.
Sep 29, 2021 3:48:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-29T15:48:47.496Z: Staged package netty-handler-proxy-4.1.52.Final-6NB4Y860vUgG4lWh1TLIZQ8POy1NAGrW0hgS6UG0L9w.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-handler-proxy-4.1.52.Final-6NB4Y860vUgG4lWh1TLIZQ8POy1NAGrW0hgS6UG0L9w.jar' is inaccessible.
Sep 29, 2021 3:48:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-29T15:48:47.538Z: Staged package netty-resolver-4.1.52.Final-butyojOXnLRbC01tRsWkEzLFfAz4qAOx78YNu-y6CaY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-resolver-4.1.52.Final-butyojOXnLRbC01tRsWkEzLFfAz4qAOx78YNu-y6CaY.jar' is inaccessible.
Sep 29, 2021 3:48:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-29T15:48:47.596Z: Staged package netty-transport-4.1.52.Final-mj5vjA5V3jY-seoQ_ngXl-yjlOYhht8q4LTrK84LRUE.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-transport-4.1.52.Final-mj5vjA5V3jY-seoQ_ngXl-yjlOYhht8q4LTrK84LRUE.jar' is inaccessible.
Sep 29, 2021 3:48:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-29T15:48:47.636Z: Staged package opencensus-api-0.28.0-DBcj8_bTBhMjhFzouIs1_dpQCBLgp1uOtfzErYyHGpU.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/opencensus-api-0.28.0-DBcj8_bTBhMjhFzouIs1_dpQCBLgp1uOtfzErYyHGpU.jar' is inaccessible.
Sep 29, 2021 3:48:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-29T15:48:47.732Z: Staged package paranamer-2.8-aIyxGKYCHYGROOhVIIyVYDFoi-S0eiS7YVvsxjrO3wc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/paranamer-2.8-aIyxGKYCHYGROOhVIIyVYDFoi-S0eiS7YVvsxjrO3wc.jar' is inaccessible.
Sep 29, 2021 3:48:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-29T15:48:47.772Z: Staged package perfmark-api-0.23.0-xwW1wQwY_zAyuegXQrwvaw5WB_am38DIrQz_ddSRMEI.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/perfmark-api-0.23.0-xwW1wQwY_zAyuegXQrwvaw5WB_am38DIrQz_ddSRMEI.jar' is inaccessible.
Sep 29, 2021 3:48:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-29T15:48:48.168Z: Staged package snappy-java-1.1.8.4-JMTR_B6J4HgzGrj0AamcrWhZm95KLkUWBCy1SMUbHD4.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/snappy-java-1.1.8.4-JMTR_B6J4HgzGrj0AamcrWhZm95KLkUWBCy1SMUbHD4.jar' is inaccessible.
Sep 29, 2021 3:48:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-09-29T15:48:48.312Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Sep 29, 2021 3:51:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-09-29T15:51:47.957Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Sep 29, 2021 3:54:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-29T15:54:44.047Z: Staged package auto-value-annotations-1.8.1-N-wJtH1-01qZ0Tkn21yG_JBx9iD5Q-rV11cURpgxCFI.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/auto-value-annotations-1.8.1-N-wJtH1-01qZ0Tkn21yG_JBx9iD5Q-rV11cURpgxCFI.jar' is inaccessible.
Sep 29, 2021 3:54:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-29T15:54:44.907Z: Staged package commons-compress-1.21-auz9VFlyillWAc-gcljRMZcv_Dm0kutIvdWWV3ovJEo.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/commons-compress-1.21-auz9VFlyillWAc-gcljRMZcv_Dm0kutIvdWWV3ovJEo.jar' is inaccessible.
Sep 29, 2021 3:54:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-29T15:54:45.051Z: Staged package conscrypt-openjdk-uber-2.5.1-AfnHQstZKhUeLmK9U5eomAYoqWcAH82s1KpHRGeGhfM.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/conscrypt-openjdk-uber-2.5.1-AfnHQstZKhUeLmK9U5eomAYoqWcAH82s1KpHRGeGhfM.jar' is inaccessible.
Sep 29, 2021 3:54:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-29T15:54:46.867Z: Staged package netty-buffer-4.1.52.Final-QcoQNa7m4PgXdZfQp2EKWqLG5vrHRKyB_eTl5Y9jlnU.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-buffer-4.1.52.Final-QcoQNa7m4PgXdZfQp2EKWqLG5vrHRKyB_eTl5Y9jlnU.jar' is inaccessible.
Sep 29, 2021 3:54:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-29T15:54:46.901Z: Staged package netty-codec-4.1.52.Final-PH543M5-NT2GrH26LCUatc3dTrhuqoic_kIpeCuCjlI.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-codec-4.1.52.Final-PH543M5-NT2GrH26LCUatc3dTrhuqoic_kIpeCuCjlI.jar' is inaccessible.
Sep 29, 2021 3:54:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-29T15:54:46.936Z: Staged package netty-codec-http-4.1.52.Final-tdoo0OqHiwfTERrPXk-vQSiO8jArJkFBkP3Rp9lzyAw.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-codec-http-4.1.52.Final-tdoo0OqHiwfTERrPXk-vQSiO8jArJkFBkP3Rp9lzyAw.jar' is inaccessible.
Sep 29, 2021 3:54:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-29T15:54:46.979Z: Staged package netty-codec-http2-4.1.52.Final-61J6QtIonRLe5tR1PKJxWXije7tBxxkVzk6VOcSfk6E.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-codec-http2-4.1.52.Final-61J6QtIonRLe5tR1PKJxWXije7tBxxkVzk6VOcSfk6E.jar' is inaccessible.
Sep 29, 2021 3:54:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-29T15:54:47.035Z: Staged package netty-codec-socks-4.1.52.Final-_sxXfgu3ch1naM_BhG7hcpEdFquA372ZldllelUdTqc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-codec-socks-4.1.52.Final-_sxXfgu3ch1naM_BhG7hcpEdFquA372ZldllelUdTqc.jar' is inaccessible.
Sep 29, 2021 3:54:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-29T15:54:47.071Z: Staged package netty-common-4.1.52.Final-DR8QHk231TDtBNfKvVfZXAdQ8Fm7az-t_tDIDUc0xxc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-common-4.1.52.Final-DR8QHk231TDtBNfKvVfZXAdQ8Fm7az-t_tDIDUc0xxc.jar' is inaccessible.
Sep 29, 2021 3:54:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-29T15:54:47.112Z: Staged package netty-handler-4.1.52.Final-LNwh-_rgSUDIWSkDl3nJP1H2GPXvshYHkxnc_PMqVcE.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-handler-4.1.52.Final-LNwh-_rgSUDIWSkDl3nJP1H2GPXvshYHkxnc_PMqVcE.jar' is inaccessible.
Sep 29, 2021 3:54:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-29T15:54:47.151Z: Staged package netty-handler-proxy-4.1.52.Final-6NB4Y860vUgG4lWh1TLIZQ8POy1NAGrW0hgS6UG0L9w.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-handler-proxy-4.1.52.Final-6NB4Y860vUgG4lWh1TLIZQ8POy1NAGrW0hgS6UG0L9w.jar' is inaccessible.
Sep 29, 2021 3:54:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-29T15:54:47.194Z: Staged package netty-resolver-4.1.52.Final-butyojOXnLRbC01tRsWkEzLFfAz4qAOx78YNu-y6CaY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-resolver-4.1.52.Final-butyojOXnLRbC01tRsWkEzLFfAz4qAOx78YNu-y6CaY.jar' is inaccessible.
Sep 29, 2021 3:54:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-29T15:54:47.254Z: Staged package netty-transport-4.1.52.Final-mj5vjA5V3jY-seoQ_ngXl-yjlOYhht8q4LTrK84LRUE.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-transport-4.1.52.Final-mj5vjA5V3jY-seoQ_ngXl-yjlOYhht8q4LTrK84LRUE.jar' is inaccessible.
Sep 29, 2021 3:54:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-29T15:54:47.293Z: Staged package opencensus-api-0.28.0-DBcj8_bTBhMjhFzouIs1_dpQCBLgp1uOtfzErYyHGpU.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/opencensus-api-0.28.0-DBcj8_bTBhMjhFzouIs1_dpQCBLgp1uOtfzErYyHGpU.jar' is inaccessible.
Sep 29, 2021 3:54:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-29T15:54:47.371Z: Staged package paranamer-2.8-aIyxGKYCHYGROOhVIIyVYDFoi-S0eiS7YVvsxjrO3wc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/paranamer-2.8-aIyxGKYCHYGROOhVIIyVYDFoi-S0eiS7YVvsxjrO3wc.jar' is inaccessible.
Sep 29, 2021 3:54:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-29T15:54:47.411Z: Staged package perfmark-api-0.23.0-xwW1wQwY_zAyuegXQrwvaw5WB_am38DIrQz_ddSRMEI.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/perfmark-api-0.23.0-xwW1wQwY_zAyuegXQrwvaw5WB_am38DIrQz_ddSRMEI.jar' is inaccessible.
Sep 29, 2021 3:54:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-29T15:54:47.835Z: Staged package snappy-java-1.1.8.4-JMTR_B6J4HgzGrj0AamcrWhZm95KLkUWBCy1SMUbHD4.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/snappy-java-1.1.8.4-JMTR_B6J4HgzGrj0AamcrWhZm95KLkUWBCy1SMUbHD4.jar' is inaccessible.
Sep 29, 2021 3:54:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-09-29T15:54:47.973Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Sep 29, 2021 3:57:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-09-29T15:57:47.424Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Sep 29, 2021 4:00:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-29T16:00:26.162Z: Cancel request is committed for workflow job: 2021-09-29_05_45_29-3890809183779990448.
Sep 29, 2021 4:00:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-29T16:00:26.197Z: Cleaning up.
Sep 29, 2021 4:00:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-29T16:00:26.282Z: Stopping **** pool...
Sep 29, 2021 4:00:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-29T16:00:26.325Z: Stopping **** pool...
Sep 29, 2021 4:00:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-29T16:00:34.655Z: generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:1089
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:1089
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:1089
Sep 29, 2021 4:02:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-29T16:02:47.909Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Sep 29, 2021 4:02:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-29T16:02:47.972Z: Worker pool stopped.
Sep 29, 2021 4:03:03 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-09-29_05_45_29-3890809183779990448 finished with status CANCELLED.
Load test results for test (ID): 685e7028-0ab6-48c7-bc07-c3efc17bedcc and timestamp: 2021-09-29T12:45:24.386000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11548.812
dataflow_v2_java11_total_bytes_count             2.64142272E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210929124332
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:d1425d4294767e687ebbd35f632b2c2c6b5e12a0fbd0da744d90ca18892ae676
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210929124332]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:d1425d4294767e687ebbd35f632b2c2c6b5e12a0fbd0da744d90ca18892ae676]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210929124332] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:d1425d4294767e687ebbd35f632b2c2c6b5e12a0fbd0da744d90ca18892ae676])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:d1425d4294767e687ebbd35f632b2c2c6b5e12a0fbd0da744d90ca18892ae676
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:d1425d4294767e687ebbd35f632b2c2c6b5e12a0fbd0da744d90ca18892ae676
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:d1425d4294767e687ebbd35f632b2c2c6b5e12a0fbd0da744d90ca18892ae676].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 50s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/57qasuxi7v6q6

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure


---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #103

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/103/display/redirect?page=changes>

Changes:

[Andrew Pilloud] [BEAM-12691] FieldAccessDescriptor for BeamCalcRel

[aydar.zaynutdinov] Init basic go backend structure

[noreply] Update container tags used by unreleased SDKs.

[aydar.zaynutdinov] Add .gitkeep file as an exclusion

[yileiyang] Remove encoding= from the json.loads call.

[noreply] [BEAM-12798] Add configurable combiner packing limit (#15391)

[noreply] [BEAM-12926] Translates Reshuffle with Samza native repartition operator

[noreply] [BEAM-11007] Allow nbconvert 6.x (#15595)

[noreply] [BEAM-12945] Fix crashes when importing the DataFrame API with pandas

[noreply] [BEAM-12611] Capture a greater proporition of logs associated to an


------------------------------------------
[...truncated 48.98 KB...]
102bc031837a: Preparing
fcd0be6cd8e4: Preparing
a98a451ba0e5: Preparing
dbc817088159: Preparing
8330a8eb6279: Preparing
6e85f2bf577d: Preparing
874ad65f91ea: Preparing
0fc2498b65e5: Preparing
d08e6b97bf21: Preparing
3054497613e6: Preparing
d35dc7f4c79e: Preparing
dabfe5b2ea81: Preparing
5e6a409f30b6: Preparing
102bc031837a: Waiting
0fc2498b65e5: Waiting
fcd0be6cd8e4: Waiting
d08e6b97bf21: Waiting
d829bd2ded64: Waiting
3054497613e6: Waiting
a98a451ba0e5: Waiting
d35dc7f4c79e: Waiting
dabfe5b2ea81: Waiting
874ad65f91ea: Waiting
5e6a409f30b6: Waiting
dbc817088159: Waiting
8330a8eb6279: Waiting
6e85f2bf577d: Waiting
0ad119aed9fc: Pushed
7ff1eed6bbc9: Pushed
dcfa9e2c6b21: Pushed
501342684d0d: Pushed
d829bd2ded64: Pushed
ae46df76eb5b: Pushed
fcd0be6cd8e4: Pushed
a98a451ba0e5: Pushed
874ad65f91ea: Layer already exists
0fc2498b65e5: Layer already exists
d08e6b97bf21: Layer already exists
3054497613e6: Layer already exists
8330a8eb6279: Pushed
d35dc7f4c79e: Layer already exists
102bc031837a: Pushed
dabfe5b2ea81: Layer already exists
6e85f2bf577d: Pushed
5e6a409f30b6: Layer already exists
dbc817088159: Pushed
20210928124333: digest: sha256:26e03b4ba2a39ade88921f2105ec0c0daa1bd2c2564f51ca086590a919cf1576 size: 4311

> Task :sdks:java:testing:load-tests:run
Sep 28, 2021 12:45:48 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Sep 28, 2021 12:45:48 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 195 files. Enable logging at DEBUG level to see which files will be staged.
Sep 28, 2021 12:45:49 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Sep 28, 2021 12:45:49 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Sep 28, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 195 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Sep 28, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 195 files cached, 0 files newly uploaded in 0 seconds
Sep 28, 2021 12:45:51 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Sep 28, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <106569 bytes, hash d93f0953b14850cca27eac110558c862f5d119db6d9c263859446a8949dce14a> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-2T8JU7FIUMyifqwRBVjIYvXRGdttnCY4WURqiUnc4Uo.pb
Sep 28, 2021 12:45:53 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Sep 28, 2021 12:45:53 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2a869a16, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@ae202c6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@46aa712c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6ada9c0c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7412ed6b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6e7c351d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b4a0aef, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@45cec376, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26d41711, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@298f0a0b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2b960a7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@31dfc6f5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@37b52340, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@663bb8ef, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2f4e40d7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@60e9c3a5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7e5843db, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@459f703f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@188ac8a3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3650d4fc]
Sep 28, 2021 12:45:53 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Sep 28, 2021 12:45:53 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Sep 28, 2021 12:45:53 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Sep 28, 2021 12:45:53 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Sep 28, 2021 12:45:53 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@441b8382, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1df1ced0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5349b246, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@32b0876c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2aaf152b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@39bbd9e0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@27fe9713, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@11c3ff67, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4397a639, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@24841372, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77114efe, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@79a7bfbc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77f68df, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3e4e4c1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7e7f3cfd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ae126d1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@46a488c2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6242ae3b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@65ddee5a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@56399b9e]
Sep 28, 2021 12:45:53 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Sep 28, 2021 12:45:53 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Sep 28, 2021 12:45:53 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Sep 28, 2021 12:45:53 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Sep 28, 2021 12:45:53 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Sep 28, 2021 12:45:53 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Sep 28, 2021 12:45:53 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Sep 28, 2021 12:45:53 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Sep 28, 2021 12:45:53 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Sep 28, 2021 12:45:53 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Sep 28, 2021 12:45:53 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Sep 28, 2021 12:45:53 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.34.0-SNAPSHOT
Sep 28, 2021 12:45:54 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-09-28_05_45_53-13740441012094939605?project=apache-beam-testing
Sep 28, 2021 12:45:54 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-09-28_05_45_53-13740441012094939605
Sep 28, 2021 12:45:54 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-09-28_05_45_53-13740441012094939605
Sep 28, 2021 12:46:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-09-28T12:46:02.361Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-09-lnn6. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Sep 28, 2021 12:46:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-28T12:46:05.348Z: Worker configuration: e2-standard-2 in us-central1-a.
Sep 28, 2021 12:46:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-28T12:46:06.194Z: Expanding SplittableParDo operations into optimizable parts.
Sep 28, 2021 12:46:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-28T12:46:06.229Z: Expanding CollectionToSingleton operations into optimizable parts.
Sep 28, 2021 12:46:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-28T12:46:06.289Z: Expanding CoGroupByKey operations into optimizable parts.
Sep 28, 2021 12:46:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-28T12:46:06.358Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Sep 28, 2021 12:46:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-28T12:46:06.385Z: Expanding GroupByKey operations into streaming Read/Write steps
Sep 28, 2021 12:46:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-28T12:46:06.452Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Sep 28, 2021 12:46:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-28T12:46:06.547Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Sep 28, 2021 12:46:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-28T12:46:06.592Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Sep 28, 2021 12:46:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-28T12:46:06.627Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Sep 28, 2021 12:46:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-28T12:46:06.660Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Sep 28, 2021 12:46:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-28T12:46:06.696Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Sep 28, 2021 12:46:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-28T12:46:06.727Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Sep 28, 2021 12:46:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-28T12:46:06.759Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Sep 28, 2021 12:46:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-28T12:46:06.789Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Sep 28, 2021 12:46:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-28T12:46:06.822Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Sep 28, 2021 12:46:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-28T12:46:06.853Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Sep 28, 2021 12:46:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-28T12:46:06.879Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Sep 28, 2021 12:46:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-28T12:46:06.903Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Sep 28, 2021 12:46:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-28T12:46:06.937Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Sep 28, 2021 12:46:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-28T12:46:06.968Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Sep 28, 2021 12:46:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-28T12:46:07.005Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Sep 28, 2021 12:46:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-28T12:46:07.037Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Sep 28, 2021 12:46:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-28T12:46:07.084Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Sep 28, 2021 12:46:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-28T12:46:07.109Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Sep 28, 2021 12:46:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-28T12:46:07.137Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Sep 28, 2021 12:46:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-28T12:46:07.174Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Sep 28, 2021 12:46:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-28T12:46:07.200Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Sep 28, 2021 12:46:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-28T12:46:07.230Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Sep 28, 2021 12:46:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-28T12:46:07.263Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Sep 28, 2021 12:46:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-28T12:46:07.641Z: Starting 5 ****s in us-central1-a...
Sep 28, 2021 12:46:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-28T12:46:13.916Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Sep 28, 2021 12:46:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-28T12:46:49.160Z: Autoscaling: Raised the number of ****s to 2 so that the pipeline can catch up with its backlog and keep up with its input rate.
Sep 28, 2021 12:46:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-28T12:46:49.188Z: Resized **** pool to 2, though goal was 5.  This could be a quota issue.
Sep 28, 2021 12:47:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-28T12:46:59.508Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Sep 28, 2021 12:47:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-28T12:47:52.450Z: Workers have started successfully.
Sep 28, 2021 12:47:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-28T12:47:52.486Z: Workers have started successfully.
Sep 28, 2021 3:46:28 PM org.apache.beam.sdk.metrics.MetricsEnvironment getCurrentContainer
WARNING: Reporting metrics are not supported in the current execution environment.
Sep 28, 2021 4:00:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-28T16:00:32.968Z: Cancel request is committed for workflow job: 2021-09-28_05_45_53-13740441012094939605.
Sep 28, 2021 4:00:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-28T16:00:33.043Z: Cleaning up.
Sep 28, 2021 4:00:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-28T16:00:33.104Z: Stopping **** pool...
Sep 28, 2021 4:00:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-28T16:00:33.184Z: Stopping **** pool...
Sep 28, 2021 4:02:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-28T16:02:51.473Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Sep 28, 2021 4:02:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-28T16:02:51.515Z: Worker pool stopped.
Sep 28, 2021 4:02:58 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-09-28_05_45_53-13740441012094939605 finished with status CANCELLED.
Load test results for test (ID): 8bc866d9-07ee-444d-9b12-1c9236191504 and timestamp: 2021-09-28T12:45:48.907000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11523.763
dataflow_v2_java11_total_bytes_count             2.80484304E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210928124333
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:26e03b4ba2a39ade88921f2105ec0c0daa1bd2c2564f51ca086590a919cf1576
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210928124333]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:26e03b4ba2a39ade88921f2105ec0c0daa1bd2c2564f51ca086590a919cf1576]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210928124333] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:26e03b4ba2a39ade88921f2105ec0c0daa1bd2c2564f51ca086590a919cf1576])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:26e03b4ba2a39ade88921f2105ec0c0daa1bd2c2564f51ca086590a919cf1576
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:26e03b4ba2a39ade88921f2105ec0c0daa1bd2c2564f51ca086590a919cf1576
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:26e03b4ba2a39ade88921f2105ec0c0daa1bd2c2564f51ca086590a919cf1576].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 43s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/n3zkw263wznzy

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #102

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/102/display/redirect>

Changes:


------------------------------------------
[...truncated 49.04 KB...]
The push refers to repository [us.gcr.io/apache-beam-testing/java-postcommit-it/java]
e1d8cf3fc310: Preparing
517d57875d1a: Preparing
e6b445379188: Preparing
b5dc947b0725: Preparing
d5e33691d531: Preparing
fa3ed5c81f79: Preparing
5bca6def87fa: Preparing
6d480301d481: Preparing
b4830f2cf7da: Preparing
659fa2f7ffc5: Preparing
2577fbe182b2: Preparing
8db2d63d863b: Preparing
3891808a925b: Preparing
d402f4f1b906: Preparing
00ef5416d927: Preparing
8555e663f65b: Preparing
d00da3cd7763: Preparing
4e61e63529c2: Preparing
799760671c38: Preparing
fa3ed5c81f79: Waiting
d00da3cd7763: Waiting
d402f4f1b906: Waiting
2577fbe182b2: Waiting
4e61e63529c2: Waiting
5bca6def87fa: Waiting
799760671c38: Waiting
8555e663f65b: Waiting
8db2d63d863b: Waiting
00ef5416d927: Waiting
6d480301d481: Waiting
3891808a925b: Waiting
659fa2f7ffc5: Waiting
d5e33691d531: Pushed
517d57875d1a: Pushed
e1d8cf3fc310: Pushed
fa3ed5c81f79: Pushed
b5dc947b0725: Pushed
b4830f2cf7da: Pushed
5bca6def87fa: Pushed
6d480301d481: Pushed
3891808a925b: Layer already exists
d402f4f1b906: Layer already exists
00ef5416d927: Layer already exists
8555e663f65b: Layer already exists
d00da3cd7763: Layer already exists
4e61e63529c2: Layer already exists
799760671c38: Layer already exists
2577fbe182b2: Pushed
8db2d63d863b: Pushed
659fa2f7ffc5: Pushed
e6b445379188: Pushed
20210927124331: digest: sha256:1caf78f9c5e9c3bd6a8ec4ad0e57addca65abd38d011c2ac3391d07786749ef8 size: 4311

> Task :sdks:java:testing:load-tests:run
Sep 27, 2021 12:45:46 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Sep 27, 2021 12:45:46 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 195 files. Enable logging at DEBUG level to see which files will be staged.
Sep 27, 2021 12:45:47 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Sep 27, 2021 12:45:47 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Sep 27, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 195 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Sep 27, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 195 files cached, 0 files newly uploaded in 1 seconds
Sep 27, 2021 12:45:51 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Sep 27, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <106569 bytes, hash 73a482e146db5acdd4cc2ed93cdf9fe2b86e4a8e50954daf7940ca7c3a48befb> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-c6SC4UbbWs3UzC7ZPN-f4rhuSo5QlU2veUDKfDpIvvs.pb
Sep 27, 2021 12:45:53 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Sep 27, 2021 12:45:53 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2a869a16, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@ae202c6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@46aa712c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6ada9c0c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7412ed6b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6e7c351d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b4a0aef, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@45cec376, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26d41711, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@298f0a0b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2b960a7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@31dfc6f5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@37b52340, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@663bb8ef, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2f4e40d7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@60e9c3a5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7e5843db, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@459f703f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@188ac8a3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3650d4fc]
Sep 27, 2021 12:45:53 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Sep 27, 2021 12:45:53 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Sep 27, 2021 12:45:53 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Sep 27, 2021 12:45:53 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Sep 27, 2021 12:45:53 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@441b8382, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1df1ced0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5349b246, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@32b0876c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2aaf152b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@39bbd9e0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@27fe9713, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@11c3ff67, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4397a639, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@24841372, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77114efe, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@79a7bfbc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77f68df, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3e4e4c1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7e7f3cfd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ae126d1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@46a488c2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6242ae3b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@65ddee5a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@56399b9e]
Sep 27, 2021 12:45:53 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Sep 27, 2021 12:45:53 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Sep 27, 2021 12:45:53 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Sep 27, 2021 12:45:53 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Sep 27, 2021 12:45:53 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Sep 27, 2021 12:45:53 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Sep 27, 2021 12:45:53 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Sep 27, 2021 12:45:53 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Sep 27, 2021 12:45:53 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Sep 27, 2021 12:45:53 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Sep 27, 2021 12:45:53 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Sep 27, 2021 12:45:53 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.34.0-SNAPSHOT
Sep 27, 2021 12:45:54 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-09-27_05_45_53-13018494378937361099?project=apache-beam-testing
Sep 27, 2021 12:45:54 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-09-27_05_45_53-13018494378937361099
Sep 27, 2021 12:45:54 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-09-27_05_45_53-13018494378937361099
Sep 27, 2021 12:46:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-09-27T12:46:01.285Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-09-mjpp. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Sep 27, 2021 12:46:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-27T12:46:05.040Z: Worker configuration: e2-standard-2 in us-central1-a.
Sep 27, 2021 12:46:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-27T12:46:05.742Z: Expanding SplittableParDo operations into optimizable parts.
Sep 27, 2021 12:46:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-27T12:46:05.794Z: Expanding CollectionToSingleton operations into optimizable parts.
Sep 27, 2021 12:46:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-27T12:46:05.851Z: Expanding CoGroupByKey operations into optimizable parts.
Sep 27, 2021 12:46:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-27T12:46:05.925Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Sep 27, 2021 12:46:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-27T12:46:05.963Z: Expanding GroupByKey operations into streaming Read/Write steps
Sep 27, 2021 12:46:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-27T12:46:06.035Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Sep 27, 2021 12:46:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-27T12:46:06.152Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Sep 27, 2021 12:46:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-27T12:46:06.202Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Sep 27, 2021 12:46:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-27T12:46:06.240Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Sep 27, 2021 12:46:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-27T12:46:06.269Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Sep 27, 2021 12:46:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-27T12:46:06.303Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Sep 27, 2021 12:46:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-27T12:46:06.339Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Sep 27, 2021 12:46:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-27T12:46:06.372Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Sep 27, 2021 12:46:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-27T12:46:06.410Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Sep 27, 2021 12:46:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-27T12:46:06.439Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Sep 27, 2021 12:46:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-27T12:46:06.487Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Sep 27, 2021 12:46:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-27T12:46:06.519Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Sep 27, 2021 12:46:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-27T12:46:06.548Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Sep 27, 2021 12:46:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-27T12:46:06.580Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Sep 27, 2021 12:46:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-27T12:46:06.605Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Sep 27, 2021 12:46:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-27T12:46:06.640Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Sep 27, 2021 12:46:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-27T12:46:06.673Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Sep 27, 2021 12:46:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-27T12:46:06.716Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Sep 27, 2021 12:46:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-27T12:46:06.746Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Sep 27, 2021 12:46:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-27T12:46:06.804Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Sep 27, 2021 12:46:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-27T12:46:06.835Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Sep 27, 2021 12:46:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-27T12:46:06.866Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Sep 27, 2021 12:46:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-27T12:46:06.902Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Sep 27, 2021 12:46:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-27T12:46:06.935Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Sep 27, 2021 12:46:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-27T12:46:07.340Z: Starting 5 ****s in us-central1-a...
Sep 27, 2021 12:46:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-27T12:46:37.690Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Sep 27, 2021 12:46:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-27T12:46:47.652Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Sep 27, 2021 12:47:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-27T12:47:44.864Z: Workers have started successfully.
Sep 27, 2021 12:47:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-27T12:47:44.882Z: Workers have started successfully.
Sep 27, 2021 4:00:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-27T16:00:27.941Z: Cancel request is committed for workflow job: 2021-09-27_05_45_53-13018494378937361099.
Sep 27, 2021 4:00:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-27T16:00:28.895Z: Cleaning up.
Sep 27, 2021 4:00:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-27T16:00:28.971Z: Stopping **** pool...
Sep 27, 2021 4:00:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-27T16:00:29.048Z: Stopping **** pool...
Sep 27, 2021 4:02:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-27T16:02:51.112Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Sep 27, 2021 4:02:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-27T16:02:51.156Z: Worker pool stopped.
Sep 27, 2021 4:03:00 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-09-27_05_45_53-13018494378937361099 finished with status CANCELLED.
Load test results for test (ID): 24917cd4-4ae7-4644-ba38-ccd18a03021b and timestamp: 2021-09-27T12:45:46.980000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11533.374
dataflow_v2_java11_total_bytes_count             2.72591798E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210927124331
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:1caf78f9c5e9c3bd6a8ec4ad0e57addca65abd38d011c2ac3391d07786749ef8
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210927124331]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:1caf78f9c5e9c3bd6a8ec4ad0e57addca65abd38d011c2ac3391d07786749ef8]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210927124331] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:1caf78f9c5e9c3bd6a8ec4ad0e57addca65abd38d011c2ac3391d07786749ef8])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:1caf78f9c5e9c3bd6a8ec4ad0e57addca65abd38d011c2ac3391d07786749ef8
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:1caf78f9c5e9c3bd6a8ec4ad0e57addca65abd38d011c2ac3391d07786749ef8
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:1caf78f9c5e9c3bd6a8ec4ad0e57addca65abd38d011c2ac3391d07786749ef8].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 45s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/ciitnhbt3vi44

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #101

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/101/display/redirect>

Changes:


------------------------------------------
[...truncated 49.91 KB...]
63f8e7a34014: Pushed
7949805e1c86: Pushed
3a963485e68e: Pushed
a131c77e6716: Pushed
6222ed7176f7: Pushed
3891808a925b: Layer already exists
d402f4f1b906: Layer already exists
00ef5416d927: Layer already exists
8555e663f65b: Layer already exists
d00da3cd7763: Layer already exists
631bf01a1485: Pushed
d776603a71a0: Pushed
4e61e63529c2: Layer already exists
799760671c38: Layer already exists
9bbcdc967ab2: Pushed
0fda83d3a2e3: Pushed
20210926124332: digest: sha256:5678472199be1c1cf321f70f99b280886c12262960474a7c6e9194f86255e67b size: 4311

> Task :sdks:java:testing:load-tests:run
Sep 26, 2021 12:45:13 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Sep 26, 2021 12:45:14 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 195 files. Enable logging at DEBUG level to see which files will be staged.
Sep 26, 2021 12:45:14 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Sep 26, 2021 12:45:14 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Sep 26, 2021 12:45:16 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 195 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Sep 26, 2021 12:45:17 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 195 files cached, 0 files newly uploaded in 0 seconds
Sep 26, 2021 12:45:17 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Sep 26, 2021 12:45:17 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <106569 bytes, hash d19829175c9682d54e17e61b9d3ace3fb8fa54f1df73a6663c6068e53199d8f3> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-0ZgpF1yWgtVOF-YbnTrOP7j6VPHfc6ZmPGBo5TGZ2PM.pb
Sep 26, 2021 12:45:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Sep 26, 2021 12:45:19 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2a869a16, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@ae202c6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@46aa712c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6ada9c0c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7412ed6b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6e7c351d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b4a0aef, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@45cec376, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26d41711, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@298f0a0b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2b960a7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@31dfc6f5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@37b52340, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@663bb8ef, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2f4e40d7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@60e9c3a5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7e5843db, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@459f703f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@188ac8a3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3650d4fc]
Sep 26, 2021 12:45:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Sep 26, 2021 12:45:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Sep 26, 2021 12:45:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Sep 26, 2021 12:45:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Sep 26, 2021 12:45:19 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@441b8382, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1df1ced0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5349b246, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@32b0876c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2aaf152b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@39bbd9e0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@27fe9713, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@11c3ff67, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4397a639, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@24841372, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77114efe, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@79a7bfbc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77f68df, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3e4e4c1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7e7f3cfd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ae126d1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@46a488c2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6242ae3b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@65ddee5a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@56399b9e]
Sep 26, 2021 12:45:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Sep 26, 2021 12:45:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Sep 26, 2021 12:45:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Sep 26, 2021 12:45:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Sep 26, 2021 12:45:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Sep 26, 2021 12:45:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Sep 26, 2021 12:45:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Sep 26, 2021 12:45:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Sep 26, 2021 12:45:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Sep 26, 2021 12:45:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Sep 26, 2021 12:45:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Sep 26, 2021 12:45:19 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.34.0-SNAPSHOT
Sep 26, 2021 12:45:20 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-09-26_05_45_19-3650379884586848115?project=apache-beam-testing
Sep 26, 2021 12:45:20 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-09-26_05_45_19-3650379884586848115
Sep 26, 2021 12:45:20 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-09-26_05_45_19-3650379884586848115
Sep 26, 2021 12:45:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-09-26T12:45:26.493Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-09-l6tm. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Sep 26, 2021 12:45:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T12:45:30.570Z: Worker configuration: e2-standard-2 in us-central1-a.
Sep 26, 2021 12:45:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T12:45:31.366Z: Expanding SplittableParDo operations into optimizable parts.
Sep 26, 2021 12:45:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T12:45:31.395Z: Expanding CollectionToSingleton operations into optimizable parts.
Sep 26, 2021 12:45:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T12:45:31.452Z: Expanding CoGroupByKey operations into optimizable parts.
Sep 26, 2021 12:45:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T12:45:31.529Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Sep 26, 2021 12:45:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T12:45:31.558Z: Expanding GroupByKey operations into streaming Read/Write steps
Sep 26, 2021 12:45:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T12:45:31.613Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Sep 26, 2021 12:45:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T12:45:31.721Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Sep 26, 2021 12:45:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T12:45:31.753Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Sep 26, 2021 12:45:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T12:45:31.786Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Sep 26, 2021 12:45:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T12:45:31.812Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Sep 26, 2021 12:45:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T12:45:31.844Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Sep 26, 2021 12:45:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T12:45:31.867Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Sep 26, 2021 12:45:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T12:45:31.900Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Sep 26, 2021 12:45:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T12:45:31.933Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Sep 26, 2021 12:45:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T12:45:31.975Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Sep 26, 2021 12:45:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T12:45:31.999Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Sep 26, 2021 12:45:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T12:45:32.032Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Sep 26, 2021 12:45:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T12:45:32.064Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Sep 26, 2021 12:45:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T12:45:32.111Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Sep 26, 2021 12:45:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T12:45:32.145Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Sep 26, 2021 12:45:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T12:45:32.181Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Sep 26, 2021 12:45:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T12:45:32.211Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Sep 26, 2021 12:45:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T12:45:32.244Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Sep 26, 2021 12:45:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T12:45:32.267Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Sep 26, 2021 12:45:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T12:45:32.291Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Sep 26, 2021 12:45:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T12:45:32.323Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Sep 26, 2021 12:45:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T12:45:32.356Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Sep 26, 2021 12:45:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T12:45:32.377Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Sep 26, 2021 12:45:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T12:45:32.409Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Sep 26, 2021 12:45:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T12:45:32.680Z: Starting 5 ****s in us-central1-a...
Sep 26, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T12:45:55.156Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Sep 26, 2021 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T12:46:17.944Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Sep 26, 2021 12:47:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T12:47:13.687Z: Workers have started successfully.
Sep 26, 2021 12:47:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T12:47:13.717Z: Workers have started successfully.
Sep 26, 2021 3:48:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-26T15:48:33.609Z: Staged package classgraph-4.8.104-ePvfM6WdiJMBzk05ytbzmopew--T1rqGqw_PYcvi6h0.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/classgraph-4.8.104-ePvfM6WdiJMBzk05ytbzmopew--T1rqGqw_PYcvi6h0.jar' is inaccessible.
Sep 26, 2021 3:48:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-26T15:48:33.974Z: Staged package flogger-0.6-MUVmnbYMex1jtOH3E36hpd5uqRxRamjoz90leU2Bt8Q.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/flogger-0.6-MUVmnbYMex1jtOH3E36hpd5uqRxRamjoz90leU2Bt8Q.jar' is inaccessible.
Sep 26, 2021 3:48:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-26T15:48:34.013Z: Staged package flogger-system-backend-0.6-r3-fBJx59nlB4eR2N7IGNFlHiDmKoKq83bx68K899bE.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/flogger-system-backend-0.6-r3-fBJx59nlB4eR2N7IGNFlHiDmKoKq83bx68K899bE.jar' is inaccessible.
Sep 26, 2021 3:48:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-26T15:48:34.497Z: Staged package google-extensions-0.6-snj9TtDjAZ6b9UseOnnNS2R0N28B5AIdtdTYnu7zjO0.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-extensions-0.6-snj9TtDjAZ6b9UseOnnNS2R0N28B5AIdtdTYnu7zjO0.jar' is inaccessible.
Sep 26, 2021 3:48:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-26T15:48:35.395Z: Staged package junit-4.13.1-wwcZ25dNZFJ5P-GRs2OKV3cAVIW64UWSQERTD_pfYSI.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/junit-4.13.1-wwcZ25dNZFJ5P-GRs2OKV3cAVIW64UWSQERTD_pfYSI.jar' is inaccessible.
Sep 26, 2021 3:48:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-26T15:48:35.475Z: Staged package kafka-clients-2.4.1-or4Vrm01S3aXE64MuzGE8iWqFvyYIFseskr9n8KjYNk.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/kafka-clients-2.4.1-or4Vrm01S3aXE64MuzGE8iWqFvyYIFseskr9n8KjYNk.jar' is inaccessible.
Sep 26, 2021 3:48:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-26T15:48:35.554Z: Staged package lz4-java-1.6.0-0ilUWqKx1SA8h2YUvbz_yswwNpf0-PJvdk4dbB7S5BY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/lz4-java-1.6.0-0ilUWqKx1SA8h2YUvbz_yswwNpf0-PJvdk4dbB7S5BY.jar' is inaccessible.
Sep 26, 2021 3:48:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-09-26T15:48:36.322Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Sep 26, 2021 3:51:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-09-26T15:51:35.804Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Sep 26, 2021 3:54:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-26T15:54:33.626Z: Staged package classgraph-4.8.104-ePvfM6WdiJMBzk05ytbzmopew--T1rqGqw_PYcvi6h0.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/classgraph-4.8.104-ePvfM6WdiJMBzk05ytbzmopew--T1rqGqw_PYcvi6h0.jar' is inaccessible.
Sep 26, 2021 3:54:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-26T15:54:33.985Z: Staged package flogger-0.6-MUVmnbYMex1jtOH3E36hpd5uqRxRamjoz90leU2Bt8Q.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/flogger-0.6-MUVmnbYMex1jtOH3E36hpd5uqRxRamjoz90leU2Bt8Q.jar' is inaccessible.
Sep 26, 2021 3:54:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-26T15:54:34.044Z: Staged package flogger-system-backend-0.6-r3-fBJx59nlB4eR2N7IGNFlHiDmKoKq83bx68K899bE.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/flogger-system-backend-0.6-r3-fBJx59nlB4eR2N7IGNFlHiDmKoKq83bx68K899bE.jar' is inaccessible.
Sep 26, 2021 3:54:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-26T15:54:34.513Z: Staged package google-extensions-0.6-snj9TtDjAZ6b9UseOnnNS2R0N28B5AIdtdTYnu7zjO0.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-extensions-0.6-snj9TtDjAZ6b9UseOnnNS2R0N28B5AIdtdTYnu7zjO0.jar' is inaccessible.
Sep 26, 2021 3:54:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-26T15:54:35.533Z: Staged package junit-4.13.1-wwcZ25dNZFJ5P-GRs2OKV3cAVIW64UWSQERTD_pfYSI.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/junit-4.13.1-wwcZ25dNZFJ5P-GRs2OKV3cAVIW64UWSQERTD_pfYSI.jar' is inaccessible.
Sep 26, 2021 3:54:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-26T15:54:35.575Z: Staged package kafka-clients-2.4.1-or4Vrm01S3aXE64MuzGE8iWqFvyYIFseskr9n8KjYNk.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/kafka-clients-2.4.1-or4Vrm01S3aXE64MuzGE8iWqFvyYIFseskr9n8KjYNk.jar' is inaccessible.
Sep 26, 2021 3:54:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-26T15:54:35.652Z: Staged package lz4-java-1.6.0-0ilUWqKx1SA8h2YUvbz_yswwNpf0-PJvdk4dbB7S5BY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/lz4-java-1.6.0-0ilUWqKx1SA8h2YUvbz_yswwNpf0-PJvdk4dbB7S5BY.jar' is inaccessible.
Sep 26, 2021 3:54:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-09-26T15:54:36.391Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Sep 26, 2021 3:57:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-09-26T15:57:36.014Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Sep 26, 2021 4:00:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T16:00:26.128Z: Cancel request is committed for workflow job: 2021-09-26_05_45_19-3650379884586848115.
Sep 26, 2021 4:00:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T16:00:26.153Z: Cleaning up.
Sep 26, 2021 4:00:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T16:00:26.224Z: Stopping **** pool...
Sep 26, 2021 4:00:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T16:00:26.300Z: Stopping **** pool...
Sep 26, 2021 4:02:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T16:02:53.030Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Sep 26, 2021 4:02:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T16:02:53.060Z: Worker pool stopped.
Sep 26, 2021 4:02:58 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-09-26_05_45_19-3650379884586848115 finished with status CANCELLED.
Load test results for test (ID): f4198f48-e720-496d-807a-02271bf0782d and timestamp: 2021-09-26T12:45:14.541000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11558.366
dataflow_v2_java11_total_bytes_count             3.60383495E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210926124332
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:5678472199be1c1cf321f70f99b280886c12262960474a7c6e9194f86255e67b
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210926124332]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:5678472199be1c1cf321f70f99b280886c12262960474a7c6e9194f86255e67b]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210926124332] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:5678472199be1c1cf321f70f99b280886c12262960474a7c6e9194f86255e67b])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:5678472199be1c1cf321f70f99b280886c12262960474a7c6e9194f86255e67b
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:5678472199be1c1cf321f70f99b280886c12262960474a7c6e9194f86255e67b
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:5678472199be1c1cf321f70f99b280886c12262960474a7c6e9194f86255e67b].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 42s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/q5tp57ab4fjti

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #100

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/100/display/redirect?page=changes>

Changes:

[Robert Bradshaw] [BEAM-12934] Use environment capabilities to determine length prefixing.

[Brian Hulette] Add missing comma

[ryanthompson591] Update tensorflow to version 2.6.0

[zhoufek] [BEAM-9487] Fix incorrected Repeatedly.may_finish implementation

[noreply] Cleanup use of futures. (#15043)

[noreply] Go Lint fix for wordcount and metrics (#15580)

[Brian Hulette] Fix whitespace lint

[noreply] [BEAM-3304] Snippets for trigger in BPG (#15409)

[noreply] [BEAM-12832] Add Go SDK xlang info to programming guide. (#15447)

[noreply] [BEAM-11097] Add SideInputCache to StateReader (#15563)

[Robert Bradshaw] Revert "Avoid apiary submission of job graph when it is not needed.

[noreply] [BEAM-12769] Java-emulating external transform. (#15546)

[noreply] [BEAM-12913] Pass query priority from ReadAllFromBigQuery (#15584)

[noreply] [BEAM-11982] Java Spanner - Implement IO Request Count metrics (#15493)


------------------------------------------
[...truncated 49.63 KB...]
d402f4f1b906: Waiting
d00da3cd7763: Waiting
00ef5416d927: Waiting
3891808a925b: Waiting
4e61e63529c2: Waiting
5c94d85de49e: Waiting
799760671c38: Waiting
cf75dda46606: Waiting
1d45e4518a0b: Waiting
b7076cd14284: Pushed
f387b53ed4eb: Pushed
076fe47c73d8: Pushed
cf75dda46606: Pushed
66bcc507711f: Pushed
6c5de256224e: Pushed
d088eb68bda8: Pushed
5c94d85de49e: Pushed
3891808a925b: Layer already exists
d402f4f1b906: Layer already exists
00ef5416d927: Layer already exists
f8ac61d12b49: Pushed
8555e663f65b: Layer already exists
4e61e63529c2: Layer already exists
d00da3cd7763: Layer already exists
799760671c38: Layer already exists
420c8f6b2459: Pushed
1d45e4518a0b: Pushed
7a245f5f2301: Pushed
20210925124344: digest: sha256:ef445dbd7be2158e45a67cf1daf163a663fe113fa17e184c7d3cf03010be22fc size: 4311

> Task :sdks:java:testing:load-tests:run
Sep 25, 2021 12:46:00 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Sep 25, 2021 12:46:00 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 195 files. Enable logging at DEBUG level to see which files will be staged.
Sep 25, 2021 12:46:01 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Sep 25, 2021 12:46:01 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Sep 25, 2021 12:46:04 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 195 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Sep 25, 2021 12:46:06 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 195 files cached, 0 files newly uploaded in 2 seconds
Sep 25, 2021 12:46:06 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Sep 25, 2021 12:46:06 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <106569 bytes, hash 52a52e31e31ed2e46ac26e343b59a3d8073cfc52469f05425b0844aeeb3e89ed> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-UqUuMeMe0uRqwm40O1mj2Ac8_FJGnwVCWwhErus-ie0.pb
Sep 25, 2021 12:46:08 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Sep 25, 2021 12:46:08 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2a869a16, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@ae202c6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@46aa712c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6ada9c0c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7412ed6b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6e7c351d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b4a0aef, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@45cec376, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26d41711, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@298f0a0b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2b960a7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@31dfc6f5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@37b52340, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@663bb8ef, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2f4e40d7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@60e9c3a5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7e5843db, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@459f703f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@188ac8a3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3650d4fc]
Sep 25, 2021 12:46:08 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Sep 25, 2021 12:46:08 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Sep 25, 2021 12:46:08 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Sep 25, 2021 12:46:08 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Sep 25, 2021 12:46:08 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@441b8382, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1df1ced0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5349b246, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@32b0876c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2aaf152b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@39bbd9e0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@27fe9713, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@11c3ff67, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4397a639, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@24841372, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77114efe, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@79a7bfbc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77f68df, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3e4e4c1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7e7f3cfd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ae126d1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@46a488c2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6242ae3b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@65ddee5a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@56399b9e]
Sep 25, 2021 12:46:08 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Sep 25, 2021 12:46:08 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Sep 25, 2021 12:46:08 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Sep 25, 2021 12:46:08 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Sep 25, 2021 12:46:08 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Sep 25, 2021 12:46:08 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Sep 25, 2021 12:46:08 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Sep 25, 2021 12:46:08 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Sep 25, 2021 12:46:08 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Sep 25, 2021 12:46:08 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Sep 25, 2021 12:46:08 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Sep 25, 2021 12:46:08 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.34.0-SNAPSHOT
Sep 25, 2021 12:46:09 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-09-25_05_46_08-17003214306424250645?project=apache-beam-testing
Sep 25, 2021 12:46:09 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-09-25_05_46_08-17003214306424250645
Sep 25, 2021 12:46:09 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-09-25_05_46_08-17003214306424250645
Sep 25, 2021 12:46:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-09-25T12:46:14.497Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-09-eb79. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Sep 25, 2021 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-25T12:46:17.677Z: Worker configuration: e2-standard-2 in us-central1-a.
Sep 25, 2021 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-25T12:46:18.473Z: Expanding SplittableParDo operations into optimizable parts.
Sep 25, 2021 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-25T12:46:18.508Z: Expanding CollectionToSingleton operations into optimizable parts.
Sep 25, 2021 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-25T12:46:18.588Z: Expanding CoGroupByKey operations into optimizable parts.
Sep 25, 2021 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-25T12:46:18.654Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Sep 25, 2021 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-25T12:46:18.680Z: Expanding GroupByKey operations into streaming Read/Write steps
Sep 25, 2021 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-25T12:46:18.741Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Sep 25, 2021 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-25T12:46:18.838Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Sep 25, 2021 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-25T12:46:18.872Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Sep 25, 2021 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-25T12:46:18.905Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Sep 25, 2021 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-25T12:46:18.928Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Sep 25, 2021 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-25T12:46:18.950Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Sep 25, 2021 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-25T12:46:18.982Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Sep 25, 2021 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-25T12:46:19.016Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Sep 25, 2021 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-25T12:46:19.038Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Sep 25, 2021 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-25T12:46:19.065Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Sep 25, 2021 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-25T12:46:19.099Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Sep 25, 2021 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-25T12:46:19.124Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Sep 25, 2021 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-25T12:46:19.158Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Sep 25, 2021 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-25T12:46:19.191Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Sep 25, 2021 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-25T12:46:19.238Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Sep 25, 2021 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-25T12:46:19.265Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Sep 25, 2021 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-25T12:46:19.309Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Sep 25, 2021 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-25T12:46:19.336Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Sep 25, 2021 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-25T12:46:19.372Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Sep 25, 2021 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-25T12:46:19.405Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Sep 25, 2021 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-25T12:46:19.438Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Sep 25, 2021 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-25T12:46:19.474Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Sep 25, 2021 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-25T12:46:19.506Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Sep 25, 2021 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-25T12:46:19.539Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Sep 25, 2021 12:46:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-25T12:46:20.078Z: Starting 5 ****s in us-central1-a...
Sep 25, 2021 12:46:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-25T12:46:26.490Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Sep 25, 2021 12:47:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-25T12:47:00.317Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Sep 25, 2021 12:47:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-25T12:47:52.343Z: Workers have started successfully.
Sep 25, 2021 12:47:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-25T12:47:52.380Z: Workers have started successfully.
Sep 25, 2021 1:43:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-25T13:43:23.702Z: Staged package snakeyaml-1.27-fnzOZ0DtcFv9-qx7RCwTddKYbS8pNZNqW9QMFOGP1zY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/snakeyaml-1.27-fnzOZ0DtcFv9-qx7RCwTddKYbS8pNZNqW9QMFOGP1zY.jar' is inaccessible.
Sep 25, 2021 1:43:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-09-25T13:43:23.925Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Sep 25, 2021 1:46:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-09-25T13:46:22.533Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Sep 25, 2021 1:49:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-25T13:49:23.669Z: Staged package snakeyaml-1.27-fnzOZ0DtcFv9-qx7RCwTddKYbS8pNZNqW9QMFOGP1zY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/snakeyaml-1.27-fnzOZ0DtcFv9-qx7RCwTddKYbS8pNZNqW9QMFOGP1zY.jar' is inaccessible.
Sep 25, 2021 1:49:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-09-25T13:49:23.812Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Sep 25, 2021 1:52:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-09-25T13:52:23.509Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Sep 25, 2021 1:55:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-25T13:55:22.733Z: Staged package snakeyaml-1.27-fnzOZ0DtcFv9-qx7RCwTddKYbS8pNZNqW9QMFOGP1zY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/snakeyaml-1.27-fnzOZ0DtcFv9-qx7RCwTddKYbS8pNZNqW9QMFOGP1zY.jar' is inaccessible.
Sep 25, 2021 1:55:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-09-25T13:55:22.899Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Sep 25, 2021 1:58:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-09-25T13:58:23.628Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Sep 25, 2021 2:01:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-25T14:01:23.495Z: Staged package snakeyaml-1.27-fnzOZ0DtcFv9-qx7RCwTddKYbS8pNZNqW9QMFOGP1zY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/snakeyaml-1.27-fnzOZ0DtcFv9-qx7RCwTddKYbS8pNZNqW9QMFOGP1zY.jar' is inaccessible.
Sep 25, 2021 2:01:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-09-25T14:01:23.651Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Sep 25, 2021 2:04:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-09-25T14:04:23.675Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Sep 25, 2021 4:00:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-25T16:00:24.378Z: Cancel request is committed for workflow job: 2021-09-25_05_46_08-17003214306424250645.
Sep 25, 2021 4:00:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-25T16:00:24.425Z: Cleaning up.
Sep 25, 2021 4:00:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-25T16:00:24.515Z: Stopping **** pool...
Sep 25, 2021 4:00:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-25T16:00:24.581Z: Stopping **** pool...
Sep 25, 2021 4:02:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-25T16:02:53.480Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Sep 25, 2021 4:02:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-25T16:02:53.523Z: Worker pool stopped.
Sep 25, 2021 4:02:59 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-09-25_05_46_08-17003214306424250645 finished with status CANCELLED.
Load test results for test (ID): a64df3c6-23e3-4c70-bc22-3a5cfaa7c273 and timestamp: 2021-09-25T12:46:00.871000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11513.154
dataflow_v2_java11_total_bytes_count             3.40228439E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210925124344
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ef445dbd7be2158e45a67cf1daf163a663fe113fa17e184c7d3cf03010be22fc
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210925124344]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ef445dbd7be2158e45a67cf1daf163a663fe113fa17e184c7d3cf03010be22fc]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210925124344] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ef445dbd7be2158e45a67cf1daf163a663fe113fa17e184c7d3cf03010be22fc])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ef445dbd7be2158e45a67cf1daf163a663fe113fa17e184c7d3cf03010be22fc
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ef445dbd7be2158e45a67cf1daf163a663fe113fa17e184c7d3cf03010be22fc
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ef445dbd7be2158e45a67cf1daf163a663fe113fa17e184c7d3cf03010be22fc].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 36s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/jzofc55bpp5dk

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #99

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/99/display/redirect?page=changes>

Changes:

[noreply] Avoid setting empty builders in proto setters

[Brian Hulette] Add back 'more' break

[Robert Bradshaw] Fix some website logos.

[chuck.yang] Run BigQuery queries with batch priority

[chuck.yang] Allow changing query priority in ReadFromBigQuery

[chuck.yang] Update CHANGES

[kawaigin] Implicitly watch and track anonymous pipeline and PCollections

[chamikaramj] Reject requests when parameter names cannot be validated unless

[chamikaramj] Updates error message

[alexander.chermenin] [BEAM-10822] Fixed typo in BigqueryClient

[zyichi] [BEAM-12898] Minor fix to Kubernetes.groovy postBuildScript.

[noreply] [BEAM-12869] Bump tensorflow from 2.5.0 to 2.5.1 in

[noreply] [BEAM-12946] Fix SmallestPerKey, add unit testing (#15567)

[zyichi] [BEAM-12908] Sickbay pubsublite.ReadWriteIT

[noreply] Relocate Go SDK breaking change note out of template into 2.33.0.


------------------------------------------
[...truncated 48.54 KB...]
46ef6de51396: Preparing
0d891c597dbc: Preparing
b1f4c39338d2: Preparing
50dcb9edf7b8: Preparing
874a0a098966: Preparing
4ef5a9a214d8: Preparing
9e1a401c9ccd: Preparing
f6d860b1fb67: Preparing
181a53937f73: Preparing
894ae42d6f88: Preparing
3891808a925b: Preparing
d402f4f1b906: Preparing
00ef5416d927: Preparing
8555e663f65b: Preparing
d00da3cd7763: Preparing
4e61e63529c2: Preparing
799760671c38: Preparing
50dcb9edf7b8: Waiting
874a0a098966: Waiting
3891808a925b: Waiting
d402f4f1b906: Waiting
4ef5a9a214d8: Waiting
00ef5416d927: Waiting
9e1a401c9ccd: Waiting
894ae42d6f88: Waiting
8555e663f65b: Waiting
4e61e63529c2: Waiting
f6d860b1fb67: Waiting
799760671c38: Waiting
d00da3cd7763: Waiting
181a53937f73: Waiting
b1f4c39338d2: Pushed
d290b24e70a8: Pushed
46ef6de51396: Pushed
1cb465b10024: Pushed
50dcb9edf7b8: Pushed
0d891c597dbc: Pushed
4ef5a9a214d8: Pushed
181a53937f73: Pushed
9e1a401c9ccd: Pushed
3891808a925b: Layer already exists
874a0a098966: Pushed
d402f4f1b906: Layer already exists
00ef5416d927: Layer already exists
8555e663f65b: Layer already exists
d00da3cd7763: Layer already exists
4e61e63529c2: Layer already exists
799760671c38: Layer already exists
894ae42d6f88: Pushed
f6d860b1fb67: Pushed
20210924124334: digest: sha256:d32e0c80f8218aac64650bb53a88e7ec9ab5ff990a30aed32e4276b2fb6c7829 size: 4311

> Task :sdks:java:testing:load-tests:run
Sep 24, 2021 12:46:10 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Sep 24, 2021 12:46:11 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 195 files. Enable logging at DEBUG level to see which files will be staged.
Sep 24, 2021 12:46:11 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Sep 24, 2021 12:46:11 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Sep 24, 2021 12:46:14 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 195 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Sep 24, 2021 12:46:15 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 195 files cached, 0 files newly uploaded in 0 seconds
Sep 24, 2021 12:46:15 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Sep 24, 2021 12:46:15 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <106569 bytes, hash 9a74309829b03d9368f9bcd5e5fbd7c4d3815d4739eb9e6f8373f288760ac35a> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-mnQwmCmwPZNo-bzV5fvXxNOBXUc5655vg3PyiHYKw1o.pb
Sep 24, 2021 12:46:16 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Sep 24, 2021 12:46:16 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@ae202c6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@46aa712c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6ada9c0c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7412ed6b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6e7c351d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b4a0aef, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@45cec376, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26d41711, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@298f0a0b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2b960a7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@31dfc6f5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@37b52340, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@663bb8ef, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2f4e40d7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@60e9c3a5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7e5843db, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@459f703f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@188ac8a3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3650d4fc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@aaa0f76]
Sep 24, 2021 12:46:16 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Sep 24, 2021 12:46:16 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Sep 24, 2021 12:46:16 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Sep 24, 2021 12:46:16 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Sep 24, 2021 12:46:16 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1df1ced0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5349b246, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@32b0876c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2aaf152b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@39bbd9e0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@27fe9713, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@11c3ff67, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4397a639, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@24841372, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77114efe, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@79a7bfbc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77f68df, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3e4e4c1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7e7f3cfd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ae126d1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@46a488c2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6242ae3b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@65ddee5a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@56399b9e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@34b9eb03]
Sep 24, 2021 12:46:16 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Sep 24, 2021 12:46:16 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Sep 24, 2021 12:46:16 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Sep 24, 2021 12:46:16 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Sep 24, 2021 12:46:16 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Sep 24, 2021 12:46:16 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Sep 24, 2021 12:46:16 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Sep 24, 2021 12:46:16 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Sep 24, 2021 12:46:16 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Sep 24, 2021 12:46:16 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Sep 24, 2021 12:46:16 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Sep 24, 2021 12:46:16 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.34.0-SNAPSHOT
Sep 24, 2021 12:46:18 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-09-24_05_46_17-15053782412180795083?project=apache-beam-testing
Sep 24, 2021 12:46:18 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-09-24_05_46_17-15053782412180795083
Sep 24, 2021 12:46:18 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-09-24_05_46_17-15053782412180795083
Sep 24, 2021 12:46:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-09-24T12:46:23.048Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-09-jvhc. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Sep 24, 2021 12:46:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-24T12:46:26.739Z: Worker configuration: e2-standard-2 in us-central1-a.
Sep 24, 2021 12:46:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-24T12:46:27.444Z: Expanding SplittableParDo operations into optimizable parts.
Sep 24, 2021 12:46:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-24T12:46:27.481Z: Expanding CollectionToSingleton operations into optimizable parts.
Sep 24, 2021 12:46:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-24T12:46:27.566Z: Expanding CoGroupByKey operations into optimizable parts.
Sep 24, 2021 12:46:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-24T12:46:27.651Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Sep 24, 2021 12:46:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-24T12:46:27.691Z: Expanding GroupByKey operations into streaming Read/Write steps
Sep 24, 2021 12:46:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-24T12:46:27.765Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Sep 24, 2021 12:46:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-24T12:46:27.889Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Sep 24, 2021 12:46:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-24T12:46:27.920Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Sep 24, 2021 12:46:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-24T12:46:27.957Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Sep 24, 2021 12:46:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-24T12:46:27.994Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Sep 24, 2021 12:46:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-24T12:46:28.035Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Sep 24, 2021 12:46:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-24T12:46:28.063Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Sep 24, 2021 12:46:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-24T12:46:28.095Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Sep 24, 2021 12:46:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-24T12:46:28.135Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Sep 24, 2021 12:46:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-24T12:46:28.162Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Sep 24, 2021 12:46:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-24T12:46:28.213Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Sep 24, 2021 12:46:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-24T12:46:28.263Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Sep 24, 2021 12:46:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-24T12:46:28.299Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Sep 24, 2021 12:46:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-24T12:46:28.340Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Sep 24, 2021 12:46:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-24T12:46:28.382Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Sep 24, 2021 12:46:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-24T12:46:28.416Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Sep 24, 2021 12:46:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-24T12:46:28.437Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Sep 24, 2021 12:46:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-24T12:46:28.464Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Sep 24, 2021 12:46:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-24T12:46:28.504Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Sep 24, 2021 12:46:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-24T12:46:28.537Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Sep 24, 2021 12:46:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-24T12:46:28.580Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Sep 24, 2021 12:46:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-24T12:46:28.604Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Sep 24, 2021 12:46:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-24T12:46:28.631Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Sep 24, 2021 12:46:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-24T12:46:28.664Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Sep 24, 2021 12:46:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-24T12:46:29.069Z: Starting 5 ****s in us-central1-a...
Sep 24, 2021 12:46:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-24T12:46:45.307Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Sep 24, 2021 12:47:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-24T12:47:14.742Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Sep 24, 2021 12:48:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-24T12:48:11.484Z: Workers have started successfully.
Sep 24, 2021 12:48:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-24T12:48:11.505Z: Workers have started successfully.
Sep 24, 2021 3:03:53 PM org.apache.beam.sdk.metrics.MetricsEnvironment getCurrentContainer
WARNING: Reporting metrics are not supported in the current execution environment.
Sep 24, 2021 4:00:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-24T16:00:25.712Z: Cancel request is committed for workflow job: 2021-09-24_05_46_17-15053782412180795083.
Sep 24, 2021 4:00:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-24T16:00:25.820Z: Cleaning up.
Sep 24, 2021 4:00:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-24T16:00:25.903Z: Stopping **** pool...
Sep 24, 2021 4:00:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-24T16:00:25.956Z: Stopping **** pool...
Sep 24, 2021 4:02:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-24T16:02:50.168Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Sep 24, 2021 4:02:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-24T16:02:50.217Z: Worker pool stopped.
Sep 24, 2021 4:02:55 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-09-24_05_46_17-15053782412180795083 finished with status CANCELLED.
Load test results for test (ID): 9a89dc7e-2ad0-493e-8495-f10da3967164 and timestamp: 2021-09-24T12:46:11.349000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11499.376
dataflow_v2_java11_total_bytes_count             2.45097132E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210924124334
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:d32e0c80f8218aac64650bb53a88e7ec9ab5ff990a30aed32e4276b2fb6c7829
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210924124334]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:d32e0c80f8218aac64650bb53a88e7ec9ab5ff990a30aed32e4276b2fb6c7829]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210924124334] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:d32e0c80f8218aac64650bb53a88e7ec9ab5ff990a30aed32e4276b2fb6c7829])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:d32e0c80f8218aac64650bb53a88e7ec9ab5ff990a30aed32e4276b2fb6c7829
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:d32e0c80f8218aac64650bb53a88e7ec9ab5ff990a30aed32e4276b2fb6c7829
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:d32e0c80f8218aac64650bb53a88e7ec9ab5ff990a30aed32e4276b2fb6c7829].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 38s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/z4irkwzm6pp5a

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #98

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/98/display/redirect?page=changes>

Changes:

[kileysok] Add MapState and SetState support

[Kyle Weaver] [BEAM-12898] Use new postbuildscript dsl in Flink load tests

[Kyle Weaver] run postbuild regardless of test result

[Kyle Weaver] spotless

[noreply] Merge pull request #15487 from [BEAM-12812] - Run Github Actions on GCP

[noreply] [BEAM-11097] Add SideInputCache to harness control type (#15530)

[kawaigin] Updated interactive integration test golden screenshots.

[noreply] Revert PR 15487 (BEAM-12812) (#15554)

[zyichi] [BEAM-12898] Trying out solution suggestion for JENKINS-66189 to solve

[noreply] [BEAM-12383] Adding Go SDK and Kafka IO to Gradle cross-language test


------------------------------------------
[...truncated 49.57 KB...]
77ce3c40d526: Preparing
747a4eef8d96: Preparing
05333715e415: Preparing
f3117e044d65: Preparing
199149875d8f: Preparing
f573d43a7df5: Preparing
6d9f8408d1d8: Preparing
3891808a925b: Preparing
d402f4f1b906: Preparing
00ef5416d927: Preparing
8555e663f65b: Preparing
d00da3cd7763: Preparing
4e61e63529c2: Preparing
799760671c38: Preparing
f573d43a7df5: Waiting
05333715e415: Waiting
d402f4f1b906: Waiting
f3117e044d65: Waiting
6d9f8408d1d8: Waiting
199149875d8f: Waiting
00ef5416d927: Waiting
3891808a925b: Waiting
8555e663f65b: Waiting
77ce3c40d526: Waiting
799760671c38: Waiting
66d0e622a342: Pushed
40c1063b98f7: Pushed
8b4798633287: Pushed
563013fdf011: Pushed
77ce3c40d526: Pushed
542d82ea7219: Pushed
05333715e415: Pushed
f3117e044d65: Pushed
f573d43a7df5: Pushed
3891808a925b: Layer already exists
d402f4f1b906: Layer already exists
00ef5416d927: Layer already exists
8555e663f65b: Layer already exists
d00da3cd7763: Layer already exists
799760671c38: Layer already exists
4e61e63529c2: Layer already exists
747a4eef8d96: Pushed
6d9f8408d1d8: Pushed
199149875d8f: Pushed
20210923124330: digest: sha256:b825d3afd98b306411674734fe358a4dcb9108480a7dd506e1b2cf5cc1204c9b size: 4311

> Task :sdks:java:testing:load-tests:run
Sep 23, 2021 12:45:29 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Sep 23, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 195 files. Enable logging at DEBUG level to see which files will be staged.
Sep 23, 2021 12:45:30 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Sep 23, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Sep 23, 2021 12:45:33 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 195 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Sep 23, 2021 12:45:33 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 195 files cached, 0 files newly uploaded in 0 seconds
Sep 23, 2021 12:45:34 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Sep 23, 2021 12:45:34 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <106569 bytes, hash 98cb4a294d1cd10162b45f0d703e1e60c1959aaaa08d532c62d5a2136630e364> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-mMtKKU0c0QFitF8NcD4eYMGVmqqgjVMsYtWiE2Yw42Q.pb
Sep 23, 2021 12:45:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Sep 23, 2021 12:45:35 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2a869a16, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@ae202c6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@46aa712c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6ada9c0c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7412ed6b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6e7c351d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b4a0aef, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@45cec376, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26d41711, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@298f0a0b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2b960a7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@31dfc6f5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@37b52340, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@663bb8ef, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2f4e40d7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@60e9c3a5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7e5843db, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@459f703f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@188ac8a3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3650d4fc]
Sep 23, 2021 12:45:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Sep 23, 2021 12:45:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Sep 23, 2021 12:45:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Sep 23, 2021 12:45:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Sep 23, 2021 12:45:35 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@441b8382, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1df1ced0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5349b246, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@32b0876c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2aaf152b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@39bbd9e0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@27fe9713, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@11c3ff67, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4397a639, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@24841372, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77114efe, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@79a7bfbc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77f68df, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3e4e4c1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7e7f3cfd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ae126d1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@46a488c2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6242ae3b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@65ddee5a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@56399b9e]
Sep 23, 2021 12:45:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Sep 23, 2021 12:45:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Sep 23, 2021 12:45:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Sep 23, 2021 12:45:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Sep 23, 2021 12:45:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Sep 23, 2021 12:45:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Sep 23, 2021 12:45:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Sep 23, 2021 12:45:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Sep 23, 2021 12:45:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Sep 23, 2021 12:45:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Sep 23, 2021 12:45:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Sep 23, 2021 12:45:35 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.34.0-SNAPSHOT
Sep 23, 2021 12:45:37 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-09-23_05_45_35-8041644847459113944?project=apache-beam-testing
Sep 23, 2021 12:45:37 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-09-23_05_45_35-8041644847459113944
Sep 23, 2021 12:45:37 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-09-23_05_45_35-8041644847459113944
Sep 23, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-09-23T12:45:42.329Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-09-9zc7. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Sep 23, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-23T12:45:47.196Z: Worker configuration: e2-standard-2 in us-central1-a.
Sep 23, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-23T12:45:47.872Z: Expanding SplittableParDo operations into optimizable parts.
Sep 23, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-23T12:45:47.908Z: Expanding CollectionToSingleton operations into optimizable parts.
Sep 23, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-23T12:45:47.969Z: Expanding CoGroupByKey operations into optimizable parts.
Sep 23, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-23T12:45:48.044Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Sep 23, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-23T12:45:48.078Z: Expanding GroupByKey operations into streaming Read/Write steps
Sep 23, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-23T12:45:48.145Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Sep 23, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-23T12:45:48.262Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Sep 23, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-23T12:45:48.286Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Sep 23, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-23T12:45:48.317Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Sep 23, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-23T12:45:48.343Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Sep 23, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-23T12:45:48.425Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Sep 23, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-23T12:45:48.458Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Sep 23, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-23T12:45:48.496Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Sep 23, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-23T12:45:48.527Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Sep 23, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-23T12:45:48.551Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Sep 23, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-23T12:45:48.584Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Sep 23, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-23T12:45:48.638Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Sep 23, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-23T12:45:48.680Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Sep 23, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-23T12:45:48.708Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Sep 23, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-23T12:45:48.744Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Sep 23, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-23T12:45:48.770Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Sep 23, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-23T12:45:48.800Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Sep 23, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-23T12:45:48.821Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Sep 23, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-23T12:45:48.853Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Sep 23, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-23T12:45:48.877Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Sep 23, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-23T12:45:48.899Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Sep 23, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-23T12:45:48.921Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Sep 23, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-23T12:45:48.954Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Sep 23, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-23T12:45:48.978Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Sep 23, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-23T12:45:49.352Z: Starting 5 ****s in us-central1-a...
Sep 23, 2021 12:46:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-23T12:46:17.031Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Sep 23, 2021 12:46:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-23T12:46:19.446Z: Autoscaling: Raised the number of ****s to 1 so that the pipeline can catch up with its backlog and keep up with its input rate.
Sep 23, 2021 12:46:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-23T12:46:19.478Z: Resized **** pool to 1, though goal was 5.  This could be a quota issue.
Sep 23, 2021 12:46:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-23T12:46:29.806Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Sep 23, 2021 12:47:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-23T12:47:24.223Z: Workers have started successfully.
Sep 23, 2021 12:47:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-23T12:47:24.264Z: Workers have started successfully.
Sep 23, 2021 2:03:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-23T14:03:52.651Z: Staged package jackson-dataformat-cbor-2.12.4-xpFy2QEl4dUh55AUzgOueyVcIoNGuAdPK6G3l4P-yCc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/jackson-dataformat-cbor-2.12.4-xpFy2QEl4dUh55AUzgOueyVcIoNGuAdPK6G3l4P-yCc.jar' is inaccessible.
Sep 23, 2021 2:03:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-09-23T14:03:54.143Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Sep 23, 2021 4:00:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-23T16:00:33.259Z: Cancel request is committed for workflow job: 2021-09-23_05_45_35-8041644847459113944.
Sep 23, 2021 4:00:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-23T16:00:33.315Z: Cleaning up.
Sep 23, 2021 4:00:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-23T16:00:33.399Z: Stopping **** pool...
Sep 23, 2021 4:00:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-23T16:00:33.462Z: Stopping **** pool...
Sep 23, 2021 4:02:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-23T16:02:53.617Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Sep 23, 2021 4:02:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-23T16:02:53.654Z: Worker pool stopped.
Sep 23, 2021 4:03:00 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-09-23_05_45_35-8041644847459113944 finished with status CANCELLED.
Load test results for test (ID): 0f75d1c2-d44a-4145-87a0-b2cea919cdc1 and timestamp: 2021-09-23T12:45:30.189000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11524.165
dataflow_v2_java11_total_bytes_count             2.93525952E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210923124330
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:b825d3afd98b306411674734fe358a4dcb9108480a7dd506e1b2cf5cc1204c9b
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210923124330]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:b825d3afd98b306411674734fe358a4dcb9108480a7dd506e1b2cf5cc1204c9b]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210923124330] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:b825d3afd98b306411674734fe358a4dcb9108480a7dd506e1b2cf5cc1204c9b])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:b825d3afd98b306411674734fe358a4dcb9108480a7dd506e1b2cf5cc1204c9b
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:b825d3afd98b306411674734fe358a4dcb9108480a7dd506e1b2cf5cc1204c9b
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:b825d3afd98b306411674734fe358a4dcb9108480a7dd506e1b2cf5cc1204c9b].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 52s
101 actionable tasks: 71 executed, 28 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/kolxg2tci2sn4

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #97

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/97/display/redirect?page=changes>

Changes:

[noreply] [BEAM-12919] Removed IBM Streams from runner matrix (#15542)

[danthev] Fix 2.32.0 release notes.

[noreply] [BEAM-12258] Re-throw exception from forked thread in

[kawaigin] [BEAM-10708] Added an example notebook for beam_sql magic

[noreply] Add a timeout for BQ streaming_insert RPCS (#15541)

[noreply] Merge pull request #15537 from [BEAM-12908] Add a sleep to the IT after


------------------------------------------
[...truncated 49.31 KB...]
b0c95b2954a3: Pushed
75862a2f912c: Pushed
3c77729b931b: Pushed
e0858b5565a7: Pushed
c23774ca1987: Pushed
3891808a925b: Layer already exists
d402f4f1b906: Layer already exists
00ef5416d927: Layer already exists
8555e663f65b: Layer already exists
d00da3cd7763: Layer already exists
4e61e63529c2: Layer already exists
b63bcb3be20b: Pushed
799760671c38: Layer already exists
5d294ac669be: Pushed
c036ede0a712: Pushed
24d403efe7b4: Pushed
20210922124335: digest: sha256:eff74314ab303743ab8314e947d1999c5f0b633378bc0789128ffce6bf27b20a size: 4311

> Task :sdks:java:testing:load-tests:run
Sep 22, 2021 12:45:33 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Sep 22, 2021 12:45:33 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 195 files. Enable logging at DEBUG level to see which files will be staged.
Sep 22, 2021 12:45:34 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Sep 22, 2021 12:45:34 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Sep 22, 2021 12:45:36 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 195 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Sep 22, 2021 12:45:36 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 195 files cached, 0 files newly uploaded in 0 seconds
Sep 22, 2021 12:45:36 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Sep 22, 2021 12:45:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <106569 bytes, hash fbdd9ba961cf8321bc47225252568573fc4a4cb0e7fb71b2ccd623e964808a11> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline--92bqWHPgyG8RyJSUlaFc_xKTLDn-3GyzNYj6WSAihE.pb
Sep 22, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Sep 22, 2021 12:45:38 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2eeb0f9b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1b1c538d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1645f294, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6325f352, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@15c4af7a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6cbd0674, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@55d58825, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@19a64eae, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@29a98d9f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2da3b078, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@544e8149, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7fb66650, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1a96d94c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2a869a16, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@ae202c6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@46aa712c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6ada9c0c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7412ed6b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6e7c351d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b4a0aef]
Sep 22, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Sep 22, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Sep 22, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Sep 22, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Sep 22, 2021 12:45:38 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2e5b7fba, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@27755487, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4f0cab0a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@fe7b6b0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7ab4ae59, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77681ce4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5d96bdf8, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6f76c2cc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@306f6f1d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7d7cac8, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6fc6deb7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@367f0121, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7da39774, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@441b8382, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1df1ced0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5349b246, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@32b0876c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2aaf152b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@39bbd9e0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@27fe9713]
Sep 22, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Sep 22, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Sep 22, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Sep 22, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Sep 22, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Sep 22, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Sep 22, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Sep 22, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Sep 22, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Sep 22, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Sep 22, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Sep 22, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.34.0-SNAPSHOT
Sep 22, 2021 12:45:39 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-09-22_05_45_38-8630686692093343177?project=apache-beam-testing
Sep 22, 2021 12:45:39 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-09-22_05_45_38-8630686692093343177
Sep 22, 2021 12:45:39 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-09-22_05_45_38-8630686692093343177
Sep 22, 2021 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-09-22T12:45:46.517Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-09-i48w. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Sep 22, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-22T12:45:50.707Z: Worker configuration: e2-standard-2 in us-central1-a.
Sep 22, 2021 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-22T12:45:51.572Z: Expanding SplittableParDo operations into optimizable parts.
Sep 22, 2021 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-22T12:45:51.608Z: Expanding CollectionToSingleton operations into optimizable parts.
Sep 22, 2021 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-22T12:45:51.673Z: Expanding CoGroupByKey operations into optimizable parts.
Sep 22, 2021 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-22T12:45:51.753Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Sep 22, 2021 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-22T12:45:51.793Z: Expanding GroupByKey operations into streaming Read/Write steps
Sep 22, 2021 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-22T12:45:51.861Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Sep 22, 2021 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-22T12:45:51.956Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Sep 22, 2021 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-22T12:45:51.989Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Sep 22, 2021 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-22T12:45:52.026Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Sep 22, 2021 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-22T12:45:52.061Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Sep 22, 2021 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-22T12:45:52.091Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Sep 22, 2021 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-22T12:45:52.124Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Sep 22, 2021 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-22T12:45:52.156Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Sep 22, 2021 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-22T12:45:52.189Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Sep 22, 2021 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-22T12:45:52.226Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Sep 22, 2021 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-22T12:45:52.262Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Sep 22, 2021 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-22T12:45:52.295Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Sep 22, 2021 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-22T12:45:52.326Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Sep 22, 2021 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-22T12:45:52.354Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Sep 22, 2021 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-22T12:45:52.392Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Sep 22, 2021 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-22T12:45:52.429Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Sep 22, 2021 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-22T12:45:52.464Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Sep 22, 2021 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-22T12:45:52.498Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Sep 22, 2021 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-22T12:45:52.535Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Sep 22, 2021 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-22T12:45:52.557Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Sep 22, 2021 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-22T12:45:52.592Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Sep 22, 2021 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-22T12:45:52.623Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Sep 22, 2021 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-22T12:45:52.646Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Sep 22, 2021 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-22T12:45:52.677Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Sep 22, 2021 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-22T12:45:53.039Z: Starting 5 ****s in us-central1-a...
Sep 22, 2021 12:46:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-22T12:46:03.523Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Sep 22, 2021 12:46:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-22T12:46:22.700Z: Autoscaling: Raised the number of ****s to 1 so that the pipeline can catch up with its backlog and keep up with its input rate.
Sep 22, 2021 12:46:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-22T12:46:22.729Z: Resized **** pool to 1, though goal was 5.  This could be a quota issue.
Sep 22, 2021 12:46:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-22T12:46:32.989Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Sep 22, 2021 12:47:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-22T12:47:29.757Z: Workers have started successfully.
Sep 22, 2021 12:47:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-22T12:47:29.788Z: Workers have started successfully.
Sep 22, 2021 2:09:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-22T14:09:53.191Z: Staged package amazon-kinesis-producer-0.14.1-5YYT_oyDFV17VWs8Of43O7N08sRfHu0SuJeqVSFCuLg.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/amazon-kinesis-producer-0.14.1-5YYT_oyDFV17VWs8Of43O7N08sRfHu0SuJeqVSFCuLg.jar' is inaccessible.
Sep 22, 2021 2:09:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-09-22T14:09:57.035Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Sep 22, 2021 4:00:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-22T16:00:27.380Z: Cancel request is committed for workflow job: 2021-09-22_05_45_38-8630686692093343177.
Sep 22, 2021 4:00:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-22T16:00:27.416Z: Cleaning up.
Sep 22, 2021 4:00:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-22T16:00:27.491Z: Stopping **** pool...
Sep 22, 2021 4:00:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-22T16:00:27.565Z: Stopping **** pool...
Sep 22, 2021 4:02:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-22T16:02:44.163Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Sep 22, 2021 4:02:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-22T16:02:44.201Z: Worker pool stopped.
Sep 22, 2021 4:02:49 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-09-22_05_45_38-8630686692093343177 finished with status CANCELLED.
Load test results for test (ID): 7e9be1e1-4f39-4908-9b1d-ea8004aa235b and timestamp: 2021-09-22T12:45:33.904000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11535.738
dataflow_v2_java11_total_bytes_count             2.92455941E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210922124335
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:eff74314ab303743ab8314e947d1999c5f0b633378bc0789128ffce6bf27b20a
Deleted: sha256:13456361d1b2fbc317b58dedd4c54c27ac5f742edfd3eaccc5528e9ac6160e94
Deleted: sha256:b3bf7ec2f7552e27427a868c7b901bed80a7bb312c9a68532f488a9da140f4b9
Deleted: sha256:76d19613a00da22afdd6e5ad3516ce02e0d388dc69fd7795c0a5cdc63bc52f75
Deleted: sha256:70181a744720f29fd5615ec56fce13bcf27a967405a0483ad3df1afc59df016f
Deleted: sha256:cf1acda140f19415c74687bb2abba4eea28fbb9a8c91b1c5854c6bb8f13e8850
Deleted: sha256:687d5a5d634adc80adb4a9701440a2954bf8dd4f4c691e17030fe0119ea79ff6
Deleted: sha256:ee219303f9b8e622d62b7a53aa16cf9bc8b25e2bdbe5ee38d272b32b488de3af
Deleted: sha256:f7787378d4113a51c42376eb1a1b368d2400dbac8537c078e881c520bbb0df79
Deleted: sha256:c5263b87bad64aa125a9ca3cc37fed078f7fa58066dc96056bc87cf5a449739a
Deleted: sha256:7dff09d37c5974b025478c8995f588bb466b8242785edbcd5138236ff37e9b9e
Deleted: sha256:a55477630e507e10750021b0e73ed7ffa3c7fa74cef86c423723c4eb74854aba
Deleted: sha256:1fecfd038bcaf4276e8ede41605ed687b653e06820a6883d224d5f2d12c43fcd
Deleted: sha256:7669ab5ce7d9ace61d245839652dab2eedc4d767598ee2d1e95a8dd76c24f701
Deleted: sha256:5c48581ce374529e720e84ffa0c20919f66f571df54cde3277ce2a377e37c03f
Deleted: sha256:c09a7fa64d597b371f04e4365e8dfe7ebd8d2f17bbf699d9f0ba3eea6c1db9ed
Deleted: sha256:d3c0aa9f2368fbafb5b1110731164576329b49c98fa93466b68b0f8663e3cb92
Deleted: sha256:784cc9603582840701620872deac520f54d9ac58dc863137bbdaf8211c45e988
Deleted: sha256:8d85aa290abd9dd675feec91a8e7b836080b4fd3be512d212156b5616c8b083d
Deleted: sha256:7cb2d62743b08bc8c22a0a5998280efab6cb0e9b4e2b1a9a8940c7d50367e898
Deleted: sha256:d3fcd50dfe22262d70cd51149d5cb9ef7f7dcf5ffdf100d855ba5e4a15415401
Deleted: sha256:6bfd67221536ed73ac3b1948d682ab8287a85662e4b2431b22a6906d4f149409
Deleted: sha256:810a2f68192eee2fbdd5a32f6cb13b6e47dbffaf3a80913bcd23c6ffcf15d620
Deleted: sha256:02e087a3cd8fff999a51746db2e1fd1a4ba9bac8b53e813a9b97b25c598f89a1
Deleted: sha256:2b18d073580d2e89148ce234a7b49f779c134587f8dfe49b4c4ad0a499113fd2
Deleted: sha256:d2f785efc30adaf5d0ebf3cd41162fd88c1b78f5f6884d11e6741536061c6e6b
Deleted: sha256:7df056af9fdca24107e272aec2cbbc8d5b930657364c10b8460dee67c37fb09a
Deleted: sha256:b6c63cd83a11dd4c8aa3449be982e3f3acfe3ad3acfb16a564100063e5ef967c
Deleted: sha256:76a645bf1651ec0c5e745edc178a4fa533b2fdfe4ff2e109c4d37336b898d60d
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210922124335]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:eff74314ab303743ab8314e947d1999c5f0b633378bc0789128ffce6bf27b20a]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210922124335] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:eff74314ab303743ab8314e947d1999c5f0b633378bc0789128ffce6bf27b20a])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:eff74314ab303743ab8314e947d1999c5f0b633378bc0789128ffce6bf27b20a
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:eff74314ab303743ab8314e947d1999c5f0b633378bc0789128ffce6bf27b20a
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:eff74314ab303743ab8314e947d1999c5f0b633378bc0789128ffce6bf27b20a].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 33s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/r4w6jff3bwbha

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #96

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/96/display/redirect?page=changes>

Changes:

[chamikaramj] Python support for directly using Java transforms using constructor and

[chamikaramj] Fixes yapf

[chamikaramj] Fixes lint

[chamikaramj] Addressing reviewer comments

[chamikaramj] Adds support for a field name format that will be ignored at expansion

[danthev] Fix flaky test.

[danthev] Fix lint errors.

[Robert Bradshaw] Additional CoGBK tests.

[chamikaramj] Addresses reviewer comments

[chamikaramj] Use correct ignore field prefix in Python side

[noreply] [BEAM-12803] Update deprecated use of _field_types (#15539)

[Robert Bradshaw] Move CoGBK tests into appropreate module.


------------------------------------------
[...truncated 49.75 KB...]
d00da3cd7763: Waiting
799760671c38: Waiting
d402f4f1b906: Waiting
8555e663f65b: Waiting
00ef5416d927: Waiting
28e102d0ceaa: Pushed
3e9dfe7530d6: Pushed
07e78b1349a3: Pushed
3de5088183ac: Pushed
6ad30953503c: Pushed
687490d53da3: Pushed
672268a545f2: Pushed
2e01ff9ddad4: Pushed
3891808a925b: Layer already exists
d402f4f1b906: Layer already exists
00ef5416d927: Layer already exists
8555e663f65b: Layer already exists
67cb3d00a566: Pushed
d00da3cd7763: Layer already exists
7e9d4d56b4da: Pushed
799760671c38: Layer already exists
4e61e63529c2: Layer already exists
0d184f2ec06c: Pushed
942298825dbf: Pushed
20210921124331: digest: sha256:ffc8a8a8aefa672ed766bb5c4b710d35a7fe8967874ee70d2fc271cc040cd37b size: 4311

> Task :sdks:java:testing:load-tests:run
Sep 21, 2021 12:45:25 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Sep 21, 2021 12:45:26 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 195 files. Enable logging at DEBUG level to see which files will be staged.
Sep 21, 2021 12:45:26 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Sep 21, 2021 12:45:26 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Sep 21, 2021 12:45:28 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 195 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Sep 21, 2021 12:45:29 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 195 files cached, 0 files newly uploaded in 0 seconds
Sep 21, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Sep 21, 2021 12:45:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <106569 bytes, hash c60db03395aede5fe28e6856a620d116dd20385ccaa0bb8a3b0165dd4eeaa0dd> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-xg2wM5Wu3l_ijmhWpiDRFt0gOFzKoLuKOwFl3U7qoN0.pb
Sep 21, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Sep 21, 2021 12:45:31 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1b1c538d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1645f294, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6325f352, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@15c4af7a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6cbd0674, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@55d58825, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@19a64eae, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@29a98d9f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2da3b078, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@544e8149, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7fb66650, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1a96d94c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2a869a16, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@ae202c6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@46aa712c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6ada9c0c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7412ed6b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6e7c351d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b4a0aef, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@45cec376]
Sep 21, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Sep 21, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Sep 21, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Sep 21, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Sep 21, 2021 12:45:31 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@27755487, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4f0cab0a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@fe7b6b0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7ab4ae59, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77681ce4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5d96bdf8, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6f76c2cc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@306f6f1d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7d7cac8, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6fc6deb7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@367f0121, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7da39774, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@441b8382, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1df1ced0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5349b246, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@32b0876c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2aaf152b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@39bbd9e0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@27fe9713, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@11c3ff67]
Sep 21, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Sep 21, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Sep 21, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Sep 21, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Sep 21, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Sep 21, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Sep 21, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Sep 21, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Sep 21, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Sep 21, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Sep 21, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Sep 21, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.34.0-SNAPSHOT
Sep 21, 2021 12:45:32 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-09-21_05_45_31-8373750593306316118?project=apache-beam-testing
Sep 21, 2021 12:45:32 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-09-21_05_45_31-8373750593306316118
Sep 21, 2021 12:45:32 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-09-21_05_45_31-8373750593306316118
Sep 21, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-09-21T12:45:39.445Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-09-b8wl. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Sep 21, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-21T12:45:42.785Z: Worker configuration: e2-standard-2 in us-central1-a.
Sep 21, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-21T12:45:43.591Z: Expanding SplittableParDo operations into optimizable parts.
Sep 21, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-21T12:45:43.630Z: Expanding CollectionToSingleton operations into optimizable parts.
Sep 21, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-21T12:45:43.698Z: Expanding CoGroupByKey operations into optimizable parts.
Sep 21, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-21T12:45:43.770Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Sep 21, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-21T12:45:43.790Z: Expanding GroupByKey operations into streaming Read/Write steps
Sep 21, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-21T12:45:43.855Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Sep 21, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-21T12:45:43.950Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Sep 21, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-21T12:45:43.985Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Sep 21, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-21T12:45:44.022Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Sep 21, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-21T12:45:44.046Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Sep 21, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-21T12:45:44.079Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Sep 21, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-21T12:45:44.104Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Sep 21, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-21T12:45:44.130Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Sep 21, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-21T12:45:44.162Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Sep 21, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-21T12:45:44.192Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Sep 21, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-21T12:45:44.222Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Sep 21, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-21T12:45:44.247Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Sep 21, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-21T12:45:44.277Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Sep 21, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-21T12:45:44.304Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Sep 21, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-21T12:45:44.339Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Sep 21, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-21T12:45:44.366Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Sep 21, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-21T12:45:44.395Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Sep 21, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-21T12:45:44.427Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Sep 21, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-21T12:45:44.463Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Sep 21, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-21T12:45:44.509Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Sep 21, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-21T12:45:44.544Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Sep 21, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-21T12:45:44.571Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Sep 21, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-21T12:45:44.607Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Sep 21, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-21T12:45:44.639Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Sep 21, 2021 12:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-21T12:45:44.961Z: Starting 5 ****s in us-central1-a...
Sep 21, 2021 12:46:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-21T12:46:12.810Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Sep 21, 2021 12:46:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-21T12:46:28.732Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Sep 21, 2021 12:47:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-21T12:47:21.847Z: Workers have started successfully.
Sep 21, 2021 12:47:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-21T12:47:21.867Z: Workers have started successfully.
Sep 21, 2021 4:00:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-21T16:00:25.219Z: Cancel request is committed for workflow job: 2021-09-21_05_45_31-8373750593306316118.
Sep 21, 2021 4:00:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-21T16:00:25.285Z: Cleaning up.
Sep 21, 2021 4:00:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-21T16:00:25.353Z: Stopping **** pool...
Sep 21, 2021 4:00:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-21T16:00:25.408Z: Stopping **** pool...
Sep 21, 2021 4:02:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-21T16:02:36.748Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Sep 21, 2021 4:02:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-21T16:02:36.791Z: Worker pool stopped.
Sep 21, 2021 4:02:42 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-09-21_05_45_31-8373750593306316118 finished with status CANCELLED.
Load test results for test (ID): 32adc8e4-ca77-4235-b316-49aaa60d6e99 and timestamp: 2021-09-21T12:45:26.590000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11510.835
dataflow_v2_java11_total_bytes_count             3.09313067E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210921124331
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ffc8a8a8aefa672ed766bb5c4b710d35a7fe8967874ee70d2fc271cc040cd37b
Deleted: sha256:270f1b7a27d8ac27fdda11fbbc47fd07d780aebd36f2c0c58f2eaaeb9dcdaf21
Deleted: sha256:534d819b73beae359564314ff030a03872f7814adcfa1b666bbd5c464d36ce63
Deleted: sha256:43e1ce4c2c4f9008eed01c1e3d8faf1fbe0c1cd42707bd3dadf12d52223cec69
Deleted: sha256:84e9e5fdf1043c63df15e16972ece1b88373e1900e8870d0e793162cacc2c20e
Deleted: sha256:1320a62222ca1c52d2fe006875c5c959916c0d14e2d3711ce24377d4f594d7d9
Deleted: sha256:bb4ff5aafab5d16c3d35d2dec6448fc70fc57a90414e18eb20e2283f63a14f9f
Deleted: sha256:dcaaecd16612d6ae3513275f1fe66c3937d882a90eb2861e764579795fb5f088
Deleted: sha256:54db48fd4f9e018fd2452f91db22b0dc8b11dca49d1f033dba154e4540c4a588
Deleted: sha256:efc2a1c31c6dbf22bd044972882e7372591e448c83b5007b22860ee989a0f254
Deleted: sha256:fa04eef5b0fecb0e5e8352a89816bf8a7c9fffe94b093c5067d6ef8a7df82a2a
Deleted: sha256:c6e08c09dd20b92186d8adf36b1d8e89ca32d43af792ba84eb99e14ff691688d
Deleted: sha256:4e800fd313af7de18e10d15b5ac9736998b3e362390a29b20e5938f02b7a2de2
Deleted: sha256:5212a5655d31994c625c107e4ea797d9842cd1f7832bcd8935810a26fd5bfff5
Deleted: sha256:2ceac9e2666c29519511fc74012982552cf2bc68c2b8d576dfa9a3a3f322a1b5
Deleted: sha256:7aa5c32295ff420e2e09c04fa36cb5559d00c0a30623d6a7cc76ff3f71cfb844
Deleted: sha256:693ada4ee2d2fe82b3a254b4cce208fba427fa42352073f45e03d634b65cb31e
Deleted: sha256:386abc8e4b61e5359e1ac2b87e9e9b6ab6140b8ff07a321105c38c756fd89ab5
Deleted: sha256:dfeadcbdab5b0db39aaf5dac91372694695d72049a59dfb35b7a7075ed97fcb6
Deleted: sha256:204d18954b11ae2fa00ed668bf1fa93ce25d796212d643fc8b2cdf25a32da4b1
Deleted: sha256:9d3f3e63d204e036f091a9b9d1926686dc851d1cb7d9d49212b5d07625a5fb01
Deleted: sha256:5298e21d346968f4ff6c5b3bcc86a8ac6b5de28523ebbc4202f416831422f9ef
Deleted: sha256:986539b8c15b327088dfd1bac55510902b263aab506775fea2f967e6b5a49b5d
Deleted: sha256:0b14c60920782f91fdf6c736f1cf9742fd420229828f37069ed556feed123da9
Deleted: sha256:d40f7de54e87e43af8a054445849f8db6f918139dc142d217444366bf95240c5
Deleted: sha256:fb925b67157f4c8d8eb731a067e640352e219e47677ea1d94b79005e8ed4a1fb
Deleted: sha256:06cccf38bc7b4181acf5bf1c5d82b53b1ad0b0e9c526a82f9ae885a13c61f7bc
Deleted: sha256:1147b300d1a0b592dfc6a9ee575ab372062cb6d23d15ebd33991ebca443a9527
Deleted: sha256:9e86ced2a9955da60a7b13f12f09c46cbf5163c847d78e56812723992cb4eace
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210921124331]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ffc8a8a8aefa672ed766bb5c4b710d35a7fe8967874ee70d2fc271cc040cd37b]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210921124331] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ffc8a8a8aefa672ed766bb5c4b710d35a7fe8967874ee70d2fc271cc040cd37b])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ffc8a8a8aefa672ed766bb5c4b710d35a7fe8967874ee70d2fc271cc040cd37b
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ffc8a8a8aefa672ed766bb5c4b710d35a7fe8967874ee70d2fc271cc040cd37b
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ffc8a8a8aefa672ed766bb5c4b710d35a7fe8967874ee70d2fc271cc040cd37b].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 28s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/6bjhdfldeej3k

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #95

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/95/display/redirect>

Changes:


------------------------------------------
[...truncated 49.22 KB...]
94ab71098673: Preparing
953732d98cbd: Preparing
b1c428ea8ea4: Preparing
4708d6cf1204: Preparing
554f5e548896: Preparing
d0f03a3418d6: Preparing
d80ccc5dfad1: Preparing
5c807eafef2f: Preparing
08c7dbaefe9c: Preparing
4708d6cf1204: Waiting
9009985da905: Preparing
3891808a925b: Preparing
554f5e548896: Waiting
d402f4f1b906: Preparing
00ef5416d927: Preparing
d0f03a3418d6: Waiting
8555e663f65b: Preparing
d00da3cd7763: Preparing
4e61e63529c2: Preparing
799760671c38: Preparing
d80ccc5dfad1: Waiting
5c807eafef2f: Waiting
8555e663f65b: Waiting
08c7dbaefe9c: Waiting
d00da3cd7763: Waiting
4e61e63529c2: Waiting
9009985da905: Waiting
799760671c38: Waiting
3891808a925b: Waiting
00ef5416d927: Waiting
d402f4f1b906: Waiting
94ab71098673: Pushed
b1c428ea8ea4: Pushed
35a5fcdac39c: Pushed
3ec30b57f4d5: Pushed
953732d98cbd: Pushed
4708d6cf1204: Pushed
d0f03a3418d6: Pushed
d80ccc5dfad1: Pushed
3891808a925b: Layer already exists
d402f4f1b906: Layer already exists
00ef5416d927: Layer already exists
08c7dbaefe9c: Pushed
8555e663f65b: Layer already exists
9009985da905: Pushed
d00da3cd7763: Layer already exists
554f5e548896: Pushed
4e61e63529c2: Layer already exists
799760671c38: Layer already exists
5c807eafef2f: Pushed
20210920124338: digest: sha256:68163f05f1c3e4be7d2e732835240ea00b3091c5cd7faabd8f2ccf367364f635 size: 4311

> Task :sdks:java:testing:load-tests:run
Sep 20, 2021 12:47:41 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Sep 20, 2021 12:47:41 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 195 files. Enable logging at DEBUG level to see which files will be staged.
Sep 20, 2021 12:47:42 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Sep 20, 2021 12:47:42 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Sep 20, 2021 12:47:45 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 195 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Sep 20, 2021 12:47:45 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /home/jenkins/.m2/repository/org/apache/beam/beam-vendor-grpc-1_36_0/0.2/beam-vendor-grpc-1_36_0-0.2.jar to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-grpc-1_36_0-0.2-nVxp3_1Xbd1ktCTMOtQxL1xQUUS_NwN-RSOhZ5xBwdg.jar
Sep 20, 2021 12:47:46 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 194 files cached, 1 files newly uploaded in 1 seconds
Sep 20, 2021 12:47:46 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Sep 20, 2021 12:47:46 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <106569 bytes, hash 210c80cec4e0c92a301a821c838a61e79b3ddff3b72bf14212648f1f95df4ed7> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-IQyAzsTgySowGoIcg4ph55s93_O3K_FCEmSPH5XfTtc.pb
Sep 20, 2021 12:47:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Sep 20, 2021 12:47:48 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2eeb0f9b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1b1c538d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1645f294, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6325f352, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@15c4af7a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6cbd0674, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@55d58825, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@19a64eae, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@29a98d9f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2da3b078, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@544e8149, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7fb66650, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1a96d94c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2a869a16, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@ae202c6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@46aa712c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6ada9c0c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7412ed6b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6e7c351d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b4a0aef]
Sep 20, 2021 12:47:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Sep 20, 2021 12:47:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Sep 20, 2021 12:47:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Sep 20, 2021 12:47:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Sep 20, 2021 12:47:48 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2e5b7fba, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@27755487, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4f0cab0a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@fe7b6b0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7ab4ae59, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77681ce4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5d96bdf8, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6f76c2cc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@306f6f1d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7d7cac8, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6fc6deb7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@367f0121, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7da39774, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@441b8382, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1df1ced0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5349b246, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@32b0876c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2aaf152b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@39bbd9e0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@27fe9713]
Sep 20, 2021 12:47:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Sep 20, 2021 12:47:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Sep 20, 2021 12:47:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Sep 20, 2021 12:47:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Sep 20, 2021 12:47:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Sep 20, 2021 12:47:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Sep 20, 2021 12:47:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Sep 20, 2021 12:47:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Sep 20, 2021 12:47:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Sep 20, 2021 12:47:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Sep 20, 2021 12:47:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Sep 20, 2021 12:47:48 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.34.0-SNAPSHOT
Sep 20, 2021 12:47:50 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-09-20_05_47_48-1978144490141291391?project=apache-beam-testing
Sep 20, 2021 12:47:50 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-09-20_05_47_48-1978144490141291391
Sep 20, 2021 12:47:50 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-09-20_05_47_48-1978144490141291391
Sep 20, 2021 12:47:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-09-20T12:47:56.403Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-09-8c8o. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Sep 20, 2021 12:48:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-20T12:48:00.671Z: Worker configuration: e2-standard-2 in us-central1-a.
Sep 20, 2021 12:48:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-20T12:48:01.229Z: Expanding SplittableParDo operations into optimizable parts.
Sep 20, 2021 12:48:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-20T12:48:01.257Z: Expanding CollectionToSingleton operations into optimizable parts.
Sep 20, 2021 12:48:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-20T12:48:01.313Z: Expanding CoGroupByKey operations into optimizable parts.
Sep 20, 2021 12:48:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-20T12:48:01.384Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Sep 20, 2021 12:48:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-20T12:48:01.406Z: Expanding GroupByKey operations into streaming Read/Write steps
Sep 20, 2021 12:48:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-20T12:48:01.500Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Sep 20, 2021 12:48:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-20T12:48:01.612Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Sep 20, 2021 12:48:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-20T12:48:01.649Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Sep 20, 2021 12:48:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-20T12:48:01.680Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Sep 20, 2021 12:48:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-20T12:48:01.715Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Sep 20, 2021 12:48:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-20T12:48:01.736Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Sep 20, 2021 12:48:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-20T12:48:01.763Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Sep 20, 2021 12:48:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-20T12:48:01.786Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Sep 20, 2021 12:48:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-20T12:48:01.806Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Sep 20, 2021 12:48:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-20T12:48:01.839Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Sep 20, 2021 12:48:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-20T12:48:01.875Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Sep 20, 2021 12:48:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-20T12:48:01.908Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Sep 20, 2021 12:48:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-20T12:48:01.941Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Sep 20, 2021 12:48:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-20T12:48:01.964Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Sep 20, 2021 12:48:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-20T12:48:01.996Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Sep 20, 2021 12:48:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-20T12:48:02.029Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Sep 20, 2021 12:48:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-20T12:48:02.054Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Sep 20, 2021 12:48:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-20T12:48:02.089Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Sep 20, 2021 12:48:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-20T12:48:02.122Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Sep 20, 2021 12:48:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-20T12:48:02.145Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Sep 20, 2021 12:48:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-20T12:48:02.171Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Sep 20, 2021 12:48:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-20T12:48:02.196Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Sep 20, 2021 12:48:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-20T12:48:02.228Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Sep 20, 2021 12:48:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-20T12:48:02.266Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Sep 20, 2021 12:48:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-20T12:48:02.596Z: Starting 5 ****s in us-central1-a...
Sep 20, 2021 12:48:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-20T12:48:30.435Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Sep 20, 2021 12:48:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-20T12:48:51.527Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Sep 20, 2021 12:49:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-20T12:49:47.212Z: Workers have started successfully.
Sep 20, 2021 12:49:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-20T12:49:47.238Z: Workers have started successfully.
Sep 20, 2021 4:00:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-20T16:00:24.695Z: Cancel request is committed for workflow job: 2021-09-20_05_47_48-1978144490141291391.
Sep 20, 2021 4:00:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-20T16:00:24.750Z: Cleaning up.
Sep 20, 2021 4:00:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-20T16:00:24.812Z: Stopping **** pool...
Sep 20, 2021 4:00:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-20T16:00:24.855Z: Stopping **** pool...
Sep 20, 2021 4:02:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-20T16:02:49.553Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Sep 20, 2021 4:02:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-20T16:02:49.585Z: Worker pool stopped.
Sep 20, 2021 4:02:55 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-09-20_05_47_48-1978144490141291391 finished with status CANCELLED.
Load test results for test (ID): ec7d27d1-5b7e-4517-a466-729f58a15f14 and timestamp: 2021-09-20T12:47:42.248000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11363.641
dataflow_v2_java11_total_bytes_count             2.05473991E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210920124338
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:68163f05f1c3e4be7d2e732835240ea00b3091c5cd7faabd8f2ccf367364f635
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210920124338]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:68163f05f1c3e4be7d2e732835240ea00b3091c5cd7faabd8f2ccf367364f635]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210920124338] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:68163f05f1c3e4be7d2e732835240ea00b3091c5cd7faabd8f2ccf367364f635])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:68163f05f1c3e4be7d2e732835240ea00b3091c5cd7faabd8f2ccf367364f635
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:68163f05f1c3e4be7d2e732835240ea00b3091c5cd7faabd8f2ccf367364f635
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:68163f05f1c3e4be7d2e732835240ea00b3091c5cd7faabd8f2ccf367364f635].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 37s
101 actionable tasks: 71 executed, 28 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/sg7hblbv7vz22

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #94

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/94/display/redirect>

Changes:


------------------------------------------
[...truncated 49.02 KB...]
The push refers to repository [us.gcr.io/apache-beam-testing/java-postcommit-it/java]
b69cf6716aaf: Preparing
cff32e42fb6f: Preparing
7f9400f001db: Preparing
80da27462470: Preparing
e8b33b763bd6: Preparing
9c0973a5e643: Preparing
5f7e05b91381: Preparing
a0fbc6d46abd: Preparing
eddc235e5e23: Preparing
646a73e325fe: Preparing
51ed3827f313: Preparing
dfd16bb7186a: Preparing
3891808a925b: Preparing
d402f4f1b906: Preparing
00ef5416d927: Preparing
8555e663f65b: Preparing
d00da3cd7763: Preparing
4e61e63529c2: Preparing
799760671c38: Preparing
eddc235e5e23: Waiting
9c0973a5e643: Waiting
646a73e325fe: Waiting
5f7e05b91381: Waiting
51ed3827f313: Waiting
dfd16bb7186a: Waiting
a0fbc6d46abd: Waiting
00ef5416d927: Waiting
d00da3cd7763: Waiting
4e61e63529c2: Waiting
3891808a925b: Waiting
d402f4f1b906: Waiting
8555e663f65b: Waiting
e8b33b763bd6: Pushed
cff32e42fb6f: Pushed
7f9400f001db: Pushed
b69cf6716aaf: Pushed
80da27462470: Pushed
9c0973a5e643: Pushed
a0fbc6d46abd: Pushed
eddc235e5e23: Pushed
3891808a925b: Layer already exists
d402f4f1b906: Layer already exists
00ef5416d927: Layer already exists
8555e663f65b: Layer already exists
5f7e05b91381: Pushed
d00da3cd7763: Layer already exists
4e61e63529c2: Layer already exists
799760671c38: Layer already exists
51ed3827f313: Pushed
dfd16bb7186a: Pushed
646a73e325fe: Pushed
20210919124332: digest: sha256:ed3fc659aa9c33a09f43434d8433a1041be188d58ffcbd0494f2307d5939469b size: 4311

> Task :sdks:java:testing:load-tests:run
Sep 19, 2021 12:45:43 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Sep 19, 2021 12:45:43 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 195 files. Enable logging at DEBUG level to see which files will be staged.
Sep 19, 2021 12:45:44 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Sep 19, 2021 12:45:44 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Sep 19, 2021 12:45:46 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 195 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Sep 19, 2021 12:45:47 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 195 files cached, 0 files newly uploaded in 0 seconds
Sep 19, 2021 12:45:47 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Sep 19, 2021 12:45:47 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <106569 bytes, hash b91d021c56b3a79b73f5e9124f299638b187d279b5ec001ed924414b96b00b41> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-uR0CHFazp5tz9ekSTymWOLGH0nm17AAe2SRBS5awC0E.pb
Sep 19, 2021 12:45:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Sep 19, 2021 12:45:48 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2eeb0f9b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1b1c538d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1645f294, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6325f352, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@15c4af7a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6cbd0674, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@55d58825, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@19a64eae, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@29a98d9f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2da3b078, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@544e8149, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7fb66650, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1a96d94c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2a869a16, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@ae202c6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@46aa712c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6ada9c0c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7412ed6b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6e7c351d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b4a0aef]
Sep 19, 2021 12:45:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Sep 19, 2021 12:45:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Sep 19, 2021 12:45:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Sep 19, 2021 12:45:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Sep 19, 2021 12:45:48 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2e5b7fba, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@27755487, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4f0cab0a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@fe7b6b0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7ab4ae59, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77681ce4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5d96bdf8, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6f76c2cc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@306f6f1d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7d7cac8, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6fc6deb7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@367f0121, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7da39774, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@441b8382, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1df1ced0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5349b246, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@32b0876c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2aaf152b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@39bbd9e0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@27fe9713]
Sep 19, 2021 12:45:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Sep 19, 2021 12:45:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Sep 19, 2021 12:45:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Sep 19, 2021 12:45:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Sep 19, 2021 12:45:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Sep 19, 2021 12:45:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Sep 19, 2021 12:45:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Sep 19, 2021 12:45:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Sep 19, 2021 12:45:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Sep 19, 2021 12:45:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Sep 19, 2021 12:45:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Sep 19, 2021 12:45:48 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.34.0-SNAPSHOT
Sep 19, 2021 12:45:52 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-09-19_05_45_48-12301866298194863507?project=apache-beam-testing
Sep 19, 2021 12:45:52 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-09-19_05_45_48-12301866298194863507
Sep 19, 2021 12:45:52 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-09-19_05_45_48-12301866298194863507
Sep 19, 2021 12:46:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-09-19T12:45:58.478Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-09-j2j. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Sep 19, 2021 12:46:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-19T12:46:02.396Z: Worker configuration: e2-standard-2 in us-central1-a.
Sep 19, 2021 12:46:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-19T12:46:03.259Z: Expanding SplittableParDo operations into optimizable parts.
Sep 19, 2021 12:46:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-19T12:46:03.296Z: Expanding CollectionToSingleton operations into optimizable parts.
Sep 19, 2021 12:46:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-19T12:46:03.359Z: Expanding CoGroupByKey operations into optimizable parts.
Sep 19, 2021 12:46:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-19T12:46:03.417Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Sep 19, 2021 12:46:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-19T12:46:03.447Z: Expanding GroupByKey operations into streaming Read/Write steps
Sep 19, 2021 12:46:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-19T12:46:03.491Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Sep 19, 2021 12:46:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-19T12:46:03.585Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Sep 19, 2021 12:46:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-19T12:46:03.621Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Sep 19, 2021 12:46:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-19T12:46:03.649Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Sep 19, 2021 12:46:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-19T12:46:03.677Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Sep 19, 2021 12:46:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-19T12:46:03.709Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Sep 19, 2021 12:46:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-19T12:46:03.744Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Sep 19, 2021 12:46:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-19T12:46:03.769Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Sep 19, 2021 12:46:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-19T12:46:03.799Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Sep 19, 2021 12:46:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-19T12:46:03.836Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Sep 19, 2021 12:46:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-19T12:46:03.867Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Sep 19, 2021 12:46:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-19T12:46:03.893Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Sep 19, 2021 12:46:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-19T12:46:03.941Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Sep 19, 2021 12:46:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-19T12:46:03.997Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Sep 19, 2021 12:46:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-19T12:46:04.037Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Sep 19, 2021 12:46:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-19T12:46:04.077Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Sep 19, 2021 12:46:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-19T12:46:04.102Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Sep 19, 2021 12:46:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-19T12:46:04.129Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Sep 19, 2021 12:46:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-19T12:46:04.163Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Sep 19, 2021 12:46:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-19T12:46:04.190Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Sep 19, 2021 12:46:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-19T12:46:04.219Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Sep 19, 2021 12:46:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-19T12:46:04.253Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Sep 19, 2021 12:46:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-19T12:46:04.290Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Sep 19, 2021 12:46:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-19T12:46:04.317Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Sep 19, 2021 12:46:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-19T12:46:04.666Z: Starting 5 ****s in us-central1-a...
Sep 19, 2021 12:46:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-19T12:46:22.205Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Sep 19, 2021 12:46:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-19T12:46:45.023Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Sep 19, 2021 12:47:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-19T12:47:40.608Z: Workers have started successfully.
Sep 19, 2021 12:47:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-19T12:47:40.633Z: Workers have started successfully.
Sep 19, 2021 4:00:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-19T16:00:28.211Z: Cancel request is committed for workflow job: 2021-09-19_05_45_48-12301866298194863507.
Sep 19, 2021 4:00:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-19T16:00:28.278Z: Cleaning up.
Sep 19, 2021 4:00:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-19T16:00:28.338Z: Stopping **** pool...
Sep 19, 2021 4:00:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-19T16:00:28.381Z: Stopping **** pool...
Sep 19, 2021 4:02:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-19T16:02:45.353Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Sep 19, 2021 4:02:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-19T16:02:45.381Z: Worker pool stopped.
Sep 19, 2021 4:02:51 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-09-19_05_45_48-12301866298194863507 finished with status CANCELLED.
Load test results for test (ID): 9f9ab5d7-67bc-4a9f-9a89-69febc7742f1 and timestamp: 2021-09-19T12:45:44.285000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11502.892
dataflow_v2_java11_total_bytes_count              2.4309485E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210919124332
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ed3fc659aa9c33a09f43434d8433a1041be188d58ffcbd0494f2307d5939469b
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210919124332]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ed3fc659aa9c33a09f43434d8433a1041be188d58ffcbd0494f2307d5939469b]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210919124332] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ed3fc659aa9c33a09f43434d8433a1041be188d58ffcbd0494f2307d5939469b])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ed3fc659aa9c33a09f43434d8433a1041be188d58ffcbd0494f2307d5939469b
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ed3fc659aa9c33a09f43434d8433a1041be188d58ffcbd0494f2307d5939469b
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ed3fc659aa9c33a09f43434d8433a1041be188d58ffcbd0494f2307d5939469b].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 37s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/nuoby3h7nt7lo

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #93

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/93/display/redirect?page=changes>

Changes:

[samuelw] [BEAM-12740] Add option to CreateOptions to avoid GetObjectMetadata for

[noreply] Minor: Prune docker volumes in Inventory job(#15532)


------------------------------------------
[...truncated 49.52 KB...]
4e61e63529c2: Layer already exists
799760671c38: Layer already exists
4d9c3cdf1c5b: Pushed
0be840f624d2: Pushed
c4b09fbab6de: Pushed
f1158c71e8f6: Pushed
20210918124341: digest: sha256:2d774c4ea296c365b422dde2269b3f6777600ca504a4a452b8f59df29e3c8d32 size: 4311

> Task :sdks:java:testing:load-tests:run
Sep 18, 2021 12:45:57 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Sep 18, 2021 12:45:57 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 195 files. Enable logging at DEBUG level to see which files will be staged.
Sep 18, 2021 12:45:58 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Sep 18, 2021 12:45:58 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Sep 18, 2021 12:46:00 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 195 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Sep 18, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 195 files cached, 0 files newly uploaded in 1 seconds
Sep 18, 2021 12:46:03 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Sep 18, 2021 12:46:03 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <106569 bytes, hash 979468afaa23619dcd48e71c5232d83c781bf76100eb45c5da4c75d3f2fc2683> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-l5Ror6ojYZ3NSOccUjLYPHgb92EA60XF2kx10_L8JoM.pb
Sep 18, 2021 12:46:04 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Sep 18, 2021 12:46:04 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1b1c538d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1645f294, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6325f352, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@15c4af7a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6cbd0674, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@55d58825, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@19a64eae, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@29a98d9f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2da3b078, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@544e8149, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7fb66650, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1a96d94c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2a869a16, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@ae202c6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@46aa712c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6ada9c0c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7412ed6b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6e7c351d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b4a0aef, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@45cec376]
Sep 18, 2021 12:46:04 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Sep 18, 2021 12:46:04 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Sep 18, 2021 12:46:04 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Sep 18, 2021 12:46:04 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Sep 18, 2021 12:46:04 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@27755487, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4f0cab0a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@fe7b6b0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7ab4ae59, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77681ce4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5d96bdf8, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6f76c2cc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@306f6f1d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7d7cac8, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6fc6deb7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@367f0121, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7da39774, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@441b8382, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1df1ced0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5349b246, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@32b0876c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2aaf152b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@39bbd9e0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@27fe9713, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@11c3ff67]
Sep 18, 2021 12:46:04 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Sep 18, 2021 12:46:04 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Sep 18, 2021 12:46:04 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Sep 18, 2021 12:46:04 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Sep 18, 2021 12:46:04 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Sep 18, 2021 12:46:04 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Sep 18, 2021 12:46:04 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Sep 18, 2021 12:46:05 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Sep 18, 2021 12:46:05 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Sep 18, 2021 12:46:05 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Sep 18, 2021 12:46:05 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Sep 18, 2021 12:46:05 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.34.0-SNAPSHOT
Sep 18, 2021 12:46:06 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-09-18_05_46_05-13954160276063115280?project=apache-beam-testing
Sep 18, 2021 12:46:06 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-09-18_05_46_05-13954160276063115280
Sep 18, 2021 12:46:06 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-09-18_05_46_05-13954160276063115280
Sep 18, 2021 12:46:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-09-18T12:46:12.612Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-09-euxk. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Sep 18, 2021 12:46:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-18T12:46:16.960Z: Worker configuration: e2-standard-2 in us-central1-a.
Sep 18, 2021 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-18T12:46:17.905Z: Expanding SplittableParDo operations into optimizable parts.
Sep 18, 2021 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-18T12:46:17.935Z: Expanding CollectionToSingleton operations into optimizable parts.
Sep 18, 2021 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-18T12:46:17.982Z: Expanding CoGroupByKey operations into optimizable parts.
Sep 18, 2021 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-18T12:46:18.042Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Sep 18, 2021 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-18T12:46:18.073Z: Expanding GroupByKey operations into streaming Read/Write steps
Sep 18, 2021 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-18T12:46:18.118Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Sep 18, 2021 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-18T12:46:18.192Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Sep 18, 2021 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-18T12:46:18.216Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Sep 18, 2021 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-18T12:46:18.249Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Sep 18, 2021 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-18T12:46:18.281Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Sep 18, 2021 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-18T12:46:18.316Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Sep 18, 2021 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-18T12:46:18.346Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Sep 18, 2021 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-18T12:46:18.380Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Sep 18, 2021 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-18T12:46:18.403Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Sep 18, 2021 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-18T12:46:18.436Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Sep 18, 2021 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-18T12:46:18.462Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Sep 18, 2021 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-18T12:46:18.495Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Sep 18, 2021 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-18T12:46:18.517Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Sep 18, 2021 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-18T12:46:18.555Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Sep 18, 2021 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-18T12:46:18.585Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Sep 18, 2021 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-18T12:46:18.612Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Sep 18, 2021 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-18T12:46:18.633Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Sep 18, 2021 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-18T12:46:18.658Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Sep 18, 2021 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-18T12:46:18.688Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Sep 18, 2021 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-18T12:46:18.722Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Sep 18, 2021 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-18T12:46:18.753Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Sep 18, 2021 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-18T12:46:18.786Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Sep 18, 2021 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-18T12:46:18.810Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Sep 18, 2021 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-18T12:46:18.847Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Sep 18, 2021 12:46:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-18T12:46:19.167Z: Starting 5 ****s in us-central1-a...
Sep 18, 2021 12:46:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-18T12:46:36.282Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Sep 18, 2021 12:47:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-18T12:47:00.929Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Sep 18, 2021 12:47:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-18T12:47:53.708Z: Workers have started successfully.
Sep 18, 2021 12:47:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-18T12:47:53.743Z: Workers have started successfully.
Sep 18, 2021 3:16:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-18T15:16:20.117Z: Staged package beam-vendor-grpc-1_36_0-0.2-2X-EhRNYA0GgyvffOgWohko1OyyyPyTMuiHHvaPG9es.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-grpc-1_36_0-0.2-2X-EhRNYA0GgyvffOgWohko1OyyyPyTMuiHHvaPG9es.jar' is inaccessible.
Sep 18, 2021 3:16:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-09-18T15:16:23.362Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Sep 18, 2021 3:19:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-09-18T15:19:23.198Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Sep 18, 2021 3:22:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-18T15:22:20.165Z: Staged package beam-vendor-grpc-1_36_0-0.2-2X-EhRNYA0GgyvffOgWohko1OyyyPyTMuiHHvaPG9es.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-grpc-1_36_0-0.2-2X-EhRNYA0GgyvffOgWohko1OyyyPyTMuiHHvaPG9es.jar' is inaccessible.
Sep 18, 2021 3:22:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-09-18T15:22:23.281Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Sep 18, 2021 3:25:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-09-18T15:25:23.230Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Sep 18, 2021 3:28:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-18T15:28:20.087Z: Staged package beam-vendor-grpc-1_36_0-0.2-2X-EhRNYA0GgyvffOgWohko1OyyyPyTMuiHHvaPG9es.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-grpc-1_36_0-0.2-2X-EhRNYA0GgyvffOgWohko1OyyyPyTMuiHHvaPG9es.jar' is inaccessible.
Sep 18, 2021 3:28:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-09-18T15:28:23.167Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Sep 18, 2021 3:31:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-09-18T15:31:23.180Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Sep 18, 2021 3:34:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-18T15:34:20.066Z: Staged package beam-vendor-grpc-1_36_0-0.2-2X-EhRNYA0GgyvffOgWohko1OyyyPyTMuiHHvaPG9es.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-grpc-1_36_0-0.2-2X-EhRNYA0GgyvffOgWohko1OyyyPyTMuiHHvaPG9es.jar' is inaccessible.
Sep 18, 2021 3:34:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-09-18T15:34:23.272Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Sep 18, 2021 3:37:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-09-18T15:37:23.332Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Sep 18, 2021 3:40:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-18T15:40:20.054Z: Staged package beam-vendor-grpc-1_36_0-0.2-2X-EhRNYA0GgyvffOgWohko1OyyyPyTMuiHHvaPG9es.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-grpc-1_36_0-0.2-2X-EhRNYA0GgyvffOgWohko1OyyyPyTMuiHHvaPG9es.jar' is inaccessible.
Sep 18, 2021 3:40:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-09-18T15:40:23.023Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Sep 18, 2021 3:43:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-09-18T15:43:23.495Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Sep 18, 2021 3:46:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-18T15:46:20.398Z: Staged package beam-vendor-grpc-1_36_0-0.2-2X-EhRNYA0GgyvffOgWohko1OyyyPyTMuiHHvaPG9es.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-grpc-1_36_0-0.2-2X-EhRNYA0GgyvffOgWohko1OyyyPyTMuiHHvaPG9es.jar' is inaccessible.
Sep 18, 2021 3:46:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-09-18T15:46:24.334Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Sep 18, 2021 3:49:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-09-18T15:49:23.660Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Sep 18, 2021 3:52:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-18T15:52:20.073Z: Staged package beam-vendor-grpc-1_36_0-0.2-2X-EhRNYA0GgyvffOgWohko1OyyyPyTMuiHHvaPG9es.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-grpc-1_36_0-0.2-2X-EhRNYA0GgyvffOgWohko1OyyyPyTMuiHHvaPG9es.jar' is inaccessible.
Sep 18, 2021 3:52:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-09-18T15:52:23.123Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Sep 18, 2021 3:55:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-09-18T15:55:23.733Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Sep 18, 2021 3:58:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-18T15:58:20.125Z: Staged package beam-vendor-grpc-1_36_0-0.2-2X-EhRNYA0GgyvffOgWohko1OyyyPyTMuiHHvaPG9es.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-grpc-1_36_0-0.2-2X-EhRNYA0GgyvffOgWohko1OyyyPyTMuiHHvaPG9es.jar' is inaccessible.
Sep 18, 2021 3:58:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-09-18T15:58:23.376Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Sep 18, 2021 4:00:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-18T16:00:26.411Z: Cancel request is committed for workflow job: 2021-09-18_05_46_05-13954160276063115280.
Sep 18, 2021 4:00:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-18T16:00:26.435Z: Cleaning up.
Sep 18, 2021 4:00:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-18T16:00:26.510Z: Stopping **** pool...
Sep 18, 2021 4:00:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-18T16:00:26.618Z: Stopping **** pool...
Sep 18, 2021 4:02:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-18T16:02:45.236Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Sep 18, 2021 4:02:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-18T16:02:45.270Z: Worker pool stopped.
Sep 18, 2021 4:02:51 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-09-18_05_46_05-13954160276063115280 finished with status CANCELLED.
Load test results for test (ID): 0d676911-3e9d-42a8-b740-bd69a8b864f4 and timestamp: 2021-09-18T12:45:58.308000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11500.684
dataflow_v2_java11_total_bytes_count             2.84601622E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210918124341
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:2d774c4ea296c365b422dde2269b3f6777600ca504a4a452b8f59df29e3c8d32
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210918124341]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:2d774c4ea296c365b422dde2269b3f6777600ca504a4a452b8f59df29e3c8d32]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210918124341] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:2d774c4ea296c365b422dde2269b3f6777600ca504a4a452b8f59df29e3c8d32])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:2d774c4ea296c365b422dde2269b3f6777600ca504a4a452b8f59df29e3c8d32
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:2d774c4ea296c365b422dde2269b3f6777600ca504a4a452b8f59df29e3c8d32
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:2d774c4ea296c365b422dde2269b3f6777600ca504a4a452b8f59df29e3c8d32].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 32s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/wsdaaft7poty2

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #92

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/92/display/redirect?page=changes>

Changes:

[randomstep] [BEAM-12899] Upgrade Gradle to version 6.9.x

[noreply] [BEAM-12701] Added extra parameter in to_csv for DeferredFrame to name

[zyichi] [BEAM-12603] Add retries to FnApiRunnerTest due to flakiness of grpc

[noreply] [BEAM-12535] add dataframes notebook (#15470)


------------------------------------------
[...truncated 49.92 KB...]
ebd13132d2dd: Pushed
9998d23a62f8: Pushed
49750b7dcccd: Pushed
257de4377a3c: Pushed
3891808a925b: Layer already exists
d402f4f1b906: Layer already exists
00ef5416d927: Layer already exists
8555e663f65b: Layer already exists
d00da3cd7763: Layer already exists
4e61e63529c2: Layer already exists
8af1e1eec513: Pushed
cdf5761c3cd6: Pushed
799760671c38: Layer already exists
e625d5644106: Pushed
9a3f32baf9a8: Pushed
20210917124334: digest: sha256:040fcf1a34c1227a112a378852e86b7db64735e7d422f7bbf3e461a5345dd510 size: 4311

> Task :sdks:java:testing:load-tests:run
Sep 17, 2021 12:45:22 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Sep 17, 2021 12:45:22 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 195 files. Enable logging at DEBUG level to see which files will be staged.
Sep 17, 2021 12:45:23 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Sep 17, 2021 12:45:23 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Sep 17, 2021 12:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 195 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Sep 17, 2021 12:45:26 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 195 files cached, 0 files newly uploaded in 1 seconds
Sep 17, 2021 12:45:26 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Sep 17, 2021 12:45:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <106569 bytes, hash 68c7ddc9492113411748bb84e881c1fcc72a3845b1a74d3ca0b9986609d6b58a> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-aMfdyUkhE0EXSLuE6IHB_McqOEWxp008oLmYZgnWtYo.pb
Sep 17, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Sep 17, 2021 12:45:28 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2d64160c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5f254608, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2eeb0f9b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1b1c538d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1645f294, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6325f352, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@15c4af7a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6cbd0674, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@55d58825, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@19a64eae, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@29a98d9f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2da3b078, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@544e8149, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7fb66650, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1a96d94c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2a869a16, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@ae202c6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@46aa712c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6ada9c0c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7412ed6b]
Sep 17, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Sep 17, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Sep 17, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Sep 17, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Sep 17, 2021 12:45:28 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@41853299, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@60d40ff4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2e5b7fba, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@27755487, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4f0cab0a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@fe7b6b0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7ab4ae59, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77681ce4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5d96bdf8, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6f76c2cc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@306f6f1d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7d7cac8, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6fc6deb7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@367f0121, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7da39774, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@441b8382, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1df1ced0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5349b246, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@32b0876c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2aaf152b]
Sep 17, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Sep 17, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Sep 17, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Sep 17, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Sep 17, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Sep 17, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Sep 17, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Sep 17, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Sep 17, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Sep 17, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Sep 17, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Sep 17, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.34.0-SNAPSHOT
Sep 17, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-09-17_05_45_28-17019632363355082856?project=apache-beam-testing
Sep 17, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-09-17_05_45_28-17019632363355082856
Sep 17, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-09-17_05_45_28-17019632363355082856
Sep 17, 2021 12:45:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-09-17T12:45:36.321Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-09-4adb. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Sep 17, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-17T12:45:41.847Z: Worker configuration: e2-standard-2 in us-central1-a.
Sep 17, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-17T12:45:42.587Z: Expanding SplittableParDo operations into optimizable parts.
Sep 17, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-17T12:45:42.623Z: Expanding CollectionToSingleton operations into optimizable parts.
Sep 17, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-17T12:45:42.696Z: Expanding CoGroupByKey operations into optimizable parts.
Sep 17, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-17T12:45:42.767Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Sep 17, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-17T12:45:42.800Z: Expanding GroupByKey operations into streaming Read/Write steps
Sep 17, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-17T12:45:42.867Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Sep 17, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-17T12:45:42.983Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Sep 17, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-17T12:45:43.023Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Sep 17, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-17T12:45:43.055Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Sep 17, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-17T12:45:43.083Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Sep 17, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-17T12:45:43.107Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Sep 17, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-17T12:45:43.138Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Sep 17, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-17T12:45:43.194Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Sep 17, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-17T12:45:43.227Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Sep 17, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-17T12:45:43.261Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Sep 17, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-17T12:45:43.294Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Sep 17, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-17T12:45:43.327Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Sep 17, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-17T12:45:43.382Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Sep 17, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-17T12:45:43.419Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Sep 17, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-17T12:45:43.453Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Sep 17, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-17T12:45:43.487Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Sep 17, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-17T12:45:43.520Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Sep 17, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-17T12:45:43.559Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Sep 17, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-17T12:45:43.592Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Sep 17, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-17T12:45:43.626Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Sep 17, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-17T12:45:43.651Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Sep 17, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-17T12:45:43.682Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Sep 17, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-17T12:45:43.719Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Sep 17, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-17T12:45:43.749Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Sep 17, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-17T12:45:44.065Z: Starting 5 ****s in us-central1-a...
Sep 17, 2021 12:45:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-17T12:45:56.770Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Sep 17, 2021 12:46:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-17T12:46:23.992Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Sep 17, 2021 12:47:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-17T12:47:23.700Z: Workers have started successfully.
Sep 17, 2021 12:47:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-17T12:47:23.730Z: Workers have started successfully.
Sep 17, 2021 4:00:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-17T16:00:27.591Z: Cancel request is committed for workflow job: 2021-09-17_05_45_28-17019632363355082856.
Sep 17, 2021 4:00:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-17T16:00:27.669Z: Cleaning up.
Sep 17, 2021 4:00:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-17T16:00:27.733Z: Stopping **** pool...
Sep 17, 2021 4:00:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-17T16:00:27.783Z: Stopping **** pool...
Sep 17, 2021 4:00:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-17T16:00:36.078Z: generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:1089
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:1089
Sep 17, 2021 4:02:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-17T16:02:47.614Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Sep 17, 2021 4:02:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-17T16:02:47.650Z: Worker pool stopped.
Sep 17, 2021 4:02:53 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-09-17_05_45_28-17019632363355082856 finished with status CANCELLED.
Load test results for test (ID): b0ec15fe-b275-441d-9c56-55e080f6e2c1 and timestamp: 2021-09-17T12:45:22.952000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11548.337
dataflow_v2_java11_total_bytes_count             2.04755766E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210917124334
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:040fcf1a34c1227a112a378852e86b7db64735e7d422f7bbf3e461a5345dd510
Deleted: sha256:51220a157e98970e581fc82c17ae080a7d75c198d4bd5dec5c6ff4c8c0110d2b
Deleted: sha256:1d1b931ea7dd3188599d30a0b00df141ada7a7a4a5ccf60c4e7aa7e60618b2c1
Deleted: sha256:5dbac7b2b241cce8f57417a7d9e6f39a8a6c8fa4d2d1d826f4f672a1ddd49f8f
Deleted: sha256:42a17ddefe018c5adf4da9391b84c1dd878323bb26c6cd1ce078c800506e988a
Deleted: sha256:b7db880cee92f9f53fe649bc6435667f285c74ac9f09897f045207a531478fd9
Deleted: sha256:fb08917ee3996cc5113be9f90a75c20b2eb236bbd16c3c3392f0384eba179726
Deleted: sha256:ded0f8bcd7e15dcb58d3b9f1e8f7d42efbfc06a5cfc6af904d9ec75981f1ecf2
Deleted: sha256:7628ace029b201d217f843a245a0a469e8009f9478092381a5b31efd99ea02be
Deleted: sha256:db83a98067892f629c0dc154a4a11a4d07b95f5546030efad558b78f4a8c34f7
Deleted: sha256:e6cbf762df70e1132a2ae92700d379b555096e58c03da72b5dcc4403e34dcfee
Deleted: sha256:adea99ad81c3fec89cbda76e18d733d27ed8475a65c292d5149d5255d4df37a3
Deleted: sha256:c54d27b5b88135d2c48b53071e2fffab92504a37258eba4f7070888d1a6e4a12
Deleted: sha256:aabf746fb1f34f31d269d68d4397b4fa1499c3a58cfba5815caf6b77852beac8
Deleted: sha256:548b9b1151932c97ed1c2e7b3ecf31165a2f8f6f9a97e44c898e97d1b30bdda3
Deleted: sha256:0bc79cf7c6bd3e9413c107bd1a89eaf0fa06f837792e422e0104ace9758f4e13
Deleted: sha256:66ec577042f379f6cab0f3aba2c14996fb4d892202db514b46f49f669b37828f
Deleted: sha256:9e555a5ad620739800a5be935c9bac8a0fdbbbb9f3c85a81751e64a4478c9432
Deleted: sha256:7a249063b8af977ec773a8ac90ff3fc936007d5a062f301532249dc8929b4625
Deleted: sha256:d2a7bcf00ba40eba94e52a90902d7f5dea6413542bd5883f09fb0503ae227849
Deleted: sha256:35401cb89023ae465a3ed458da089982ec2ada8929aaa9eab7b5c0774406fe97
Deleted: sha256:f5e4f0993712ead6150fe6808de2216017732a6313eae29b966dfdec41128c26
Deleted: sha256:40d95b9e1a99f529776a937184a32d7ccaec26e3c236f9072694e9492d966b46
Deleted: sha256:31db754218e7983d9d30dbf3af0b78c3866904296ace832c41e9e6b238391c34
Deleted: sha256:5e98107d8799b66fdc959243e6051aaedd58bf8fd3a2fa32f528d89608c28912
Deleted: sha256:596eff7aac946664bc43bbcacf63fa5c0759211999a5ec34c9d76a29b340d19f
Deleted: sha256:4485b5c195cb622147812aa99f66a6c759e5358c9e458836d012ac296aa44c96
Deleted: sha256:30fcfe67a4e49646910b554d343c76bd336257c8ae792f41978ea002f32e9ec9
Deleted: sha256:2a61b929e2767dac14ba8a95aa1a8c68916cdd4694f2166f73e207e7b6ecd516
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210917124334]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:040fcf1a34c1227a112a378852e86b7db64735e7d422f7bbf3e461a5345dd510]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210917124334] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:040fcf1a34c1227a112a378852e86b7db64735e7d422f7bbf3e461a5345dd510])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:040fcf1a34c1227a112a378852e86b7db64735e7d422f7bbf3e461a5345dd510
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:040fcf1a34c1227a112a378852e86b7db64735e7d422f7bbf3e461a5345dd510
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:040fcf1a34c1227a112a378852e86b7db64735e7d422f7bbf3e461a5345dd510].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 38s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/sqswlj37yu3za

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #91

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/91/display/redirect?page=changes>

Changes:

[dpcollins] [BEAM-12882] - fix test that is flaky when jenkins is overloaded

[noreply] [BEAM-12885] Enable NeedsRunner Tests for Samza Portable Runner (#15512)

[noreply] [BEAM-12100][BEAM-10379][BEAM-9514][BEAM-12647][BEAM-12099]

[noreply] [BEAM-12543] Fix DataFrrame typo (#15509)

[noreply] [BEAM-12794] Remove obsolete uses of sys.exc_info. (#15507)

[noreply] [BEAM-11666]  flake on RecordingManagerTest (#15118)

[kawaigin] [BEAM-10708] Introspect beam_sql output

[noreply] Minor: Restore "Bugfix" section in CHANGES.md (#15516)

[Kyle Weaver] [BEAM-10459] Unignore numeric aggregation tests.

[ajamato] [BEAM-12898] Disable Flink Load tests which are leading Dataproc


------------------------------------------
[...truncated 48.96 KB...]
The push refers to repository [us.gcr.io/apache-beam-testing/java-postcommit-it/java]
619f3fbf92b9: Preparing
af3a23159c7d: Preparing
1270d47666fd: Preparing
6d0fed1f7ad0: Preparing
c508d04f70e6: Preparing
2014e59c76b4: Preparing
fea88fcf18b1: Preparing
57c8a67db75b: Preparing
7fb2e83f9629: Preparing
96ac1d08cb55: Preparing
a42d90664913: Preparing
60b6f299bc46: Preparing
3891808a925b: Preparing
d402f4f1b906: Preparing
00ef5416d927: Preparing
8555e663f65b: Preparing
d00da3cd7763: Preparing
4e61e63529c2: Preparing
7fb2e83f9629: Waiting
799760671c38: Preparing
3891808a925b: Waiting
d00da3cd7763: Waiting
d402f4f1b906: Waiting
4e61e63529c2: Waiting
799760671c38: Waiting
00ef5416d927: Waiting
8555e663f65b: Waiting
2014e59c76b4: Waiting
fea88fcf18b1: Waiting
57c8a67db75b: Waiting
a42d90664913: Waiting
60b6f299bc46: Waiting
c508d04f70e6: Pushed
1270d47666fd: Pushed
af3a23159c7d: Pushed
2014e59c76b4: Pushed
57c8a67db75b: Pushed
619f3fbf92b9: Pushed
6d0fed1f7ad0: Pushed
7fb2e83f9629: Pushed
3891808a925b: Layer already exists
d402f4f1b906: Layer already exists
00ef5416d927: Layer already exists
8555e663f65b: Layer already exists
d00da3cd7763: Layer already exists
4e61e63529c2: Layer already exists
a42d90664913: Pushed
799760671c38: Layer already exists
60b6f299bc46: Pushed
fea88fcf18b1: Pushed
96ac1d08cb55: Pushed
20210916124428: digest: sha256:30ac48ef18e0b01215057c18fd66d15bcc4d927ca67a8575beda181b516f2adb size: 4311

> Task :sdks:java:testing:load-tests:run
Sep 16, 2021 12:48:27 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Sep 16, 2021 12:48:28 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 195 files. Enable logging at DEBUG level to see which files will be staged.
Sep 16, 2021 12:48:30 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Sep 16, 2021 12:48:30 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Sep 16, 2021 12:48:34 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 195 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Sep 16, 2021 12:48:39 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 195 files cached, 0 files newly uploaded in 4 seconds
Sep 16, 2021 12:48:39 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Sep 16, 2021 12:48:40 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <106569 bytes, hash c34dcce505512389ee60a4fed7f649627026e48cb41bc54621bd457366a034d5> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-w03M5QVRI4nuYKT-1_ZJYnAm5Iy0G8VGIb1Fc2agNNU.pb
Sep 16, 2021 12:48:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Sep 16, 2021 12:48:43 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1645f294, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6325f352, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@15c4af7a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6cbd0674, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@55d58825, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@19a64eae, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@29a98d9f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2da3b078, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@544e8149, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7fb66650, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1a96d94c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2a869a16, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@ae202c6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@46aa712c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6ada9c0c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7412ed6b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6e7c351d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b4a0aef, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@45cec376, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26d41711]
Sep 16, 2021 12:48:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Sep 16, 2021 12:48:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Sep 16, 2021 12:48:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Sep 16, 2021 12:48:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Sep 16, 2021 12:48:43 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4f0cab0a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@fe7b6b0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7ab4ae59, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77681ce4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5d96bdf8, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6f76c2cc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@306f6f1d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7d7cac8, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6fc6deb7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@367f0121, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7da39774, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@441b8382, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1df1ced0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5349b246, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@32b0876c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2aaf152b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@39bbd9e0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@27fe9713, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@11c3ff67, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4397a639]
Sep 16, 2021 12:48:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Sep 16, 2021 12:48:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Sep 16, 2021 12:48:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Sep 16, 2021 12:48:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Sep 16, 2021 12:48:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Sep 16, 2021 12:48:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Sep 16, 2021 12:48:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Sep 16, 2021 12:48:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Sep 16, 2021 12:48:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Sep 16, 2021 12:48:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Sep 16, 2021 12:48:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Sep 16, 2021 12:48:44 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.34.0-SNAPSHOT
Sep 16, 2021 12:48:48 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-09-16_05_48_44-18227987266575069105?project=apache-beam-testing
Sep 16, 2021 12:48:48 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-09-16_05_48_44-18227987266575069105
Sep 16, 2021 12:48:48 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-09-16_05_48_44-18227987266575069105
Sep 16, 2021 12:48:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-09-16T12:48:54.833Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-09-ioq9. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Sep 16, 2021 12:49:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-16T12:48:59.689Z: Worker configuration: e2-standard-2 in us-central1-a.
Sep 16, 2021 12:49:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-16T12:49:00.610Z: Expanding SplittableParDo operations into optimizable parts.
Sep 16, 2021 12:49:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-16T12:49:00.656Z: Expanding CollectionToSingleton operations into optimizable parts.
Sep 16, 2021 12:49:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-16T12:49:00.726Z: Expanding CoGroupByKey operations into optimizable parts.
Sep 16, 2021 12:49:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-16T12:49:00.802Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Sep 16, 2021 12:49:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-16T12:49:00.836Z: Expanding GroupByKey operations into streaming Read/Write steps
Sep 16, 2021 12:49:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-16T12:49:00.916Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Sep 16, 2021 12:49:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-16T12:49:01.036Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Sep 16, 2021 12:49:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-16T12:49:01.073Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Sep 16, 2021 12:49:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-16T12:49:01.109Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Sep 16, 2021 12:49:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-16T12:49:01.143Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Sep 16, 2021 12:49:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-16T12:49:01.197Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Sep 16, 2021 12:49:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-16T12:49:01.243Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Sep 16, 2021 12:49:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-16T12:49:01.281Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Sep 16, 2021 12:49:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-16T12:49:01.325Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Sep 16, 2021 12:49:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-16T12:49:01.355Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Sep 16, 2021 12:49:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-16T12:49:01.395Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Sep 16, 2021 12:49:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-16T12:49:01.437Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Sep 16, 2021 12:49:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-16T12:49:01.478Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Sep 16, 2021 12:49:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-16T12:49:01.531Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Sep 16, 2021 12:49:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-16T12:49:01.598Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Sep 16, 2021 12:49:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-16T12:49:01.663Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Sep 16, 2021 12:49:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-16T12:49:01.707Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Sep 16, 2021 12:49:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-16T12:49:01.770Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Sep 16, 2021 12:49:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-16T12:49:01.818Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Sep 16, 2021 12:49:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-16T12:49:01.882Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Sep 16, 2021 12:49:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-16T12:49:01.929Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Sep 16, 2021 12:49:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-16T12:49:01.993Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Sep 16, 2021 12:49:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-16T12:49:02.045Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Sep 16, 2021 12:49:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-16T12:49:02.106Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Sep 16, 2021 12:49:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-16T12:49:02.757Z: Starting 5 ****s in us-central1-a...
Sep 16, 2021 12:49:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-16T12:49:11.875Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Sep 16, 2021 12:49:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-16T12:49:47.462Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Sep 16, 2021 12:50:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-16T12:50:16.270Z: Workers have started successfully.
Sep 16, 2021 12:50:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-16T12:50:16.309Z: Workers have started successfully.
Sep 16, 2021 4:00:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-16T16:00:29.931Z: Cancel request is committed for workflow job: 2021-09-16_05_48_44-18227987266575069105.
Sep 16, 2021 4:00:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-16T16:00:30.102Z: Cleaning up.
Sep 16, 2021 4:00:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-16T16:00:30.175Z: Stopping **** pool...
Sep 16, 2021 4:00:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-16T16:00:30.252Z: Stopping **** pool...
Sep 16, 2021 4:02:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-16T16:02:49.981Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Sep 16, 2021 4:02:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-16T16:02:50.023Z: Worker pool stopped.
Sep 16, 2021 4:02:57 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-09-16_05_48_44-18227987266575069105 finished with status CANCELLED.
Load test results for test (ID): 6af5c062-d71c-461d-816b-11e5ea70a3e0 and timestamp: 2021-09-16T12:48:29.490000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11346.163
dataflow_v2_java11_total_bytes_count             2.18755428E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210916124428
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:30ac48ef18e0b01215057c18fd66d15bcc4d927ca67a8575beda181b516f2adb
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210916124428]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:30ac48ef18e0b01215057c18fd66d15bcc4d927ca67a8575beda181b516f2adb]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210916124428] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:30ac48ef18e0b01215057c18fd66d15bcc4d927ca67a8575beda181b516f2adb])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:30ac48ef18e0b01215057c18fd66d15bcc4d927ca67a8575beda181b516f2adb
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:30ac48ef18e0b01215057c18fd66d15bcc4d927ca67a8575beda181b516f2adb
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:30ac48ef18e0b01215057c18fd66d15bcc4d927ca67a8575beda181b516f2adb].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 20s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/yfimtcaj4hgw4

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #90

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/90/display/redirect?page=changes>

Changes:

[dhuntsperger] updated Maven-to-Gradle conversion step in Java quickstart

[noreply] [BEAM-10913] - Updating Grafana from v6.7.3 to v8.1.2 (#15503)

[noreply] [BEAM-12876] Adding doc and glossary entry for resource hints (#15499)

[noreply] [BEAM-12153] revert "implement GroupByKey with CombinePerKey with

[noreply] [BEAM-12845] Add AWS services as a runtime dependency to Spark Job


------------------------------------------
[...truncated 49.30 KB...]
The push refers to repository [us.gcr.io/apache-beam-testing/java-postcommit-it/java]
8198328d7c99: Preparing
2693c61bbf17: Preparing
afebf81fdc05: Preparing
81dc587f5cfd: Preparing
d6913312cfd2: Preparing
86557afc1619: Preparing
b99dbcf86420: Preparing
5e57d7492af4: Preparing
7aba11bbf702: Preparing
53158045ed29: Preparing
a250b7dbea7d: Preparing
f302772ac2ea: Preparing
3891808a925b: Preparing
d402f4f1b906: Preparing
00ef5416d927: Preparing
8555e663f65b: Preparing
d00da3cd7763: Preparing
4e61e63529c2: Preparing
799760671c38: Preparing
b99dbcf86420: Waiting
5e57d7492af4: Waiting
d00da3cd7763: Waiting
d402f4f1b906: Waiting
7aba11bbf702: Waiting
4e61e63529c2: Waiting
53158045ed29: Waiting
799760671c38: Waiting
86557afc1619: Waiting
00ef5416d927: Waiting
8555e663f65b: Waiting
3891808a925b: Waiting
a250b7dbea7d: Waiting
d6913312cfd2: Pushed
8198328d7c99: Pushed
2693c61bbf17: Pushed
afebf81fdc05: Pushed
81dc587f5cfd: Pushed
86557afc1619: Pushed
7aba11bbf702: Pushed
5e57d7492af4: Pushed
3891808a925b: Layer already exists
d402f4f1b906: Layer already exists
00ef5416d927: Layer already exists
8555e663f65b: Layer already exists
d00da3cd7763: Layer already exists
4e61e63529c2: Layer already exists
799760671c38: Layer already exists
a250b7dbea7d: Pushed
f302772ac2ea: Pushed
b99dbcf86420: Pushed
53158045ed29: Pushed
20210915124338: digest: sha256:4f331aa57513ab483d707531f198b1f395299fa5d5f4d8758bf0d57dd120e45d size: 4311

> Task :sdks:java:testing:load-tests:run
Sep 15, 2021 12:46:05 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Sep 15, 2021 12:46:05 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 195 files. Enable logging at DEBUG level to see which files will be staged.
Sep 15, 2021 12:46:06 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Sep 15, 2021 12:46:06 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Sep 15, 2021 12:46:08 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 195 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Sep 15, 2021 12:46:09 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 195 files cached, 0 files newly uploaded in 0 seconds
Sep 15, 2021 12:46:09 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Sep 15, 2021 12:46:09 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <106569 bytes, hash c30ece65034eb132cbdb8afc1ead21e770f1fd42fe856009cf65eba7aed22f72> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-ww7OZQNOsTLL24r8Hq0h53Dx_UL-hWAJz2Xrp67SL3I.pb
Sep 15, 2021 12:46:10 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Sep 15, 2021 12:46:10 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1b1c538d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1645f294, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6325f352, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@15c4af7a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6cbd0674, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@55d58825, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@19a64eae, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@29a98d9f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2da3b078, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@544e8149, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7fb66650, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1a96d94c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2a869a16, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@ae202c6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@46aa712c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6ada9c0c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7412ed6b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6e7c351d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b4a0aef, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@45cec376]
Sep 15, 2021 12:46:11 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Sep 15, 2021 12:46:11 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Sep 15, 2021 12:46:11 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Sep 15, 2021 12:46:11 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Sep 15, 2021 12:46:11 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@27755487, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4f0cab0a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@fe7b6b0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7ab4ae59, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77681ce4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5d96bdf8, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6f76c2cc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@306f6f1d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7d7cac8, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6fc6deb7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@367f0121, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7da39774, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@441b8382, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1df1ced0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5349b246, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@32b0876c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2aaf152b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@39bbd9e0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@27fe9713, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@11c3ff67]
Sep 15, 2021 12:46:11 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Sep 15, 2021 12:46:11 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Sep 15, 2021 12:46:11 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Sep 15, 2021 12:46:11 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Sep 15, 2021 12:46:11 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Sep 15, 2021 12:46:11 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Sep 15, 2021 12:46:11 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Sep 15, 2021 12:46:11 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Sep 15, 2021 12:46:11 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Sep 15, 2021 12:46:11 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Sep 15, 2021 12:46:11 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Sep 15, 2021 12:46:11 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.34.0-SNAPSHOT
Sep 15, 2021 12:46:12 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-09-15_05_46_11-1910158258827079702?project=apache-beam-testing
Sep 15, 2021 12:46:12 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-09-15_05_46_11-1910158258827079702
Sep 15, 2021 12:46:12 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-09-15_05_46_11-1910158258827079702
Sep 15, 2021 12:46:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-09-15T12:46:18.254Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-09-lphq. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Sep 15, 2021 12:46:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-15T12:46:22.220Z: Worker configuration: e2-standard-2 in us-central1-a.
Sep 15, 2021 12:46:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-15T12:46:22.984Z: Expanding SplittableParDo operations into optimizable parts.
Sep 15, 2021 12:46:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-15T12:46:23.027Z: Expanding CollectionToSingleton operations into optimizable parts.
Sep 15, 2021 12:46:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-15T12:46:23.104Z: Expanding CoGroupByKey operations into optimizable parts.
Sep 15, 2021 12:46:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-15T12:46:23.198Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Sep 15, 2021 12:46:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-15T12:46:23.244Z: Expanding GroupByKey operations into streaming Read/Write steps
Sep 15, 2021 12:46:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-15T12:46:23.306Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Sep 15, 2021 12:46:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-15T12:46:23.426Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Sep 15, 2021 12:46:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-15T12:46:23.470Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Sep 15, 2021 12:46:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-15T12:46:23.495Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Sep 15, 2021 12:46:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-15T12:46:23.528Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Sep 15, 2021 12:46:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-15T12:46:23.565Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Sep 15, 2021 12:46:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-15T12:46:23.600Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Sep 15, 2021 12:46:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-15T12:46:23.636Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Sep 15, 2021 12:46:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-15T12:46:23.670Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Sep 15, 2021 12:46:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-15T12:46:23.709Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Sep 15, 2021 12:46:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-15T12:46:23.757Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Sep 15, 2021 12:46:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-15T12:46:23.786Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Sep 15, 2021 12:46:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-15T12:46:23.812Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Sep 15, 2021 12:46:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-15T12:46:23.835Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Sep 15, 2021 12:46:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-15T12:46:23.887Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Sep 15, 2021 12:46:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-15T12:46:23.922Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Sep 15, 2021 12:46:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-15T12:46:23.973Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Sep 15, 2021 12:46:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-15T12:46:24.006Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Sep 15, 2021 12:46:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-15T12:46:24.052Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Sep 15, 2021 12:46:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-15T12:46:24.079Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Sep 15, 2021 12:46:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-15T12:46:24.108Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Sep 15, 2021 12:46:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-15T12:46:24.142Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Sep 15, 2021 12:46:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-15T12:46:24.176Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Sep 15, 2021 12:46:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-15T12:46:24.213Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Sep 15, 2021 12:46:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-15T12:46:24.649Z: Starting 5 ****s in us-central1-a...
Sep 15, 2021 12:46:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-15T12:46:55.358Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Sep 15, 2021 12:47:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-15T12:47:08.765Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Sep 15, 2021 12:47:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-15T12:47:41.143Z: Workers have started successfully.
Sep 15, 2021 12:47:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-15T12:47:41.182Z: Workers have started successfully.
Sep 15, 2021 4:00:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-15T16:00:32.489Z: Cancel request is committed for workflow job: 2021-09-15_05_46_11-1910158258827079702.
Sep 15, 2021 4:00:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-15T16:00:35.562Z: Cleaning up.
Sep 15, 2021 4:00:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-15T16:00:35.665Z: Stopping **** pool...
Sep 15, 2021 4:00:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-15T16:00:35.724Z: Stopping **** pool...
Sep 15, 2021 4:03:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-15T16:02:59.577Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Sep 15, 2021 4:03:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-15T16:02:59.615Z: Worker pool stopped.
Sep 15, 2021 4:03:04 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-09-15_05_46_11-1910158258827079702 finished with status CANCELLED.
Load test results for test (ID): db12e9c0-bc25-43b4-b05e-dc4ae84c3998 and timestamp: 2021-09-15T12:46:05.999000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11508.297
dataflow_v2_java11_total_bytes_count             3.33595543E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210915124338
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:4f331aa57513ab483d707531f198b1f395299fa5d5f4d8758bf0d57dd120e45d
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210915124338]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:4f331aa57513ab483d707531f198b1f395299fa5d5f4d8758bf0d57dd120e45d]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210915124338] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:4f331aa57513ab483d707531f198b1f395299fa5d5f4d8758bf0d57dd120e45d])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:4f331aa57513ab483d707531f198b1f395299fa5d5f4d8758bf0d57dd120e45d
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:4f331aa57513ab483d707531f198b1f395299fa5d5f4d8758bf0d57dd120e45d
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:4f331aa57513ab483d707531f198b1f395299fa5d5f4d8758bf0d57dd120e45d].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 56s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/tsskotcggr3to

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #89

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/89/display/redirect?page=changes>

Changes:

[rohde.samuel] [BEAM-12842] Add timestamp to test work item to deflake

[dhuntsperger] fixed broken Python tab on HCatalog IO page

[suztomo] [BEAM-12873] HL7v2IO: to leave schematizedData null, not empty

[noreply] Avoid apiary submission of job graph when it is not needed. (#15458)

[noreply] [BEAM-7261] Add support for BasicSessionCredentials for AWS credentials.

[noreply] Bump dataflow java container version to beam-master-20210913 (#15506)

[noreply] [BEAM-11980] Java GCS - Implement IO Request Count metrics (#15394)


------------------------------------------
[...truncated 49.05 KB...]
The push refers to repository [us.gcr.io/apache-beam-testing/java-postcommit-it/java]
1ee98579fc46: Preparing
b54e7baec2d3: Preparing
f612e1c6b2d2: Preparing
4b667ff364a2: Preparing
cc4bbd4ca4e2: Preparing
4948d45599e7: Preparing
21b5103441a9: Preparing
bc91acaffd9e: Preparing
e9f9e2ba990c: Preparing
7d8292d6730b: Preparing
12c7f9e457ef: Preparing
d37471bdb99e: Preparing
3891808a925b: Preparing
d402f4f1b906: Preparing
00ef5416d927: Preparing
8555e663f65b: Preparing
d00da3cd7763: Preparing
4e61e63529c2: Preparing
799760671c38: Preparing
12c7f9e457ef: Waiting
d37471bdb99e: Waiting
d00da3cd7763: Waiting
799760671c38: Waiting
21b5103441a9: Waiting
3891808a925b: Waiting
e9f9e2ba990c: Waiting
bc91acaffd9e: Waiting
d402f4f1b906: Waiting
00ef5416d927: Waiting
8555e663f65b: Waiting
7d8292d6730b: Waiting
4948d45599e7: Waiting
b54e7baec2d3: Pushed
cc4bbd4ca4e2: Pushed
f612e1c6b2d2: Pushed
4948d45599e7: Pushed
1ee98579fc46: Pushed
4b667ff364a2: Pushed
bc91acaffd9e: Pushed
e9f9e2ba990c: Pushed
3891808a925b: Layer already exists
d402f4f1b906: Layer already exists
00ef5416d927: Layer already exists
8555e663f65b: Layer already exists
d00da3cd7763: Layer already exists
4e61e63529c2: Layer already exists
21b5103441a9: Pushed
d37471bdb99e: Pushed
12c7f9e457ef: Pushed
799760671c38: Layer already exists
7d8292d6730b: Pushed
20210914124408: digest: sha256:7ef12d2436940dd2cbf0580d73a7825d2f817dfbcbd4aca75d266b79927be9d6 size: 4311

> Task :sdks:java:testing:load-tests:run
Sep 14, 2021 12:46:22 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Sep 14, 2021 12:46:22 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 195 files. Enable logging at DEBUG level to see which files will be staged.
Sep 14, 2021 12:46:23 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Sep 14, 2021 12:46:23 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Sep 14, 2021 12:46:25 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 195 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Sep 14, 2021 12:46:25 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 195 files cached, 0 files newly uploaded in 0 seconds
Sep 14, 2021 12:46:26 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Sep 14, 2021 12:46:26 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <106569 bytes, hash b180b453a33235a76be50b03c08b766e5c93e37f397942242f81202c6b15ccf0> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-sYC0U6MyNadr5QsDwIt2blyT4385eUIkL4EgLGsVzPA.pb
Sep 14, 2021 12:46:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Sep 14, 2021 12:46:27 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2d64160c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5f254608, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2eeb0f9b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1b1c538d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1645f294, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6325f352, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@15c4af7a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6cbd0674, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@55d58825, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@19a64eae, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@29a98d9f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2da3b078, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@544e8149, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7fb66650, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1a96d94c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2a869a16, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@ae202c6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@46aa712c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6ada9c0c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7412ed6b]
Sep 14, 2021 12:46:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Sep 14, 2021 12:46:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Sep 14, 2021 12:46:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Sep 14, 2021 12:46:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Sep 14, 2021 12:46:27 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@41853299, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@60d40ff4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2e5b7fba, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@27755487, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4f0cab0a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@fe7b6b0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7ab4ae59, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77681ce4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5d96bdf8, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6f76c2cc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@306f6f1d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7d7cac8, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6fc6deb7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@367f0121, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7da39774, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@441b8382, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1df1ced0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5349b246, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@32b0876c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2aaf152b]
Sep 14, 2021 12:46:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Sep 14, 2021 12:46:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Sep 14, 2021 12:46:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Sep 14, 2021 12:46:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Sep 14, 2021 12:46:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Sep 14, 2021 12:46:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Sep 14, 2021 12:46:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Sep 14, 2021 12:46:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Sep 14, 2021 12:46:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Sep 14, 2021 12:46:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Sep 14, 2021 12:46:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Sep 14, 2021 12:46:27 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.34.0-SNAPSHOT
Sep 14, 2021 12:46:29 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-09-14_05_46_28-9592787095606509542?project=apache-beam-testing
Sep 14, 2021 12:46:29 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-09-14_05_46_28-9592787095606509542
Sep 14, 2021 12:46:29 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-09-14_05_46_28-9592787095606509542
Sep 14, 2021 12:46:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-09-14T12:46:35.418Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-09-sz6l. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Sep 14, 2021 12:46:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-14T12:46:42.775Z: Worker configuration: e2-standard-2 in us-central1-a.
Sep 14, 2021 12:46:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-14T12:46:43.429Z: Expanding SplittableParDo operations into optimizable parts.
Sep 14, 2021 12:46:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-14T12:46:43.464Z: Expanding CollectionToSingleton operations into optimizable parts.
Sep 14, 2021 12:46:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-14T12:46:43.524Z: Expanding CoGroupByKey operations into optimizable parts.
Sep 14, 2021 12:46:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-14T12:46:43.592Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Sep 14, 2021 12:46:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-14T12:46:43.644Z: Expanding GroupByKey operations into streaming Read/Write steps
Sep 14, 2021 12:46:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-14T12:46:43.710Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Sep 14, 2021 12:46:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-14T12:46:43.811Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Sep 14, 2021 12:46:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-14T12:46:43.846Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Sep 14, 2021 12:46:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-14T12:46:43.879Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Sep 14, 2021 12:46:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-14T12:46:43.914Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Sep 14, 2021 12:46:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-14T12:46:43.947Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Sep 14, 2021 12:46:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-14T12:46:43.967Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Sep 14, 2021 12:46:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-14T12:46:44.001Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Sep 14, 2021 12:46:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-14T12:46:44.026Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Sep 14, 2021 12:46:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-14T12:46:44.058Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Sep 14, 2021 12:46:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-14T12:46:44.083Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Sep 14, 2021 12:46:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-14T12:46:44.114Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Sep 14, 2021 12:46:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-14T12:46:44.138Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Sep 14, 2021 12:46:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-14T12:46:44.168Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Sep 14, 2021 12:46:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-14T12:46:44.192Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Sep 14, 2021 12:46:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-14T12:46:44.224Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Sep 14, 2021 12:46:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-14T12:46:44.258Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Sep 14, 2021 12:46:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-14T12:46:44.284Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Sep 14, 2021 12:46:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-14T12:46:44.315Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Sep 14, 2021 12:46:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-14T12:46:44.350Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Sep 14, 2021 12:46:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-14T12:46:44.382Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Sep 14, 2021 12:46:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-14T12:46:44.406Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Sep 14, 2021 12:46:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-14T12:46:44.439Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Sep 14, 2021 12:46:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-14T12:46:44.475Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Sep 14, 2021 12:46:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-14T12:46:44.790Z: Starting 5 ****s in us-central1-a...
Sep 14, 2021 12:47:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-14T12:47:16.242Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Sep 14, 2021 12:47:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-14T12:47:28.377Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Sep 14, 2021 12:47:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-14T12:47:56.988Z: Workers have started successfully.
Sep 14, 2021 12:47:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-14T12:47:57.025Z: Workers have started successfully.
Sep 14, 2021 4:00:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-14T16:00:25.222Z: Cancel request is committed for workflow job: 2021-09-14_05_46_28-9592787095606509542.
Sep 14, 2021 4:00:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-14T16:00:25.283Z: Cleaning up.
Sep 14, 2021 4:00:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-14T16:00:25.390Z: Stopping **** pool...
Sep 14, 2021 4:00:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-14T16:00:25.433Z: Stopping **** pool...
Sep 14, 2021 4:02:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-14T16:02:50.141Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Sep 14, 2021 4:02:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-14T16:02:50.176Z: Worker pool stopped.
Sep 14, 2021 4:02:55 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-09-14_05_46_28-9592787095606509542 finished with status CANCELLED.
Load test results for test (ID): 930ef99c-602c-44b6-bb94-d9037d14ce6f and timestamp: 2021-09-14T12:46:22.857000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                  11477.29
dataflow_v2_java11_total_bytes_count             2.02747331E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210914124408
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:7ef12d2436940dd2cbf0580d73a7825d2f817dfbcbd4aca75d266b79927be9d6
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210914124408]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:7ef12d2436940dd2cbf0580d73a7825d2f817dfbcbd4aca75d266b79927be9d6]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210914124408] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:7ef12d2436940dd2cbf0580d73a7825d2f817dfbcbd4aca75d266b79927be9d6])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:7ef12d2436940dd2cbf0580d73a7825d2f817dfbcbd4aca75d266b79927be9d6
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:7ef12d2436940dd2cbf0580d73a7825d2f817dfbcbd4aca75d266b79927be9d6
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:7ef12d2436940dd2cbf0580d73a7825d2f817dfbcbd4aca75d266b79927be9d6].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 19s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/rvrf6gazkmdoi

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #88

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/88/display/redirect>

Changes:


------------------------------------------
[...truncated 40.78 KB...]
> Task :runners:google-cloud-dataflow-java:jar
> Task :sdks:java:container:java11:copyDockerfileDependencies
> Task :sdks:java:container:buildLinuxAmd64
> Task :sdks:java:container:goBuild
> Task :sdks:java:container:java11:copySdkHarnessLauncher
> Task :sdks:java:container:generateLicenseReport

> Task :sdks:java:container:pullLicenses
Copying already-fetched licenses from <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src/sdks/java/container/build/reports/dependency-license> to <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src/sdks/java/container/build/target/java_third_party_licenses>
Already using interpreter /usr/bin/python3
Using base prefix '/usr'
New python executable in <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src/sdks/java/container/build/virtualenv/bin/python3>
Also creating executable in <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src/sdks/java/container/build/virtualenv/bin/python>
Installing setuptools, pip, wheel...
done.
DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality.
Collecting beautifulsoup4<5.0,>=4.9.0
  Using cached beautifulsoup4-4.10.0-py3-none-any.whl (97 kB)
Collecting future<1.0.0,>=0.16.0
  Using cached future-0.18.2-py3-none-any.whl
Collecting pyyaml<6.0.0,>=3.12
  Using cached PyYAML-5.3.1-cp35-cp35m-linux_x86_64.whl
Collecting tenacity<6.0,>=5.0.2
  Using cached tenacity-5.1.5-py2.py3-none-any.whl (34 kB)
Collecting soupsieve>1.2
  Using cached soupsieve-2.1-py3-none-any.whl (32 kB)
Collecting six>=1.9.0
  Using cached six-1.16.0-py2.py3-none-any.whl (11 kB)
Installing collected packages: soupsieve, six, tenacity, pyyaml, future, beautifulsoup4
Successfully installed beautifulsoup4-4.10.0 future-0.18.2 pyyaml-5.3.1 six-1.16.0 soupsieve-2.1 tenacity-5.1.5
Executing <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src/sdks/java/container/build/virtualenv/bin/python> <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src/sdks/java/container/license_scripts/pull_licenses_java.py> --license_index=<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src/sdks/java/container/build/reports/dependency-license/index.json>        --output_dir=<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src/sdks/java/container/build/target/java_third_party_licenses>        --dep_url_yaml=<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src/sdks/java/container/license_scripts/dep_urls_java.yaml> 
INFO:root:Pulling license for 207 dependencies using 16 threads.
INFO:root:pull_licenses_java.py succeed. It took 1.776828 seconds with 16 threads.
Copying licenses from <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src/sdks/java/container/build/target/java_third_party_licenses> to <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src/sdks/java/container/build/target/third_party_licenses.>
Finished license_scripts.sh

> Task :sdks:java:container:java11:copyJavaThirdPartyLicenses

> Task :release:go-licenses:java:dockerRun
+ go-licenses save github.com/apache/beam/sdks/java/container --save_path=/output/licenses
+ go-licenses csv github.com/apache/beam/sdks/java/container
+ tee /output/licenses/list.csv
github.com/golang/protobuf,https://github.com/golang/protobuf/blob/master/LICENSE,BSD-3-Clause
google.golang.org/genproto/googleapis/rpc/status,https://github.com/googleapis/go-genproto/blob/master/LICENSE,Apache-2.0
github.com/apache/beam/sdks/java/container,https://github.com/apache/beam/blob/master/LICENSE,Apache-2.0
golang.org/x/net,https://go.googlesource.com/net/+/refs/heads/master/LICENSE,BSD-3-Clause
google.golang.org/protobuf,https://go.googlesource.com/protobuf/+/refs/heads/master/LICENSE,BSD-3-Clause
golang.org/x/sys,https://go.googlesource.com/sys/+/refs/heads/master/LICENSE,BSD-3-Clause
golang.org/x/text,https://go.googlesource.com/text/+/refs/heads/master/LICENSE,BSD-3-Clause
github.com/apache/beam/sdks/go/pkg/beam,https://github.com/apache/beam/blob/master/sdks/go/README.md,Apache-2.0
google.golang.org/grpc,https://github.com/grpc/grpc-go/blob/master/LICENSE,Apache-2.0
+ chmod -R a+w /output/licenses

> Task :release:go-licenses:java:createLicenses
> Task :sdks:java:container:java11:copyGolangLicenses
> Task :sdks:java:container:java11:dockerPrepare
> Task :sdks:java:container:java11:docker

> Task :runners:google-cloud-dataflow-java:buildAndPushDockerContainer
WARNING: `gcloud docker` will not be supported for Docker client versions above 18.03.

As an alternative, use `gcloud auth configure-docker` to configure `docker` to
use `gcloud` as a credential helper, then use `docker` as you would for non-GCR
registries, e.g. `docker pull gcr.io/project-id/my-image`. Add
`--verbosity=error` to silence this warning: `gcloud docker
--verbosity=error -- pull gcr.io/project-id/my-image`.

See: https://cloud.google.com/container-registry/docs/support/deprecation-notices#gcloud-docker

The push refers to repository [us.gcr.io/apache-beam-testing/java-postcommit-it/java]
cb48aa7c3179: Preparing
6e9c6659ed75: Preparing
89019dcc934a: Preparing
e500a7c139f7: Preparing
aad82327faab: Preparing
3a1781cb8501: Preparing
aa57da95a980: Preparing
8308d19d28ab: Preparing
eceaf9948f0b: Preparing
ca6d6093da44: Preparing
1646e7e61e2f: Preparing
a4dadbe25bfa: Preparing
3891808a925b: Preparing
d402f4f1b906: Preparing
00ef5416d927: Preparing
8555e663f65b: Preparing
d00da3cd7763: Preparing
4e61e63529c2: Preparing
799760671c38: Preparing
eceaf9948f0b: Waiting
3a1781cb8501: Waiting
ca6d6093da44: Waiting
1646e7e61e2f: Waiting
a4dadbe25bfa: Waiting
8308d19d28ab: Waiting
aa57da95a980: Waiting
3891808a925b: Waiting
8555e663f65b: Waiting
d402f4f1b906: Waiting
d00da3cd7763: Waiting
799760671c38: Waiting
aad82327faab: Pushed
6e9c6659ed75: Pushed
89019dcc934a: Pushed
cb48aa7c3179: Pushed
3a1781cb8501: Pushed
e500a7c139f7: Pushed
8308d19d28ab: Pushed
eceaf9948f0b: Pushed
3891808a925b: Layer already exists
d402f4f1b906: Layer already exists
00ef5416d927: Layer already exists
8555e663f65b: Layer already exists
1646e7e61e2f: Pushed
d00da3cd7763: Layer already exists
a4dadbe25bfa: Pushed
799760671c38: Layer already exists
4e61e63529c2: Layer already exists
aa57da95a980: Pushed
ca6d6093da44: Pushed
20210913124334: digest: sha256:13c61f0a3c9b5ea8be327ef9eeb69ec575c0fd95c30c78307c7da6641ab9aae0 size: 4311

> Task :sdks:java:testing:load-tests:run
Sep 13, 2021 12:45:19 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Sep 13, 2021 12:45:20 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 195 files. Enable logging at DEBUG level to see which files will be staged.
Sep 13, 2021 12:45:20 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Sep 13, 2021 12:45:20 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Sep 13, 2021 12:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 195 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Sep 13, 2021 12:45:23 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 195 files cached, 0 files newly uploaded in 0 seconds
Sep 13, 2021 12:45:23 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Sep 13, 2021 12:45:23 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <106569 bytes, hash fb1c024c1df5d5db05a96558d9dc52719b1cd78cffbc301084b5614ebc8c7438> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline--xwCTB311dsFqWVY2dxScZsc14z_vDAQhLVhTryMdDg.pb
Sep 13, 2021 12:45:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Sep 13, 2021 12:45:25 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2eeb0f9b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1b1c538d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1645f294, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6325f352, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@15c4af7a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6cbd0674, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@55d58825, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@19a64eae, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@29a98d9f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2da3b078, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@544e8149, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7fb66650, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1a96d94c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2a869a16, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@ae202c6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@46aa712c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6ada9c0c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7412ed6b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6e7c351d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b4a0aef]
Sep 13, 2021 12:45:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Sep 13, 2021 12:45:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Sep 13, 2021 12:45:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Sep 13, 2021 12:45:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Sep 13, 2021 12:45:25 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2e5b7fba, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@27755487, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4f0cab0a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@fe7b6b0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7ab4ae59, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77681ce4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5d96bdf8, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6f76c2cc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@306f6f1d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7d7cac8, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6fc6deb7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@367f0121, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7da39774, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@441b8382, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1df1ced0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5349b246, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@32b0876c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2aaf152b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@39bbd9e0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@27fe9713]
Sep 13, 2021 12:45:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Sep 13, 2021 12:45:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Sep 13, 2021 12:45:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Sep 13, 2021 12:45:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Sep 13, 2021 12:45:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Sep 13, 2021 12:45:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Sep 13, 2021 12:45:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Sep 13, 2021 12:45:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Sep 13, 2021 12:45:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Sep 13, 2021 12:45:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Sep 13, 2021 12:45:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Sep 13, 2021 12:45:25 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.34.0-SNAPSHOT
Sep 13, 2021 12:45:26 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-09-13_05_45_25-8165500818705185032?project=apache-beam-testing
Sep 13, 2021 12:45:26 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-09-13_05_45_25-8165500818705185032
Sep 13, 2021 12:45:26 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-09-13_05_45_25-8165500818705185032
Sep 13, 2021 12:45:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-09-13T12:45:32.952Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-09-3zb5. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Sep 13, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-13T12:45:38.295Z: Worker configuration: e2-standard-2 in us-central1-a.
Sep 13, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-13T12:45:39.084Z: Workflow failed. Causes: Project apache-beam-testing has insufficient quota(s) to execute this workflow with 1 instances in region us-central1. Quota summary (required/available): 1/24386 instances, 2/0 CPUs, 30/183701 disk GB, 0/2397 SSD disk GB, 1/288 instance groups, 1/291 managed instance groups, 1/517 instance templates, 1/615 in-use IP addresses.

Please see https://cloud.google.com/compute/docs/resource-quotas about requesting more quota.
Sep 13, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-13T12:45:39.118Z: Cleaning up.
Sep 13, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-13T12:45:39.176Z: Worker pool stopped.
Sep 13, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-13T12:45:40.339Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Sep 13, 2021 12:45:44 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-09-13_05_45_25-8165500818705185032 failed with status FAILED.
Sep 13, 2021 12:45:44 PM org.apache.beam.sdk.testutils.metrics.MetricsReader getCounterMetric
SEVERE: Failed to get metric totalBytes.count, from namespace co_gbk
Load test results for test (ID): 81bc422d-745c-4dac-85e0-d7a45094d161 and timestamp: 2021-09-13T12:45:20.539000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                       0.0
dataflow_v2_java11_total_bytes_count                      -1.0
Exception in thread "main" java.lang.RuntimeException: Invalid job state: FAILED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210913124334
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:13c61f0a3c9b5ea8be327ef9eeb69ec575c0fd95c30c78307c7da6641ab9aae0
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210913124334]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:13c61f0a3c9b5ea8be327ef9eeb69ec575c0fd95c30c78307c7da6641ab9aae0]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210913124334] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:13c61f0a3c9b5ea8be327ef9eeb69ec575c0fd95c30c78307c7da6641ab9aae0])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:13c61f0a3c9b5ea8be327ef9eeb69ec575c0fd95c30c78307c7da6641ab9aae0
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:13c61f0a3c9b5ea8be327ef9eeb69ec575c0fd95c30c78307c7da6641ab9aae0
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:13c61f0a3c9b5ea8be327ef9eeb69ec575c0fd95c30c78307c7da6641ab9aae0].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 2m 29s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/klzclaj5rpfdc

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #87

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/87/display/redirect>

Changes:


------------------------------------------
[...truncated 41.96 KB...]
Using base prefix '/usr'
New python executable in <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src/sdks/java/container/build/virtualenv/bin/python3>
Also creating executable in <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src/sdks/java/container/build/virtualenv/bin/python>
Installing setuptools, pip, wheel...
done.
DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality.
Collecting beautifulsoup4<5.0,>=4.9.0
  Using cached beautifulsoup4-4.10.0-py3-none-any.whl (97 kB)
Collecting future<1.0.0,>=0.16.0
  Using cached future-0.18.2-py3-none-any.whl
Collecting pyyaml<6.0.0,>=3.12
  Using cached PyYAML-5.3.1-cp35-cp35m-linux_x86_64.whl
Collecting tenacity<6.0,>=5.0.2
  Using cached tenacity-5.1.5-py2.py3-none-any.whl (34 kB)
Collecting soupsieve>1.2
  Using cached soupsieve-2.1-py3-none-any.whl (32 kB)
Collecting six>=1.9.0
  Using cached six-1.16.0-py2.py3-none-any.whl (11 kB)
Installing collected packages: soupsieve, six, tenacity, pyyaml, future, beautifulsoup4
Successfully installed beautifulsoup4-4.10.0 future-0.18.2 pyyaml-5.3.1 six-1.16.0 soupsieve-2.1 tenacity-5.1.5
Executing <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src/sdks/java/container/build/virtualenv/bin/python> <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src/sdks/java/container/license_scripts/pull_licenses_java.py> --license_index=<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src/sdks/java/container/build/reports/dependency-license/index.json>        --output_dir=<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src/sdks/java/container/build/target/java_third_party_licenses>        --dep_url_yaml=<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src/sdks/java/container/license_scripts/dep_urls_java.yaml> 
INFO:root:Pulling license for 207 dependencies using 16 threads.
INFO:root:pull_licenses_java.py succeed. It took 1.993543 seconds with 16 threads.
Copying licenses from <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src/sdks/java/container/build/target/java_third_party_licenses> to <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src/sdks/java/container/build/target/third_party_licenses.>

> Task :sdks:java:testing:load-tests:compileJava
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.

> Task :sdks:java:testing:load-tests:classes
> Task :sdks:java:testing:load-tests:jar

> Task :sdks:java:container:pullLicenses
Finished license_scripts.sh

> Task :sdks:java:container:java11:copyJavaThirdPartyLicenses

> Task :release:go-licenses:java:dockerRun
+ go-licenses save github.com/apache/beam/sdks/java/container --save_path=/output/licenses
+ go-licenses csv github.com/apache/beam/sdks/java/container
+ tee /output/licenses/list.csv
google.golang.org/genproto/googleapis/rpc/status,https://github.com/googleapis/go-genproto/blob/master/LICENSE,Apache-2.0
golang.org/x/text,https://go.googlesource.com/text/+/refs/heads/master/LICENSE,BSD-3-Clause
github.com/apache/beam/sdks/go/pkg/beam,https://github.com/apache/beam/blob/master/sdks/go/README.md,Apache-2.0
google.golang.org/grpc,https://github.com/grpc/grpc-go/blob/master/LICENSE,Apache-2.0
golang.org/x/net,https://go.googlesource.com/net/+/refs/heads/master/LICENSE,BSD-3-Clause
github.com/golang/protobuf,https://github.com/golang/protobuf/blob/master/LICENSE,BSD-3-Clause
google.golang.org/protobuf,https://go.googlesource.com/protobuf/+/refs/heads/master/LICENSE,BSD-3-Clause
golang.org/x/sys,https://go.googlesource.com/sys/+/refs/heads/master/LICENSE,BSD-3-Clause
github.com/apache/beam/sdks/java/container,https://github.com/apache/beam/blob/master/LICENSE,Apache-2.0
+ chmod -R a+w /output/licenses

> Task :release:go-licenses:java:createLicenses
> Task :sdks:java:container:java11:copyGolangLicenses
> Task :sdks:java:container:java11:dockerPrepare
> Task :sdks:java:container:java11:docker

> Task :runners:google-cloud-dataflow-java:buildAndPushDockerContainer
WARNING: `gcloud docker` will not be supported for Docker client versions above 18.03.

As an alternative, use `gcloud auth configure-docker` to configure `docker` to
use `gcloud` as a credential helper, then use `docker` as you would for non-GCR
registries, e.g. `docker pull gcr.io/project-id/my-image`. Add
`--verbosity=error` to silence this warning: `gcloud docker
--verbosity=error -- pull gcr.io/project-id/my-image`.

See: https://cloud.google.com/container-registry/docs/support/deprecation-notices#gcloud-docker

The push refers to repository [us.gcr.io/apache-beam-testing/java-postcommit-it/java]
965419297464: Preparing
af934d8ebe65: Preparing
a7b8054e29c9: Preparing
098bbdafffcb: Preparing
a28016f210da: Preparing
01a67a58c544: Preparing
3dbf36d043e0: Preparing
428bd452b211: Preparing
e387380d47be: Preparing
84336d956992: Preparing
c88bf45cc2c4: Preparing
eb070d43fe59: Preparing
3891808a925b: Preparing
d402f4f1b906: Preparing
00ef5416d927: Preparing
8555e663f65b: Preparing
d00da3cd7763: Preparing
4e61e63529c2: Preparing
799760671c38: Preparing
428bd452b211: Waiting
84336d956992: Waiting
c88bf45cc2c4: Waiting
e387380d47be: Waiting
8555e663f65b: Waiting
01a67a58c544: Waiting
d00da3cd7763: Waiting
eb070d43fe59: Waiting
799760671c38: Waiting
4e61e63529c2: Waiting
3dbf36d043e0: Waiting
00ef5416d927: Waiting
d402f4f1b906: Waiting
a7b8054e29c9: Pushed
a28016f210da: Pushed
af934d8ebe65: Pushed
965419297464: Pushed
01a67a58c544: Pushed
098bbdafffcb: Pushed
428bd452b211: Pushed
e387380d47be: Pushed
3891808a925b: Layer already exists
d402f4f1b906: Layer already exists
00ef5416d927: Layer already exists
8555e663f65b: Layer already exists
3dbf36d043e0: Pushed
d00da3cd7763: Layer already exists
eb070d43fe59: Pushed
4e61e63529c2: Layer already exists
799760671c38: Layer already exists
c88bf45cc2c4: Pushed
84336d956992: Pushed
20210912124340: digest: sha256:0e7f1ce6a58536211c674557a311d6d86359f8368b0b2861656162f876ada5f4 size: 4311

> Task :sdks:java:testing:load-tests:run
Sep 12, 2021 12:46:09 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Sep 12, 2021 12:46:10 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 195 files. Enable logging at DEBUG level to see which files will be staged.
Sep 12, 2021 12:46:11 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Sep 12, 2021 12:46:11 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Sep 12, 2021 12:46:13 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 195 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Sep 12, 2021 12:46:14 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 195 files cached, 0 files newly uploaded in 0 seconds
Sep 12, 2021 12:46:14 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Sep 12, 2021 12:46:14 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <106569 bytes, hash e9f936bb1b87ac265f45e5791c0ab45d144a7985ab86aa04cca515fdea242295> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-6fk2uxuHrCZfReV5HAq0XRRKeYWrhqoEzKUV_eokIpU.pb
Sep 12, 2021 12:46:16 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Sep 12, 2021 12:46:16 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2d64160c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5f254608, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2eeb0f9b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1b1c538d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1645f294, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6325f352, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@15c4af7a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6cbd0674, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@55d58825, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@19a64eae, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@29a98d9f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2da3b078, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@544e8149, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7fb66650, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1a96d94c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2a869a16, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@ae202c6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@46aa712c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6ada9c0c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7412ed6b]
Sep 12, 2021 12:46:16 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Sep 12, 2021 12:46:16 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Sep 12, 2021 12:46:16 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Sep 12, 2021 12:46:16 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Sep 12, 2021 12:46:16 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@41853299, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@60d40ff4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2e5b7fba, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@27755487, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4f0cab0a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@fe7b6b0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7ab4ae59, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77681ce4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5d96bdf8, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6f76c2cc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@306f6f1d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7d7cac8, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6fc6deb7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@367f0121, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7da39774, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@441b8382, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1df1ced0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5349b246, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@32b0876c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2aaf152b]
Sep 12, 2021 12:46:16 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Sep 12, 2021 12:46:16 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Sep 12, 2021 12:46:16 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Sep 12, 2021 12:46:16 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Sep 12, 2021 12:46:16 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Sep 12, 2021 12:46:16 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Sep 12, 2021 12:46:16 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Sep 12, 2021 12:46:16 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Sep 12, 2021 12:46:16 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Sep 12, 2021 12:46:16 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Sep 12, 2021 12:46:16 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Sep 12, 2021 12:46:16 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.34.0-SNAPSHOT
Sep 12, 2021 12:46:17 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-09-12_05_46_16-1467752297453580481?project=apache-beam-testing
Sep 12, 2021 12:46:17 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-09-12_05_46_16-1467752297453580481
Sep 12, 2021 12:46:17 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-09-12_05_46_16-1467752297453580481
Sep 12, 2021 12:46:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-09-12T12:46:24.495Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-09-4g7j. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Sep 12, 2021 12:46:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-12T12:46:29.782Z: Worker configuration: e2-standard-2 in us-central1-a.
Sep 12, 2021 12:46:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-12T12:46:30.604Z: Workflow failed. Causes: Project apache-beam-testing has insufficient quota(s) to execute this workflow with 1 instances in region us-central1. Quota summary (required/available): 1/24378 instances, 2/0 CPUs, 30/186331 disk GB, 0/2397 SSD disk GB, 1/280 instance groups, 1/283 managed instance groups, 1/508 instance templates, 1/607 in-use IP addresses.

Please see https://cloud.google.com/compute/docs/resource-quotas about requesting more quota.
Sep 12, 2021 12:46:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-12T12:46:30.642Z: Cleaning up.
Sep 12, 2021 12:46:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-12T12:46:30.693Z: Worker pool stopped.
Sep 12, 2021 12:46:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-12T12:46:31.857Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Sep 12, 2021 12:46:38 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-09-12_05_46_16-1467752297453580481 failed with status FAILED.
Sep 12, 2021 12:46:38 PM org.apache.beam.sdk.testutils.metrics.MetricsReader getCounterMetric
SEVERE: Failed to get metric totalBytes.count, from namespace co_gbk
Load test results for test (ID): 34b8d352-cbb2-47e4-9b8b-76c9e2782560 and timestamp: 2021-09-12T12:46:10.674000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                       0.0
dataflow_v2_java11_total_bytes_count                      -1.0
Exception in thread "main" java.lang.RuntimeException: Invalid job state: FAILED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210912124340
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:0e7f1ce6a58536211c674557a311d6d86359f8368b0b2861656162f876ada5f4
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210912124340]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:0e7f1ce6a58536211c674557a311d6d86359f8368b0b2861656162f876ada5f4]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210912124340] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:0e7f1ce6a58536211c674557a311d6d86359f8368b0b2861656162f876ada5f4])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:0e7f1ce6a58536211c674557a311d6d86359f8368b0b2861656162f876ada5f4
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:0e7f1ce6a58536211c674557a311d6d86359f8368b0b2861656162f876ada5f4
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:0e7f1ce6a58536211c674557a311d6d86359f8368b0b2861656162f876ada5f4].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3m 18s
101 actionable tasks: 71 executed, 28 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/az74l4n3vnsjm

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #86

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/86/display/redirect?page=changes>

Changes:

[noreply] Added type annotations to some combiners missing it. (#15414)

[noreply] [BEAM-12634] JmsIO auto scaling feature (#15464)

[noreply] [BEAM-12662] Get Flink version from cluster. (#15223)

[noreply] Port changes from Pub/Sub Lite to beam (#15418)

[heejong] [BEAM-12805] Fix XLang CombinePerKey test by explicitly assigning the

[BenWhitehead] [BEAM-8376] Google Cloud Firestore Connector - Add handling for

[noreply] Decreasing peak memory usage for beam.TupleCombineFn (#15494)

[noreply] [BEAM-12802] Add support for prefetch through data layers down through

[noreply] [BEAM-11097] Add implementation of side input cache (#15483)


------------------------------------------
[...truncated 41.27 KB...]
> Task :sdks:java:container:buildLinuxAmd64
> Task :sdks:java:container:goBuild
> Task :sdks:java:container:java11:copySdkHarnessLauncher
> Task :sdks:java:container:generateLicenseReport

> Task :sdks:java:container:pullLicenses
Copying already-fetched licenses from <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src/sdks/java/container/build/reports/dependency-license> to <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src/sdks/java/container/build/target/java_third_party_licenses>
Already using interpreter /usr/bin/python3
Using base prefix '/usr'
New python executable in <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src/sdks/java/container/build/virtualenv/bin/python3>
Also creating executable in <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src/sdks/java/container/build/virtualenv/bin/python>
Installing setuptools, pip, wheel...
done.
DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality.
Collecting beautifulsoup4<5.0,>=4.9.0
  Using cached beautifulsoup4-4.10.0-py3-none-any.whl (97 kB)
Collecting future<1.0.0,>=0.16.0
  Using cached future-0.18.2-py3-none-any.whl
Collecting pyyaml<6.0.0,>=3.12
  Using cached PyYAML-5.3.1-cp35-cp35m-linux_x86_64.whl
Collecting tenacity<6.0,>=5.0.2
  Using cached tenacity-5.1.5-py2.py3-none-any.whl (34 kB)
Collecting soupsieve>1.2
  Using cached soupsieve-2.1-py3-none-any.whl (32 kB)
Collecting six>=1.9.0
  Using cached six-1.16.0-py2.py3-none-any.whl (11 kB)
Installing collected packages: soupsieve, six, tenacity, pyyaml, future, beautifulsoup4
Successfully installed beautifulsoup4-4.10.0 future-0.18.2 pyyaml-5.3.1 six-1.16.0 soupsieve-2.1 tenacity-5.1.5
Executing <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src/sdks/java/container/build/virtualenv/bin/python> <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src/sdks/java/container/license_scripts/pull_licenses_java.py> --license_index=<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src/sdks/java/container/build/reports/dependency-license/index.json>        --output_dir=<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src/sdks/java/container/build/target/java_third_party_licenses>        --dep_url_yaml=<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src/sdks/java/container/license_scripts/dep_urls_java.yaml> 
INFO:root:Pulling license for 207 dependencies using 16 threads.
INFO:root:pull_licenses_java.py succeed. It took 1.735595 seconds with 16 threads.
Copying licenses from <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src/sdks/java/container/build/target/java_third_party_licenses> to <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src/sdks/java/container/build/target/third_party_licenses.>
Finished license_scripts.sh

> Task :sdks:java:container:java11:copyJavaThirdPartyLicenses

> Task :release:go-licenses:java:dockerRun
+ go-licenses save github.com/apache/beam/sdks/java/container --save_path=/output/licenses
+ go-licenses csv github.com/apache/beam/sdks/java/container
+ tee /output/licenses/list.csv
github.com/apache/beam/sdks/go/pkg/beam,https://github.com/apache/beam/blob/master/sdks/go/README.md,Apache-2.0
github.com/apache/beam/sdks/java/container,https://github.com/apache/beam/blob/master/LICENSE,Apache-2.0
google.golang.org/grpc,https://github.com/grpc/grpc-go/blob/master/LICENSE,Apache-2.0
golang.org/x/net,https://go.googlesource.com/net/+/refs/heads/master/LICENSE,BSD-3-Clause
github.com/golang/protobuf,https://github.com/golang/protobuf/blob/master/LICENSE,BSD-3-Clause
google.golang.org/protobuf,https://go.googlesource.com/protobuf/+/refs/heads/master/LICENSE,BSD-3-Clause
google.golang.org/genproto/googleapis/rpc/status,https://github.com/googleapis/go-genproto/blob/master/LICENSE,Apache-2.0
golang.org/x/sys,https://go.googlesource.com/sys/+/refs/heads/master/LICENSE,BSD-3-Clause
golang.org/x/text,https://go.googlesource.com/text/+/refs/heads/master/LICENSE,BSD-3-Clause
+ chmod -R a+w /output/licenses

> Task :release:go-licenses:java:createLicenses
> Task :sdks:java:container:java11:copyGolangLicenses
> Task :sdks:java:container:java11:dockerPrepare
> Task :sdks:java:container:java11:docker

> Task :runners:google-cloud-dataflow-java:buildAndPushDockerContainer
WARNING: `gcloud docker` will not be supported for Docker client versions above 18.03.

As an alternative, use `gcloud auth configure-docker` to configure `docker` to
use `gcloud` as a credential helper, then use `docker` as you would for non-GCR
registries, e.g. `docker pull gcr.io/project-id/my-image`. Add
`--verbosity=error` to silence this warning: `gcloud docker
--verbosity=error -- pull gcr.io/project-id/my-image`.

See: https://cloud.google.com/container-registry/docs/support/deprecation-notices#gcloud-docker

The push refers to repository [us.gcr.io/apache-beam-testing/java-postcommit-it/java]
94155687fd15: Preparing
1a6154aca6a5: Preparing
4db553e1667a: Preparing
1db591f28d49: Preparing
e08ddbdc02fc: Preparing
010d17a9161d: Preparing
b34dc0cc89f9: Preparing
3b95ff4a5a2f: Preparing
cbaa10add834: Preparing
c09300ef95e5: Preparing
1c88a5e3119e: Preparing
1e87170fa4e6: Preparing
3891808a925b: Preparing
d402f4f1b906: Preparing
00ef5416d927: Preparing
8555e663f65b: Preparing
d00da3cd7763: Preparing
4e61e63529c2: Preparing
799760671c38: Preparing
c09300ef95e5: Waiting
1c88a5e3119e: Waiting
1e87170fa4e6: Waiting
010d17a9161d: Waiting
b34dc0cc89f9: Waiting
3891808a925b: Waiting
3b95ff4a5a2f: Waiting
8555e663f65b: Waiting
d00da3cd7763: Waiting
00ef5416d927: Waiting
cbaa10add834: Waiting
4e61e63529c2: Waiting
d402f4f1b906: Waiting
799760671c38: Waiting
1a6154aca6a5: Pushed
4db553e1667a: Pushed
e08ddbdc02fc: Pushed
010d17a9161d: Pushed
94155687fd15: Pushed
3b95ff4a5a2f: Pushed
1db591f28d49: Pushed
cbaa10add834: Pushed
3891808a925b: Layer already exists
d402f4f1b906: Layer already exists
1c88a5e3119e: Pushed
00ef5416d927: Layer already exists
b34dc0cc89f9: Pushed
1e87170fa4e6: Pushed
8555e663f65b: Layer already exists
d00da3cd7763: Layer already exists
4e61e63529c2: Layer already exists
799760671c38: Layer already exists
c09300ef95e5: Pushed
20210911124335: digest: sha256:ec8a6b706ab30a1f4484e5b634dffcda5a41d1ce4d0225ad66f82959c24f0953 size: 4311

> Task :sdks:java:testing:load-tests:run
Sep 11, 2021 12:45:23 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Sep 11, 2021 12:45:24 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 195 files. Enable logging at DEBUG level to see which files will be staged.
Sep 11, 2021 12:45:24 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Sep 11, 2021 12:45:24 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Sep 11, 2021 12:45:26 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 195 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Sep 11, 2021 12:45:27 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 195 files cached, 0 files newly uploaded in 0 seconds
Sep 11, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Sep 11, 2021 12:45:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <106569 bytes, hash 53f1ff7280f0aaa797c7ce909826ae925c0fbce7a9dce4b552c7139927065582> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-U_H_coDwqqeXx86QmCauklwPvOep3OS1UscTmScGVYI.pb
Sep 11, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Sep 11, 2021 12:45:29 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5f254608, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2eeb0f9b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1b1c538d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1645f294, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6325f352, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@15c4af7a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6cbd0674, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@55d58825, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@19a64eae, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@29a98d9f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2da3b078, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@544e8149, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7fb66650, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1a96d94c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2a869a16, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@ae202c6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@46aa712c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6ada9c0c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7412ed6b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6e7c351d]
Sep 11, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Sep 11, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Sep 11, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Sep 11, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Sep 11, 2021 12:45:29 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@60d40ff4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2e5b7fba, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@27755487, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4f0cab0a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@fe7b6b0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7ab4ae59, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77681ce4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5d96bdf8, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6f76c2cc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@306f6f1d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7d7cac8, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6fc6deb7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@367f0121, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7da39774, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@441b8382, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1df1ced0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5349b246, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@32b0876c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2aaf152b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@39bbd9e0]
Sep 11, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Sep 11, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Sep 11, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Sep 11, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Sep 11, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Sep 11, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Sep 11, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Sep 11, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Sep 11, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Sep 11, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Sep 11, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Sep 11, 2021 12:45:29 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.34.0-SNAPSHOT
Sep 11, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-09-11_05_45_29-2969291427559089470?project=apache-beam-testing
Sep 11, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-09-11_05_45_29-2969291427559089470
Sep 11, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-09-11_05_45_29-2969291427559089470
Sep 11, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-09-11T12:45:36.679Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-09-ft0i. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Sep 11, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-11T12:45:40.905Z: Worker configuration: e2-standard-2 in us-central1-a.
Sep 11, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-11T12:45:41.574Z: Workflow failed. Causes: Project apache-beam-testing has insufficient quota(s) to execute this workflow with 1 instances in region us-central1. Quota summary (required/available): 1/24323 instances, 2/0 CPUs, 30/168311 disk GB, 0/2397 SSD disk GB, 1/226 instance groups, 1/229 managed instance groups, 1/452 instance templates, 1/552 in-use IP addresses.

Please see https://cloud.google.com/compute/docs/resource-quotas about requesting more quota.
Sep 11, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-11T12:45:41.663Z: Cleaning up.
Sep 11, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-11T12:45:41.712Z: Worker pool stopped.
Sep 11, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-11T12:45:42.985Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Sep 11, 2021 12:45:47 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-09-11_05_45_29-2969291427559089470 failed with status FAILED.
Sep 11, 2021 12:45:47 PM org.apache.beam.sdk.testutils.metrics.MetricsReader getCounterMetric
SEVERE: Failed to get metric totalBytes.count, from namespace co_gbk
Load test results for test (ID): a220d965-d152-4485-9e32-8c692a5803a6 and timestamp: 2021-09-11T12:45:24.406000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                       0.0
dataflow_v2_java11_total_bytes_count                      -1.0
Exception in thread "main" java.lang.RuntimeException: Invalid job state: FAILED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210911124335
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ec8a6b706ab30a1f4484e5b634dffcda5a41d1ce4d0225ad66f82959c24f0953
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210911124335]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ec8a6b706ab30a1f4484e5b634dffcda5a41d1ce4d0225ad66f82959c24f0953]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210911124335] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ec8a6b706ab30a1f4484e5b634dffcda5a41d1ce4d0225ad66f82959c24f0953])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ec8a6b706ab30a1f4484e5b634dffcda5a41d1ce4d0225ad66f82959c24f0953
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ec8a6b706ab30a1f4484e5b634dffcda5a41d1ce4d0225ad66f82959c24f0953
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ec8a6b706ab30a1f4484e5b634dffcda5a41d1ce4d0225ad66f82959c24f0953].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 2m 31s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/bvfr5nun6tgdk

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #85

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/85/display/redirect?page=changes>

Changes:

[ruwan.lambrichts] Clarify additional_bq_parameters argument

[kawaigin] [BEAM-10708] Support streaming cache in beam_sql magic

[noreply] Fix broken 'differences from pandas' link

[noreply] Added GroupBy row in Aggregation table.

[Luke Cwik] [BEAM-12769] Fix typo in test class name, CLass -> Class

[Etienne Chauchot] [BEAM-5172] Temporary ignore testSplit and testSizes tests waiting for a

[samuelw] [BEAM-12740] Remove matching to filter files when renaming gcs files in

[noreply] [BEAM-3304] Helper functions for triggers (#15430)

[esert] Bump a throttling counter on BigQueryRead retries due to

[noreply] [BEAM-5097] Increment counter for "small words" in go SDK example

[noreply] Register MapCoder, some comments/cleanup. (#15471)

[noreply] [BEAM-12588] Multimap user state proto changes (#15473)


------------------------------------------
[...truncated 49.16 KB...]
ab7ede592472: Preparing
c55123596e7b: Preparing
8eb9330a5aff: Preparing
4838125b2797: Preparing
36cd146c6e6f: Preparing
ed91b4e49e12: Preparing
e78bef178800: Preparing
2d6c483da6e1: Preparing
6411bb8a04e7: Preparing
3891808a925b: Preparing
d402f4f1b906: Preparing
00ef5416d927: Preparing
8555e663f65b: Preparing
d00da3cd7763: Preparing
4e61e63529c2: Preparing
799760671c38: Preparing
ed91b4e49e12: Waiting
8eb9330a5aff: Waiting
4838125b2797: Waiting
36cd146c6e6f: Waiting
4e61e63529c2: Waiting
00ef5416d927: Waiting
8555e663f65b: Waiting
2d6c483da6e1: Waiting
d00da3cd7763: Waiting
3891808a925b: Waiting
799760671c38: Waiting
6411bb8a04e7: Waiting
d402f4f1b906: Waiting
c55123596e7b: Pushed
8e4c1e232330: Pushed
1acfefc472d7: Pushed
8eb9330a5aff: Pushed
02e43a8dfbc6: Pushed
ab7ede592472: Pushed
36cd146c6e6f: Pushed
ed91b4e49e12: Pushed
3891808a925b: Layer already exists
d402f4f1b906: Layer already exists
2d6c483da6e1: Pushed
4838125b2797: Pushed
00ef5416d927: Layer already exists
8555e663f65b: Layer already exists
d00da3cd7763: Layer already exists
4e61e63529c2: Layer already exists
799760671c38: Layer already exists
6411bb8a04e7: Pushed
e78bef178800: Pushed
20210910124335: digest: sha256:7e1e2a73098a7cf881d80fbc9ca948dc8e6417db46bb677cea5b898b45ba5bc2 size: 4311

> Task :sdks:java:testing:load-tests:run
Sep 10, 2021 12:45:24 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Sep 10, 2021 12:45:25 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 195 files. Enable logging at DEBUG level to see which files will be staged.
Sep 10, 2021 12:45:25 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Sep 10, 2021 12:45:25 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Sep 10, 2021 12:45:28 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 195 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Sep 10, 2021 12:45:28 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 195 files cached, 0 files newly uploaded in 0 seconds
Sep 10, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Sep 10, 2021 12:45:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <106575 bytes, hash 4335c6f3d8791c595b3cad64712c15b7f7ff05802d5bab6374f09eab0715ceb3> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-QzXG89h5HFlbPK1kcSwVt_f_BYAtW6tjdPCeqwcVzrM.pb
Sep 10, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Sep 10, 2021 12:45:30 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5f254608, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2eeb0f9b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1b1c538d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1645f294, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6325f352, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@15c4af7a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6cbd0674, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@55d58825, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@19a64eae, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@29a98d9f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2da3b078, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@544e8149, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7fb66650, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1a96d94c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2a869a16, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@ae202c6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@46aa712c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6ada9c0c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7412ed6b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6e7c351d]
Sep 10, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Sep 10, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Sep 10, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Sep 10, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Sep 10, 2021 12:45:30 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@60d40ff4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2e5b7fba, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@27755487, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4f0cab0a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@fe7b6b0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7ab4ae59, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77681ce4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5d96bdf8, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6f76c2cc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@306f6f1d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7d7cac8, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6fc6deb7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@367f0121, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7da39774, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@441b8382, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1df1ced0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5349b246, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@32b0876c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2aaf152b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@39bbd9e0]
Sep 10, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Sep 10, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Sep 10, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Sep 10, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Sep 10, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Sep 10, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Sep 10, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Sep 10, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Sep 10, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Sep 10, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Sep 10, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Sep 10, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.34.0-SNAPSHOT
Sep 10, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-09-10_05_45_30-16303082107601742899?project=apache-beam-testing
Sep 10, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-09-10_05_45_30-16303082107601742899
Sep 10, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-09-10_05_45_30-16303082107601742899
Sep 10, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-09-10T12:45:37.901Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-09-iqmx. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Sep 10, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-10T12:45:42.997Z: Worker configuration: e2-standard-2 in us-central1-a.
Sep 10, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-10T12:45:43.616Z: Expanding SplittableParDo operations into optimizable parts.
Sep 10, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-10T12:45:43.671Z: Expanding CollectionToSingleton operations into optimizable parts.
Sep 10, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-10T12:45:43.742Z: Expanding CoGroupByKey operations into optimizable parts.
Sep 10, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-10T12:45:43.827Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Sep 10, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-10T12:45:43.865Z: Expanding GroupByKey operations into streaming Read/Write steps
Sep 10, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-10T12:45:43.930Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Sep 10, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-10T12:45:44.114Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Sep 10, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-10T12:45:44.171Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Sep 10, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-10T12:45:44.337Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Sep 10, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-10T12:45:44.608Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Sep 10, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-10T12:45:44.686Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Sep 10, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-10T12:45:44.721Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Sep 10, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-10T12:45:44.753Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Sep 10, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-10T12:45:44.780Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Sep 10, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-10T12:45:44.824Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Sep 10, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-10T12:45:44.855Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Sep 10, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-10T12:45:44.890Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Sep 10, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-10T12:45:44.926Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Sep 10, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-10T12:45:44.961Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Sep 10, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-10T12:45:44.997Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Sep 10, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-10T12:45:45.038Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Sep 10, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-10T12:45:45.078Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Sep 10, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-10T12:45:45.115Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Sep 10, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-10T12:45:45.153Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Sep 10, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-10T12:45:45.192Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Sep 10, 2021 12:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-10T12:45:45.241Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Sep 10, 2021 12:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-10T12:45:45.277Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Sep 10, 2021 12:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-10T12:45:45.308Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Sep 10, 2021 12:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-10T12:45:45.342Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Sep 10, 2021 12:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-10T12:45:45.818Z: Starting 5 ****s in us-central1-a...
Sep 10, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-10T12:45:48.163Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Sep 10, 2021 12:46:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-10T12:46:28.567Z: Autoscaling: Raised the number of ****s to 1 so that the pipeline can catch up with its backlog and keep up with its input rate.
Sep 10, 2021 12:46:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-10T12:46:28.599Z: Resized **** pool to 1, though goal was 5.  This could be a quota issue.
Sep 10, 2021 12:46:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-10T12:46:38.875Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Sep 10, 2021 12:47:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-10T12:47:07.132Z: Workers have started successfully.
Sep 10, 2021 12:47:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-10T12:47:07.171Z: Workers have started successfully.
Sep 10, 2021 4:00:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-10T16:00:34.046Z: Cancel request is committed for workflow job: 2021-09-10_05_45_30-16303082107601742899.
Sep 10, 2021 4:00:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-10T16:00:34.121Z: Cleaning up.
Sep 10, 2021 4:00:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-10T16:00:34.193Z: Stopping **** pool...
Sep 10, 2021 4:00:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-10T16:00:34.247Z: Stopping **** pool...
Sep 10, 2021 4:02:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-10T16:02:48.478Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Sep 10, 2021 4:02:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-10T16:02:48.513Z: Worker pool stopped.
Sep 10, 2021 4:02:57 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-09-10_05_45_30-16303082107601742899 finished with status CANCELLED.
Load test results for test (ID): 36072a24-4597-4503-a317-4cfc7eb062b4 and timestamp: 2021-09-10T12:45:25.600000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11565.002
dataflow_v2_java11_total_bytes_count              3.7795923E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210910124335
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:7e1e2a73098a7cf881d80fbc9ca948dc8e6417db46bb677cea5b898b45ba5bc2
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210910124335]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:7e1e2a73098a7cf881d80fbc9ca948dc8e6417db46bb677cea5b898b45ba5bc2]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210910124335] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:7e1e2a73098a7cf881d80fbc9ca948dc8e6417db46bb677cea5b898b45ba5bc2])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:7e1e2a73098a7cf881d80fbc9ca948dc8e6417db46bb677cea5b898b45ba5bc2
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:7e1e2a73098a7cf881d80fbc9ca948dc8e6417db46bb677cea5b898b45ba5bc2
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:7e1e2a73098a7cf881d80fbc9ca948dc8e6417db46bb677cea5b898b45ba5bc2].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 40s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/tcftuaqrpxlcg

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #84

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/84/display/redirect?page=changes>

Changes:

[suztomo] [BEAM-11205] Upgrading the Libraries BOM to v22

[randomstep] [BEAM-12708] Bump arrow-memory-netty

[Etienne Chauchot] [BEAM-12153] implement GroupByKey with CombinePerKey with Concatenate

[Etienne Chauchot] [BEAM-11023] Increase memory in SS Validates runner tests to avoid OOM

[vincent.marquez] [BEAM-9008] adds CassandraIO.readAll

[Etienne Chauchot] [BEAM-12727] extract Concatenate CombineFn to runner-core module to

[noreply] Add display data for JdbcIO.write (#15460)

[noreply] Merge pull request #15480: [BEAM-12356] Make sure DatasetService is

[noreply] [BEAM-11981] Java Bigtable - Implement IO Request Count metrics (#15342)

[noreply] [BEAM-12834] Improve Go SDK cross-language documentation and API.


------------------------------------------
[...truncated 50.01 KB...]
3891808a925b: Layer already exists
d402f4f1b906: Layer already exists
00ef5416d927: Layer already exists
8555e663f65b: Layer already exists
d00da3cd7763: Layer already exists
4e61e63529c2: Layer already exists
799760671c38: Layer already exists
c56c93cd7e33: Pushed
b9058d6ab69a: Pushed
531cceab33de: Pushed
bcd7bf3fb3a7: Pushed
20210909124347: digest: sha256:770040ac97f1638e45ff50fa8f6d3fea3075431957f1651ae2d17e27f3d98903 size: 4311

> Task :sdks:java:testing:load-tests:run
Sep 09, 2021 12:46:16 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Sep 09, 2021 12:46:17 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 195 files. Enable logging at DEBUG level to see which files will be staged.
Sep 09, 2021 12:46:18 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Sep 09, 2021 12:46:18 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Sep 09, 2021 12:46:21 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 195 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Sep 09, 2021 12:46:22 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 195 files cached, 0 files newly uploaded in 0 seconds
Sep 09, 2021 12:46:22 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Sep 09, 2021 12:46:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <106575 bytes, hash 956d46f529de46e19e099f830be31e52a78a78912eaf90a5a61141c860e4e592> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-lW1G9SneRuGeCZ-DC-MeUqeKeJEur5ClphFByGDk5ZI.pb
Sep 09, 2021 12:46:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Sep 09, 2021 12:46:24 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2d64160c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5f254608, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2eeb0f9b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1b1c538d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1645f294, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6325f352, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@15c4af7a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6cbd0674, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@55d58825, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@19a64eae, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@29a98d9f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2da3b078, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@544e8149, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7fb66650, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1a96d94c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2a869a16, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@ae202c6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@46aa712c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6ada9c0c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7412ed6b]
Sep 09, 2021 12:46:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Sep 09, 2021 12:46:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Sep 09, 2021 12:46:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Sep 09, 2021 12:46:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Sep 09, 2021 12:46:24 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@41853299, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@60d40ff4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2e5b7fba, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@27755487, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4f0cab0a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@fe7b6b0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7ab4ae59, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77681ce4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5d96bdf8, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6f76c2cc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@306f6f1d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7d7cac8, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6fc6deb7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@367f0121, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7da39774, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@441b8382, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1df1ced0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5349b246, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@32b0876c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2aaf152b]
Sep 09, 2021 12:46:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Sep 09, 2021 12:46:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Sep 09, 2021 12:46:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Sep 09, 2021 12:46:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Sep 09, 2021 12:46:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Sep 09, 2021 12:46:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Sep 09, 2021 12:46:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Sep 09, 2021 12:46:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Sep 09, 2021 12:46:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Sep 09, 2021 12:46:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Sep 09, 2021 12:46:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Sep 09, 2021 12:46:24 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.34.0-SNAPSHOT
Sep 09, 2021 12:46:25 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-09-09_05_46_25-17409469238405457671?project=apache-beam-testing
Sep 09, 2021 12:46:25 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-09-09_05_46_25-17409469238405457671
Sep 09, 2021 12:46:25 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-09-09_05_46_25-17409469238405457671
Sep 09, 2021 12:46:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-09-09T12:46:32.613Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-09-s61a. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Sep 09, 2021 12:46:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-09T12:46:37.305Z: Worker configuration: e2-standard-2 in us-central1-a.
Sep 09, 2021 12:46:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-09T12:46:38.055Z: Expanding SplittableParDo operations into optimizable parts.
Sep 09, 2021 12:46:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-09T12:46:38.097Z: Expanding CollectionToSingleton operations into optimizable parts.
Sep 09, 2021 12:46:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-09T12:46:38.167Z: Expanding CoGroupByKey operations into optimizable parts.
Sep 09, 2021 12:46:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-09T12:46:38.250Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Sep 09, 2021 12:46:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-09T12:46:38.292Z: Expanding GroupByKey operations into streaming Read/Write steps
Sep 09, 2021 12:46:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-09T12:46:38.393Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Sep 09, 2021 12:46:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-09T12:46:38.509Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Sep 09, 2021 12:46:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-09T12:46:38.567Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Sep 09, 2021 12:46:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-09T12:46:38.594Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Sep 09, 2021 12:46:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-09T12:46:38.656Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Sep 09, 2021 12:46:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-09T12:46:38.720Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Sep 09, 2021 12:46:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-09T12:46:38.764Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Sep 09, 2021 12:46:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-09T12:46:38.803Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Sep 09, 2021 12:46:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-09T12:46:38.837Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Sep 09, 2021 12:46:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-09T12:46:38.907Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Sep 09, 2021 12:46:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-09T12:46:38.935Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Sep 09, 2021 12:46:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-09T12:46:38.971Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Sep 09, 2021 12:46:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-09T12:46:39.008Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Sep 09, 2021 12:46:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-09T12:46:39.062Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Sep 09, 2021 12:46:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-09T12:46:39.087Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Sep 09, 2021 12:46:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-09T12:46:39.118Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Sep 09, 2021 12:46:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-09T12:46:39.138Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Sep 09, 2021 12:46:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-09T12:46:39.180Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Sep 09, 2021 12:46:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-09T12:46:39.213Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Sep 09, 2021 12:46:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-09T12:46:39.262Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Sep 09, 2021 12:46:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-09T12:46:39.293Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Sep 09, 2021 12:46:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-09T12:46:39.330Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Sep 09, 2021 12:46:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-09T12:46:39.385Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Sep 09, 2021 12:46:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-09T12:46:39.422Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Sep 09, 2021 12:46:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-09T12:46:39.840Z: Starting 5 ****s in us-central1-a...
Sep 09, 2021 12:47:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-09T12:47:12.063Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Sep 09, 2021 12:47:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-09T12:47:25.646Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Sep 09, 2021 12:47:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-09T12:47:53.378Z: Workers have started successfully.
Sep 09, 2021 12:47:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-09T12:47:53.406Z: Workers have started successfully.
Sep 09, 2021 2:43:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-09T14:43:40.174Z: Staged package aws-java-sdk-cloudwatch-1.11.974-Ub2yicrz1p27ushPndFcVBigc3SuIZ4G-Yaq45B1iUo.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/aws-java-sdk-cloudwatch-1.11.974-Ub2yicrz1p27ushPndFcVBigc3SuIZ4G-Yaq45B1iUo.jar' is inaccessible.
Sep 09, 2021 2:43:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-09T14:43:40.261Z: Staged package aws-java-sdk-core-1.11.974-s3lPhICKsq1xExGOyj03VQHRqOsuX5YUc-UUlnhiA64.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/aws-java-sdk-core-1.11.974-s3lPhICKsq1xExGOyj03VQHRqOsuX5YUc-UUlnhiA64.jar' is inaccessible.
Sep 09, 2021 2:43:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-09T14:43:40.341Z: Staged package aws-java-sdk-dynamodb-1.11.974-RLvIDrX-TDIDN06gNnaHm9dQbuHCD_XSBesOuniSTk4.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/aws-java-sdk-dynamodb-1.11.974-RLvIDrX-TDIDN06gNnaHm9dQbuHCD_XSBesOuniSTk4.jar' is inaccessible.
Sep 09, 2021 2:43:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-09T14:43:40.419Z: Staged package aws-java-sdk-kinesis-1.11.974-kbU_CjbRpkrhhFeoyIMQA81nTYmniJY77w8Smd7PLpY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/aws-java-sdk-kinesis-1.11.974-kbU_CjbRpkrhhFeoyIMQA81nTYmniJY77w8Smd7PLpY.jar' is inaccessible.
Sep 09, 2021 2:43:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-09T14:43:40.528Z: Staged package aws-java-sdk-kms-1.11.974-HWH_mHclw3Wv4_GJ56x62xFDHSPSkZWjeKAlx40XrTk.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/aws-java-sdk-kms-1.11.974-HWH_mHclw3Wv4_GJ56x62xFDHSPSkZWjeKAlx40XrTk.jar' is inaccessible.
Sep 09, 2021 2:43:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-09T14:43:40.577Z: Staged package aws-java-sdk-s3-1.11.974-Qgd0wgaOuI_uyxxQI4Ulljk9bZEq-3JZArQF0zU5yzA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/aws-java-sdk-s3-1.11.974-Qgd0wgaOuI_uyxxQI4Ulljk9bZEq-3JZArQF0zU5yzA.jar' is inaccessible.
Sep 09, 2021 2:43:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-09T14:43:41.026Z: Staged package beam-sdks-java-io-kinesis-2.34.0-SNAPSHOT-l-wlPwuqL3JoShJc8MdSNmXKDFI5maGcuJPHhyiddpA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-kinesis-2.34.0-SNAPSHOT-l-wlPwuqL3JoShJc8MdSNmXKDFI5maGcuJPHhyiddpA.jar' is inaccessible.
Sep 09, 2021 2:43:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-09T14:43:41.108Z: Staged package beam-sdks-java-load-tests-2.34.0-SNAPSHOT-V9yHrJQiSt9kiJZ7EuXMHFTsASG3sfXHAWinjagDbFA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-load-tests-2.34.0-SNAPSHOT-V9yHrJQiSt9kiJZ7EuXMHFTsASG3sfXHAWinjagDbFA.jar' is inaccessible.
Sep 09, 2021 2:43:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-09T14:43:41.504Z: Staged package commons-io-2.6--HfTBGYKwqFC84ZbrfyXHex-1zx0fH-NXS9ROcpzZRM.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/commons-io-2.6--HfTBGYKwqFC84ZbrfyXHex-1zx0fH-NXS9ROcpzZRM.jar' is inaccessible.
Sep 09, 2021 2:43:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-09T14:43:41.587Z: Staged package commons-lang-2.6-UPEbCfh3wpTVbyRGP0fSj5Kc9QRPZIZhwPDPuumi9Jw.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/commons-lang-2.6-UPEbCfh3wpTVbyRGP0fSj5Kc9QRPZIZhwPDPuumi9Jw.jar' is inaccessible.
Sep 09, 2021 2:43:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-09T14:43:43.290Z: Staged package ion-java-1.0.2-DRJ7IFofzgq8KjdXoEF0hlG8ZsFc9MBZusWDOyfUcaU.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/ion-java-1.0.2-DRJ7IFofzgq8KjdXoEF0hlG8ZsFc9MBZusWDOyfUcaU.jar' is inaccessible.
Sep 09, 2021 2:43:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-09T14:43:43.592Z: Staged package jmespath-java-1.11.974-QaUt5Hnw6wd7ULTQ5Oj4j_zpRr-rnMCdfGwiwkSo8-0.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/jmespath-java-1.11.974-QaUt5Hnw6wd7ULTQ5Oj4j_zpRr-rnMCdfGwiwkSo8-0.jar' is inaccessible.
Sep 09, 2021 2:43:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-09-09T14:43:44.782Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Sep 09, 2021 4:00:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-09T16:00:33.427Z: Cancel request is committed for workflow job: 2021-09-09_05_46_25-17409469238405457671.
Sep 09, 2021 4:00:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-09T16:00:33.488Z: Cleaning up.
Sep 09, 2021 4:00:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-09T16:00:33.567Z: Stopping **** pool...
Sep 09, 2021 4:00:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-09T16:00:33.653Z: Stopping **** pool...
Sep 09, 2021 4:02:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-09T16:02:56.343Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Sep 09, 2021 4:02:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-09T16:02:56.384Z: Worker pool stopped.
Sep 09, 2021 4:03:03 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-09-09_05_46_25-17409469238405457671 finished with status CANCELLED.
Load test results for test (ID): b246f9db-4c1d-4f63-b595-0da50ffdb037 and timestamp: 2021-09-09T12:46:17.821000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                  11438.78
dataflow_v2_java11_total_bytes_count             2.74644356E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210909124347
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:770040ac97f1638e45ff50fa8f6d3fea3075431957f1651ae2d17e27f3d98903
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210909124347]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:770040ac97f1638e45ff50fa8f6d3fea3075431957f1651ae2d17e27f3d98903]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210909124347] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:770040ac97f1638e45ff50fa8f6d3fea3075431957f1651ae2d17e27f3d98903])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:770040ac97f1638e45ff50fa8f6d3fea3075431957f1651ae2d17e27f3d98903
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:770040ac97f1638e45ff50fa8f6d3fea3075431957f1651ae2d17e27f3d98903
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:770040ac97f1638e45ff50fa8f6d3fea3075431957f1651ae2d17e27f3d98903].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 49s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date

Publishing build scan...

Publishing failed.

The build scan server appears to be unavailable.
Please check https://status.gradle.com for the latest service status.

If the service is reported as available, please report this problem via https://gradle.com/help/plugin and include the following via copy/paste:

----------
Gradle version: 6.8.3
Plugin version: 3.4.1
Request URL: https://status.gradle.com
Request ID: bcddcb42-1a0f-463a-8e3a-33eafa647e2a
Response status code: 405
Response server type: Varnish
----------

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #83

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/83/display/redirect?page=changes>

Changes:

[heejong] [BEAM-12838] Update artifact local path for DataflowRunner Java

[kwu] [BEAM-12828] Convert UseTestStream tests to use Long instead of Integer

[kwu] Apply SpotlessJava

[kwu] Apply SpotlessJava

[aydar.zaynutdinov] [BEAM-3385] Add requires about `equals()` and `hashMethod()` to

[aydar.zaynutdinov] [BEAM-3385] Changes regarding spotlessApply task

[noreply] Update runners/flink/job-server/flink_job_server.gradle

[Valentyn Tymofieiev] Disable Kafka perf tests.

[heejong] separate into resolveArtifacts method

[heejong] add test

[aydar.zaynutdinov] [BEAM-3385] wrap up equals() and hashCode() methods into links

[Andrew Pilloud] [BEAM-12850] Calcite drops empty Calc now

[Andrew Pilloud] [BEAM-12853] VALUES produces a UNION, window can't be set afterwards

[Andrew Pilloud] [BEAM-12852] Revert BigTable changes, just cast to bigint

[heejong] update

[heejong] fix formatting

[Andrew Pilloud] [BEAM-12851] Map output table names

[Luke Cwik] [BEAM-12802] Define a prefetchable iterator and iterable and utility


------------------------------------------
[...truncated 48.94 KB...]

The push refers to repository [us.gcr.io/apache-beam-testing/java-postcommit-it/java]
7c58cba1e6f9: Preparing
f721ec9f4ab1: Preparing
12acee2498dd: Preparing
667eb8a7414a: Preparing
e20bedb9b97d: Preparing
5e0dc7bb1f8e: Preparing
d0f214a362a5: Preparing
ffc6486f06fc: Preparing
ed0f37db8c66: Preparing
5b8e20977a10: Preparing
3baa4b6ce888: Preparing
116a124110c6: Preparing
3891808a925b: Preparing
d402f4f1b906: Preparing
00ef5416d927: Preparing
8555e663f65b: Preparing
d00da3cd7763: Preparing
4e61e63529c2: Preparing
799760671c38: Preparing
5b8e20977a10: Waiting
3baa4b6ce888: Waiting
116a124110c6: Waiting
3891808a925b: Waiting
ed0f37db8c66: Waiting
d402f4f1b906: Waiting
ffc6486f06fc: Waiting
4e61e63529c2: Waiting
00ef5416d927: Waiting
8555e663f65b: Waiting
5e0dc7bb1f8e: Waiting
799760671c38: Waiting
f721ec9f4ab1: Pushed
12acee2498dd: Pushed
e20bedb9b97d: Pushed
7c58cba1e6f9: Pushed
667eb8a7414a: Pushed
ffc6486f06fc: Pushed
5e0dc7bb1f8e: Pushed
116a124110c6: Pushed
ed0f37db8c66: Pushed
3baa4b6ce888: Pushed
d0f214a362a5: Pushed
3891808a925b: Layer already exists
d402f4f1b906: Layer already exists
00ef5416d927: Layer already exists
8555e663f65b: Layer already exists
d00da3cd7763: Layer already exists
799760671c38: Layer already exists
4e61e63529c2: Layer already exists
5b8e20977a10: Pushed
20210908124343: digest: sha256:54fc37a69a3430f9fff3e5e127a076233e85ed91cb7bcb7250836d1b99770faf size: 4311

> Task :sdks:java:testing:load-tests:run
Sep 08, 2021 12:46:13 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Sep 08, 2021 12:46:14 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 193 files. Enable logging at DEBUG level to see which files will be staged.
Sep 08, 2021 12:46:15 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Sep 08, 2021 12:46:15 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Sep 08, 2021 12:46:18 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 193 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Sep 08, 2021 12:46:18 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 193 files cached, 0 files newly uploaded in 0 seconds
Sep 08, 2021 12:46:18 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Sep 08, 2021 12:46:18 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <105944 bytes, hash 5e6b87ad512cd20e199ed8427dd1b7afe30f1718fe5803dd49d95db51c284267> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-XmuHrVEs0g4ZnthCfdG3r-MPFxj-WAPdSdldtRwoQmc.pb
Sep 08, 2021 12:46:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Sep 08, 2021 12:46:20 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@173a6728, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1a22e0ef, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@67514bdd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3b920bdc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7f5538a1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@72fd8a3c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1f9f381d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@34780cd9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@623ebac7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7c8c70d6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1ab5f08a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6edf29c1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@79a04e5f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@66b59b7d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@53311681, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4437ac07, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3f1ed068, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@753aca85, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2e645fbd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@574a89e2]
Sep 08, 2021 12:46:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Sep 08, 2021 12:46:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Sep 08, 2021 12:46:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Sep 08, 2021 12:46:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Sep 08, 2021 12:46:20 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3fba233d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@427ae189, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@16a9eb2e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@76332405, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@187e5235, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@d1d8e1a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5434e40c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3b48e183, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@514de325, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@30c1da48, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43a65cd8, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3f1ef9d6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@17461db, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3fd9e827, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4e682398, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@670b3ca, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@24a86066, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@54402c04, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5b3bb1f7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58d6b7b9]
Sep 08, 2021 12:46:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Sep 08, 2021 12:46:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Sep 08, 2021 12:46:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Sep 08, 2021 12:46:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Sep 08, 2021 12:46:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Sep 08, 2021 12:46:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Sep 08, 2021 12:46:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Sep 08, 2021 12:46:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Sep 08, 2021 12:46:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Sep 08, 2021 12:46:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Sep 08, 2021 12:46:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Sep 08, 2021 12:46:20 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.34.0-SNAPSHOT
Sep 08, 2021 12:46:21 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-09-08_05_46_20-108916100684351049?project=apache-beam-testing
Sep 08, 2021 12:46:21 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-09-08_05_46_20-108916100684351049
Sep 08, 2021 12:46:21 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-09-08_05_46_20-108916100684351049
Sep 08, 2021 12:46:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-09-08T12:46:27.861Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-09-ikmy. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Sep 08, 2021 12:46:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-08T12:46:31.626Z: Worker configuration: e2-standard-2 in us-central1-a.
Sep 08, 2021 12:46:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-08T12:46:32.439Z: Expanding SplittableParDo operations into optimizable parts.
Sep 08, 2021 12:46:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-08T12:46:32.469Z: Expanding CollectionToSingleton operations into optimizable parts.
Sep 08, 2021 12:46:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-08T12:46:32.526Z: Expanding CoGroupByKey operations into optimizable parts.
Sep 08, 2021 12:46:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-08T12:46:32.585Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Sep 08, 2021 12:46:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-08T12:46:32.617Z: Expanding GroupByKey operations into streaming Read/Write steps
Sep 08, 2021 12:46:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-08T12:46:32.682Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Sep 08, 2021 12:46:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-08T12:46:32.772Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Sep 08, 2021 12:46:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-08T12:46:32.807Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Sep 08, 2021 12:46:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-08T12:46:32.842Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Sep 08, 2021 12:46:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-08T12:46:32.871Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Sep 08, 2021 12:46:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-08T12:46:32.905Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Sep 08, 2021 12:46:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-08T12:46:32.929Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Sep 08, 2021 12:46:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-08T12:46:32.967Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Sep 08, 2021 12:46:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-08T12:46:32.995Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Sep 08, 2021 12:46:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-08T12:46:33.030Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Sep 08, 2021 12:46:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-08T12:46:33.059Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Sep 08, 2021 12:46:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-08T12:46:33.085Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Sep 08, 2021 12:46:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-08T12:46:33.117Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Sep 08, 2021 12:46:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-08T12:46:33.143Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Sep 08, 2021 12:46:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-08T12:46:33.179Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Sep 08, 2021 12:46:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-08T12:46:33.212Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Sep 08, 2021 12:46:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-08T12:46:33.239Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Sep 08, 2021 12:46:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-08T12:46:33.265Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Sep 08, 2021 12:46:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-08T12:46:33.295Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Sep 08, 2021 12:46:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-08T12:46:33.323Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Sep 08, 2021 12:46:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-08T12:46:33.347Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Sep 08, 2021 12:46:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-08T12:46:33.381Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Sep 08, 2021 12:46:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-08T12:46:33.409Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Sep 08, 2021 12:46:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-08T12:46:33.436Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Sep 08, 2021 12:46:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-08T12:46:33.908Z: Starting 5 ****s in us-central1-a...
Sep 08, 2021 12:46:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-08T12:46:55.666Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Sep 08, 2021 12:47:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-08T12:47:17.877Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Sep 08, 2021 12:47:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-08T12:47:46.517Z: Workers have started successfully.
Sep 08, 2021 12:47:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-08T12:47:46.538Z: Workers have started successfully.
Sep 08, 2021 4:00:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-08T16:00:33.472Z: Cancel request is committed for workflow job: 2021-09-08_05_46_20-108916100684351049.
Sep 08, 2021 4:00:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-08T16:00:33.556Z: Cleaning up.
Sep 08, 2021 4:00:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-08T16:00:33.638Z: Stopping **** pool...
Sep 08, 2021 4:00:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-08T16:00:33.692Z: Stopping **** pool...
Sep 08, 2021 4:03:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-08T16:02:58.140Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Sep 08, 2021 4:03:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-08T16:02:58.167Z: Worker pool stopped.
Sep 08, 2021 4:03:04 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-09-08_05_46_20-108916100684351049 finished with status CANCELLED.
Load test results for test (ID): 954abb25-22a5-40a8-a945-5ae32ee958f2 and timestamp: 2021-09-08T12:46:14.695000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                  11490.85
dataflow_v2_java11_total_bytes_count              2.8917746E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210908124343
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:54fc37a69a3430f9fff3e5e127a076233e85ed91cb7bcb7250836d1b99770faf
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210908124343]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:54fc37a69a3430f9fff3e5e127a076233e85ed91cb7bcb7250836d1b99770faf]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210908124343] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:54fc37a69a3430f9fff3e5e127a076233e85ed91cb7bcb7250836d1b99770faf])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:54fc37a69a3430f9fff3e5e127a076233e85ed91cb7bcb7250836d1b99770faf
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:54fc37a69a3430f9fff3e5e127a076233e85ed91cb7bcb7250836d1b99770faf
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:54fc37a69a3430f9fff3e5e127a076233e85ed91cb7bcb7250836d1b99770faf].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 57s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/2xspsb37v2hoa

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #82

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/82/display/redirect>

Changes:


------------------------------------------
[...truncated 49.74 KB...]
799760671c38: Waiting
1e067eb403b7: Waiting
dfc1beb04eb1: Waiting
3891808a925b: Waiting
1acfaaadef40: Waiting
77626a2aec85: Pushed
3e025079073c: Pushed
d6835f8dd6a9: Pushed
f2b591abe6c3: Pushed
ea22f514ce71: Pushed
25640ed3d00f: Pushed
68dbbfa719b2: Pushed
1e067eb403b7: Pushed
3891808a925b: Layer already exists
d402f4f1b906: Layer already exists
00ef5416d927: Layer already exists
dfc1beb04eb1: Pushed
8555e663f65b: Layer already exists
d00da3cd7763: Layer already exists
4e61e63529c2: Layer already exists
799760671c38: Layer already exists
db1f3ca5662f: Pushed
5c79eec53c05: Pushed
1acfaaadef40: Pushed
20210907124333: digest: sha256:abf24d1fd0e73bed63bbe02387ea8fa7d6712137bbdf407582c77a6681b03024 size: 4311

> Task :sdks:java:testing:load-tests:run
Sep 07, 2021 12:45:22 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Sep 07, 2021 12:45:22 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 193 files. Enable logging at DEBUG level to see which files will be staged.
Sep 07, 2021 12:45:23 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Sep 07, 2021 12:45:23 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Sep 07, 2021 12:45:25 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Sep 07, 2021 12:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <113364 bytes, hash 6d09b17456a64f842ff9cf6088f3375b1ad8c93130b2c39a4f929a288d1379d2> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-bQmxdFamT4Qv-c9giPM3WxrYyTEwssOaT5KaKI0TedI.pb
Sep 07, 2021 12:45:27 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 193 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Sep 07, 2021 12:45:27 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 193 files cached, 0 files newly uploaded in 0 seconds
Sep 07, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Sep 07, 2021 12:45:27 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@524270b8, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4acb7ecc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2a4f5433, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6812fa3a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@29149030, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@38b8b6c0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@68868328, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@173a6728, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1a22e0ef, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@67514bdd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3b920bdc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7f5538a1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@72fd8a3c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1f9f381d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@34780cd9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@623ebac7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7c8c70d6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1ab5f08a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6edf29c1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@79a04e5f]
Sep 07, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Sep 07, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Sep 07, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Sep 07, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Sep 07, 2021 12:45:27 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@257e0827, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@22752544, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@21ba2445, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@69d23296, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c3820bb, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@376c7d7d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4784efd9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3fba233d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@427ae189, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@16a9eb2e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@76332405, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@187e5235, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@d1d8e1a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5434e40c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3b48e183, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@514de325, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@30c1da48, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43a65cd8, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3f1ef9d6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@17461db]
Sep 07, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Sep 07, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Sep 07, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Sep 07, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Sep 07, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Sep 07, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Sep 07, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Sep 07, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Sep 07, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Sep 07, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Sep 07, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Sep 07, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.34.0-SNAPSHOT
Sep 07, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-09-07_05_45_28-3843583549148030106?project=apache-beam-testing
Sep 07, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-09-07_05_45_28-3843583549148030106
Sep 07, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-09-07_05_45_28-3843583549148030106
Sep 07, 2021 12:45:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-09-07T12:45:35.237Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-09-jae8. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Sep 07, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-07T12:45:39.299Z: Worker configuration: e2-standard-2 in us-central1-a.
Sep 07, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-07T12:45:40.045Z: Expanding SplittableParDo operations into optimizable parts.
Sep 07, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-07T12:45:40.071Z: Expanding CollectionToSingleton operations into optimizable parts.
Sep 07, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-07T12:45:40.141Z: Expanding CoGroupByKey operations into optimizable parts.
Sep 07, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-07T12:45:40.194Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Sep 07, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-07T12:45:40.225Z: Expanding GroupByKey operations into streaming Read/Write steps
Sep 07, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-07T12:45:40.281Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Sep 07, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-07T12:45:40.414Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Sep 07, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-07T12:45:40.453Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Sep 07, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-07T12:45:40.506Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Sep 07, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-07T12:45:40.539Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Sep 07, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-07T12:45:40.567Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Sep 07, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-07T12:45:40.590Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Sep 07, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-07T12:45:40.616Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Sep 07, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-07T12:45:40.654Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Sep 07, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-07T12:45:40.688Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Sep 07, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-07T12:45:40.721Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Sep 07, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-07T12:45:40.755Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Sep 07, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-07T12:45:40.792Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Sep 07, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-07T12:45:40.824Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Sep 07, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-07T12:45:40.854Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Sep 07, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-07T12:45:40.881Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Sep 07, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-07T12:45:40.936Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Sep 07, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-07T12:45:40.965Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Sep 07, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-07T12:45:40.993Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Sep 07, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-07T12:45:41.023Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Sep 07, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-07T12:45:41.048Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Sep 07, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-07T12:45:41.081Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Sep 07, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-07T12:45:41.113Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Sep 07, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-07T12:45:41.172Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Sep 07, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-07T12:45:41.546Z: Starting 5 ****s in us-central1-a...
Sep 07, 2021 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-07T12:45:52.024Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Sep 07, 2021 12:46:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-07T12:46:25.558Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Sep 07, 2021 12:47:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-07T12:46:59.024Z: Workers have started successfully.
Sep 07, 2021 12:47:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-07T12:46:59.053Z: Workers have started successfully.
Sep 07, 2021 4:00:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-07T16:00:34.195Z: Cancel request is committed for workflow job: 2021-09-07_05_45_28-3843583549148030106.
Sep 07, 2021 4:00:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-07T16:00:34.260Z: Cleaning up.
Sep 07, 2021 4:00:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-07T16:00:34.343Z: Stopping **** pool...
Sep 07, 2021 4:00:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-07T16:00:34.419Z: Stopping **** pool...
Sep 07, 2021 4:02:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-07T16:02:54.429Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Sep 07, 2021 4:02:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-07T16:02:54.457Z: Worker pool stopped.
Sep 07, 2021 4:02:59 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-09-07_05_45_28-3843583549148030106 finished with status CANCELLED.
Load test results for test (ID): 306f9e3d-dd3e-48a9-95d0-771fe7e415f8 and timestamp: 2021-09-07T12:45:22.793000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11556.194
dataflow_v2_java11_total_bytes_count             2.83974741E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210907124333
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:abf24d1fd0e73bed63bbe02387ea8fa7d6712137bbdf407582c77a6681b03024
Deleted: sha256:9d3487198a48809b5738057aa85efdcc2b46a5686b969b819bcdaf4b4de1e34d
Deleted: sha256:965020a8b6ccf745cd3aeaebca837269c6680b51c4c3a7363c5189278961c55d
Deleted: sha256:d657f2e1382e93376f5dce0557d9876bd3b38cfa85c85b8e116ff1ea8ad47063
Deleted: sha256:ea148eb0b05575d7dc740fa116dabc31e1c22018c26d5b1fedb647a5d751facb
Deleted: sha256:038c5a077ce6a50686363bdd319dd7eaa168871c51bd0d2584f57fb2609b1330
Deleted: sha256:b8f52c9b26337e5b26efff1fea8e55493e143e9280f5dce9a2ac94fdb4154a82
Deleted: sha256:251d54469515a426297e617ab34e0baeb2ce8d0f9df3679e736a0143dcf83c4a
Deleted: sha256:92343772d32eed8bf075784bc4aa7b88e3a02463a0fa9b5f8503e868eb619dbc
Deleted: sha256:61ae94f6f2bb53653dc2f6c7173398f47e84c762091311553a430840e292b34f
Deleted: sha256:f83414ab6065847ead0590525ef1b035cd5be884cf85fc3d0c25bf9c4c6c8ede
Deleted: sha256:63e5846a26f22875ecffde52fdc31cec4177ef2b891b0641e642d754db412285
Deleted: sha256:abece5572e6254cbc1e5fb0ba8750fb86556887e4d3b2427ba93b39b74a2ddfe
Deleted: sha256:3996a6612b347e6d25f67a059344d61bd409e52951b08d0ae90b6420de32308b
Deleted: sha256:f3173a81953ffc13a4f1c08efbf9d0d7fe83a77ce261d053d0b6cb28d128698c
Deleted: sha256:b018b878a8c5ee7c0f6610e73419a758ec925c35f2288fd099bce8141e278b49
Deleted: sha256:09154947965657befbc6697a7e0f06bbc5554af5ccade2ebb7997969073fbb41
Deleted: sha256:288aa6a6bf96ba699488d6ba18449c89c848693a16299ada620ff27474370fbe
Deleted: sha256:ede37928ffdf3df6c007ff0019e7aed6aac5b57ffe782322906b0db1cd3217a3
Deleted: sha256:76c26e1a8f52201801c6f98971c8a88907aecfd3ac38981077866ddd054c93af
Deleted: sha256:74ef76bda7342563bc7b91109c8abf6daa2127d7c66ff1b79660b55f8df66a67
Deleted: sha256:08369c834ae2320abc233ebc4a35d712296d6f929fc62ab35966380dfc8a2b04
Deleted: sha256:787d5b1c9ab9cbcc2a02e3a4923c7d5c4df9e7be85e691f19e457a8d0553cf60
Deleted: sha256:a23e7ee4a159b14cb0bdad5c55dfed0b32d24d43d4fb1fa3d87df2a01a96710d
Deleted: sha256:94464e5823c3de25eebd2a229afb187dee4efc81ec051cc2b3fa54e93a414dad
Deleted: sha256:4cc2529a233eaaa19066faa6d7c567bffc4b20a8d228aa0ebf0e16e9c4815d33
Deleted: sha256:9a32b2ddc2db902987f0b67d61c74f64208f089d0cb69e3269d013c6e292eea5
Deleted: sha256:cf869b4c0c512cc2d8a9aa8099d6e0f4fec2b57724ad28bbaf6f0a497c9fe549
Deleted: sha256:b20bb8818dabb839bdad597a93cf2475008402cf784bd707bdf1468719b84a3e
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210907124333]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:abf24d1fd0e73bed63bbe02387ea8fa7d6712137bbdf407582c77a6681b03024]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210907124333] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:abf24d1fd0e73bed63bbe02387ea8fa7d6712137bbdf407582c77a6681b03024])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:abf24d1fd0e73bed63bbe02387ea8fa7d6712137bbdf407582c77a6681b03024
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:abf24d1fd0e73bed63bbe02387ea8fa7d6712137bbdf407582c77a6681b03024
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:abf24d1fd0e73bed63bbe02387ea8fa7d6712137bbdf407582c77a6681b03024].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 45s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/rpn5g4xwexx64

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #81

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/81/display/redirect?page=changes>

Changes:

[noreply] Merge pull request #14927 from [BEAM-12400] MongoDBIO support for update


------------------------------------------
[...truncated 49.24 KB...]
ebada0a227da: Waiting
aaa8d9ad10c8: Pushed
e309fe505ebe: Pushed
c51e4298c0c8: Pushed
b12d9fef893d: Pushed
c3c78ed7b3cc: Pushed
1d3cba141116: Pushed
a4a5e7049b7d: Pushed
81b476dd8847: Pushed
3891808a925b: Layer already exists
d402f4f1b906: Layer already exists
00ef5416d927: Layer already exists
8555e663f65b: Layer already exists
d00da3cd7763: Layer already exists
4e61e63529c2: Layer already exists
799760671c38: Layer already exists
6343c579865e: Pushed
ebada0a227da: Pushed
e89dbc60924f: Pushed
beccdf54c0dd: Pushed
20210906124336: digest: sha256:bca54dd156ecf8ee7804d6bb455f203b6f05d25add4e76f7b9776973911598dd size: 4311

> Task :sdks:java:testing:load-tests:run
Sep 06, 2021 12:45:36 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Sep 06, 2021 12:45:36 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 193 files. Enable logging at DEBUG level to see which files will be staged.
Sep 06, 2021 12:45:37 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Sep 06, 2021 12:45:37 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Sep 06, 2021 12:45:39 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Sep 06, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <113364 bytes, hash 0b8c925cde51f91d707f03a029d22c248597a179b86d36b10b8f0d334cc05628> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-C4ySXN5R-R1wfwOgKdIsJIWXoXm4bTaxC48NM0zAVig.pb
Sep 06, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 193 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Sep 06, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 193 files cached, 0 files newly uploaded in 0 seconds
Sep 06, 2021 12:45:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Sep 06, 2021 12:45:42 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@524270b8, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4acb7ecc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2a4f5433, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6812fa3a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@29149030, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@38b8b6c0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@68868328, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@173a6728, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1a22e0ef, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@67514bdd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3b920bdc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7f5538a1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@72fd8a3c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1f9f381d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@34780cd9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@623ebac7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7c8c70d6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1ab5f08a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6edf29c1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@79a04e5f]
Sep 06, 2021 12:45:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Sep 06, 2021 12:45:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Sep 06, 2021 12:45:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Sep 06, 2021 12:45:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Sep 06, 2021 12:45:42 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@257e0827, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@22752544, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@21ba2445, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@69d23296, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c3820bb, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@376c7d7d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4784efd9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3fba233d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@427ae189, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@16a9eb2e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@76332405, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@187e5235, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@d1d8e1a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5434e40c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3b48e183, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@514de325, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@30c1da48, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43a65cd8, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3f1ef9d6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@17461db]
Sep 06, 2021 12:45:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Sep 06, 2021 12:45:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Sep 06, 2021 12:45:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Sep 06, 2021 12:45:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Sep 06, 2021 12:45:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Sep 06, 2021 12:45:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Sep 06, 2021 12:45:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Sep 06, 2021 12:45:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Sep 06, 2021 12:45:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Sep 06, 2021 12:45:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Sep 06, 2021 12:45:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Sep 06, 2021 12:45:42 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.34.0-SNAPSHOT
Sep 06, 2021 12:45:43 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-09-06_05_45_42-3235823371487490209?project=apache-beam-testing
Sep 06, 2021 12:45:43 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-09-06_05_45_42-3235823371487490209
Sep 06, 2021 12:45:43 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-09-06_05_45_42-3235823371487490209
Sep 06, 2021 12:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-09-06T12:45:50.652Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-09-vkes. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Sep 06, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-06T12:45:55.041Z: Worker configuration: e2-standard-2 in us-central1-a.
Sep 06, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-06T12:45:55.769Z: Expanding SplittableParDo operations into optimizable parts.
Sep 06, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-06T12:45:55.874Z: Expanding CollectionToSingleton operations into optimizable parts.
Sep 06, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-06T12:45:56.040Z: Expanding CoGroupByKey operations into optimizable parts.
Sep 06, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-06T12:45:56.268Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Sep 06, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-06T12:45:56.324Z: Expanding GroupByKey operations into streaming Read/Write steps
Sep 06, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-06T12:45:56.388Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Sep 06, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-06T12:45:56.500Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Sep 06, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-06T12:45:56.534Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Sep 06, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-06T12:45:56.566Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Sep 06, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-06T12:45:56.599Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Sep 06, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-06T12:45:56.633Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Sep 06, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-06T12:45:56.656Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Sep 06, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-06T12:45:56.685Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Sep 06, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-06T12:45:56.722Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Sep 06, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-06T12:45:56.752Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Sep 06, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-06T12:45:56.783Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Sep 06, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-06T12:45:56.806Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Sep 06, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-06T12:45:56.832Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Sep 06, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-06T12:45:56.864Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Sep 06, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-06T12:45:56.898Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Sep 06, 2021 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-06T12:45:56.933Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Sep 06, 2021 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-06T12:45:56.971Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Sep 06, 2021 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-06T12:45:57.001Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Sep 06, 2021 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-06T12:45:57.029Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Sep 06, 2021 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-06T12:45:57.054Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Sep 06, 2021 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-06T12:45:57.088Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Sep 06, 2021 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-06T12:45:57.111Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Sep 06, 2021 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-06T12:45:57.143Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Sep 06, 2021 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-06T12:45:57.166Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Sep 06, 2021 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-06T12:45:57.572Z: Starting 5 ****s in us-central1-a...
Sep 06, 2021 12:46:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-06T12:46:04.297Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Sep 06, 2021 12:46:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-06T12:46:48.166Z: Autoscaling: Raised the number of ****s to 4 so that the pipeline can catch up with its backlog and keep up with its input rate.
Sep 06, 2021 12:46:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-06T12:46:48.221Z: Resized **** pool to 4, though goal was 5.  This could be a quota issue.
Sep 06, 2021 12:47:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-06T12:46:58.461Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Sep 06, 2021 12:47:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-06T12:47:23.352Z: Workers have started successfully.
Sep 06, 2021 12:47:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-06T12:47:23.381Z: Workers have started successfully.
Sep 06, 2021 4:00:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-06T16:00:28.712Z: Cancel request is committed for workflow job: 2021-09-06_05_45_42-3235823371487490209.
Sep 06, 2021 4:00:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-06T16:00:28.785Z: Cleaning up.
Sep 06, 2021 4:00:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-06T16:00:28.856Z: Stopping **** pool...
Sep 06, 2021 4:00:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-06T16:00:28.900Z: Stopping **** pool...
Sep 06, 2021 4:02:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-06T16:02:48.589Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Sep 06, 2021 4:02:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-06T16:02:48.631Z: Worker pool stopped.
Sep 06, 2021 4:02:53 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-09-06_05_45_42-3235823371487490209 finished with status CANCELLED.
Load test results for test (ID): 1eb18644-6732-476d-a11f-60297b36c5fa and timestamp: 2021-09-06T12:45:36.993000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11535.266
dataflow_v2_java11_total_bytes_count             3.22451797E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210906124336
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:bca54dd156ecf8ee7804d6bb455f203b6f05d25add4e76f7b9776973911598dd
Deleted: sha256:2e4bb998a8ba7193368589c23c6701cb3b549d7e1c843dd08851a76db9982a79
Deleted: sha256:5109cf0af2b37fc73aeb71f1fadab498a745ef1beb7a6699e7b0855330b81a05
Deleted: sha256:2acba81053e879bc89c99c2d6c049a9376b881d30b3cf44f74049625fece44eb
Deleted: sha256:0cc68c9212a7c56dcc0ce2c0c76e5ac2c231914478216316ae07d76e07d6a3e1
Deleted: sha256:f25e34606e28027cd66c8bc7413ecf8a13903b4094e2701a2ddf6f8606aabdca
Deleted: sha256:60e5367c3822e8ffd5b034c5a2a246107d8ad51cfc8bd7582a5269f90b37e5ed
Deleted: sha256:102bb05409b9fdf9b66958e7b70d4ab94f1654d8dca2421264bdf61571df2789
Deleted: sha256:4d8b130a736b0f32b30d8cc432a195338b9f84fd89168d1b5e159cb7f7323cb9
Deleted: sha256:c0097047afb0ef68c6152ed9ef323268f948b6cb26656360eebab5aa72121591
Deleted: sha256:d8053af259e7780a66618c64e5007112656d0e4ce73f48cdb41a10db374fc34a
Deleted: sha256:c1dc70c7d1154d3f0db0a71b8be2c531f5d30cf58d5e977c239276b81106803b
Deleted: sha256:3ff5bbc31b07881a3484b50ed7707ce0e48eb7221bca537081356a634572852e
Deleted: sha256:5394fbd51b8008464263577e10d921a977b238c6768276a76d4d7256ee8f8500
Deleted: sha256:41f415991b02df13de08cf7ad7862132dfd73e0ae9da7f874c8dfca0e7ba99b9
Deleted: sha256:4bc1952512e5ce4657a948242d658a97527ee9e6197cbe200a7e3d2fe19d4e1a
Deleted: sha256:f3fe81cd33bcd38469df5c1dc69324b62e9d761554dcc9f5744cdd38a72ac6b0
Deleted: sha256:b45a0723905119481c67ac8ae94f095444a6871a6473889cc7e6b089f8a414e6
Deleted: sha256:e4bbb0778c1486fdf205fa5623bde73702234a6948baf841574ed1853e2fd7f8
Deleted: sha256:a6bdeab852f1ace969e067807d6f9cc6368fbba29316b65851f9fda2517def38
Deleted: sha256:c38d5828a0021405f7898dbeddf7fba4786f6e2eda28b5ac3bf4bab48c34500c
Deleted: sha256:31e7baf37107def1b2581773419c99db5795bf704eb3c224b1039395572ab425
Deleted: sha256:bc2d8c2ceaa8e97a2bcf3f382d35214c7554f41f04c29a4607e0fb9cd41d149c
Deleted: sha256:027d1946b3eecb14e056e0a7d933caba913198c453a23636596533fafaec62a8
Deleted: sha256:e68e7178efb6059585d395159fd84fe86ed1d05b3c69c4d7252328ae39e6ec5c
Deleted: sha256:a1e4e60796dc109b9c8e6c861a2686ffa62d0c05150839223327b2e8a1d0ac89
Deleted: sha256:a4c201718c676716afd7938dc2f0e826b9fb6c0ef10435fa214d239e0695c333
Deleted: sha256:3fd0fa4daa974077e3c4aa61bdf2661481e05e46e5b7fc42983338549a6a5a69
Deleted: sha256:ba6c904dcffa1e13ff88e3d608f24f18bee92a60f1cb460c10c8b28b95f14f15
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210906124336]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:bca54dd156ecf8ee7804d6bb455f203b6f05d25add4e76f7b9776973911598dd]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210906124336] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:bca54dd156ecf8ee7804d6bb455f203b6f05d25add4e76f7b9776973911598dd])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:bca54dd156ecf8ee7804d6bb455f203b6f05d25add4e76f7b9776973911598dd
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:bca54dd156ecf8ee7804d6bb455f203b6f05d25add4e76f7b9776973911598dd
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:bca54dd156ecf8ee7804d6bb455f203b6f05d25add4e76f7b9776973911598dd].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 36s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/7s6zvnl2mnol4

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 - Build # 80 - Aborted!

Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 - Build # 80 - Aborted:

Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/80/ to view the results.

Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #79

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/79/display/redirect>

Changes:


------------------------------------------
[...truncated 49.54 KB...]
799760671c38: Preparing
0a38f9dbed15: Waiting
1e6631f24392: Waiting
6cb24ad91c7a: Waiting
6e757c144b51: Waiting
b7287b909ddb: Waiting
327d944e586f: Waiting
3891808a925b: Waiting
d00da3cd7763: Waiting
4e61e63529c2: Waiting
d402f4f1b906: Waiting
8555e663f65b: Waiting
799760671c38: Waiting
00ef5416d927: Waiting
9c9319f6ac52: Waiting
12b68a168729: Pushed
b320a683f0b3: Pushed
f3e85fdd2218: Pushed
3350b451b3f4: Pushed
521d147e1711: Pushed
9c9319f6ac52: Pushed
b7287b909ddb: Pushed
1e6631f24392: Pushed
3891808a925b: Layer already exists
d402f4f1b906: Layer already exists
0a38f9dbed15: Pushed
00ef5416d927: Layer already exists
8555e663f65b: Layer already exists
d00da3cd7763: Layer already exists
4e61e63529c2: Layer already exists
799760671c38: Layer already exists
6cb24ad91c7a: Pushed
6e757c144b51: Pushed
327d944e586f: Pushed
20210904124334: digest: sha256:0bef8b98fb24f275fb9c1157ba1474668e10ebf39d19978394346fee5f45bbae size: 4311

> Task :sdks:java:testing:load-tests:run
Sep 04, 2021 12:45:24 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Sep 04, 2021 12:45:25 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 191 files. Enable logging at DEBUG level to see which files will be staged.
Sep 04, 2021 12:45:25 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Sep 04, 2021 12:45:25 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Sep 04, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Sep 04, 2021 12:45:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <112622 bytes, hash 7dc843c7b9aee4c76fe797bd5178fac51c9922537a21204c63546713b1110d40> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-fchDx7mu5Mdv55e9UXj6xRyZIlN6ISBMY1RnE7ERDUA.pb
Sep 04, 2021 12:45:29 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 191 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Sep 04, 2021 12:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 191 files cached, 0 files newly uploaded in 0 seconds
Sep 04, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Sep 04, 2021 12:45:30 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6812fa3a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@29149030, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@38b8b6c0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@68868328, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@173a6728, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1a22e0ef, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@67514bdd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3b920bdc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7f5538a1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@72fd8a3c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1f9f381d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@34780cd9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@623ebac7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7c8c70d6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1ab5f08a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6edf29c1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@79a04e5f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@66b59b7d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@53311681, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4437ac07]
Sep 04, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Sep 04, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Sep 04, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Sep 04, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Sep 04, 2021 12:45:30 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@697a34af, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@70211df5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4c5228e7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@38e7ed69, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@806996, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@78b612c6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@257e0827, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@22752544, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@21ba2445, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@69d23296, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c3820bb, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@376c7d7d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4784efd9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3fba233d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@427ae189, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@16a9eb2e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@76332405, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@187e5235, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@d1d8e1a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5434e40c]
Sep 04, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Sep 04, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Sep 04, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Sep 04, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Sep 04, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Sep 04, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Sep 04, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Sep 04, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Sep 04, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Sep 04, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Sep 04, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Sep 04, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.34.0-SNAPSHOT
Sep 04, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-09-04_05_45_30-7711831206780935778?project=apache-beam-testing
Sep 04, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-09-04_05_45_30-7711831206780935778
Sep 04, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-09-04_05_45_30-7711831206780935778
Sep 04, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-09-04T12:45:37.659Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-09-cg3c. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Sep 04, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-04T12:45:41.928Z: Worker configuration: e2-standard-2 in us-central1-a.
Sep 04, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-04T12:45:42.583Z: Expanding SplittableParDo operations into optimizable parts.
Sep 04, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-04T12:45:42.610Z: Expanding CollectionToSingleton operations into optimizable parts.
Sep 04, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-04T12:45:42.651Z: Expanding CoGroupByKey operations into optimizable parts.
Sep 04, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-04T12:45:42.697Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Sep 04, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-04T12:45:42.717Z: Expanding GroupByKey operations into streaming Read/Write steps
Sep 04, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-04T12:45:42.762Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Sep 04, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-04T12:45:42.835Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Sep 04, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-04T12:45:42.860Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Sep 04, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-04T12:45:42.883Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Sep 04, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-04T12:45:42.901Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Sep 04, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-04T12:45:42.925Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Sep 04, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-04T12:45:42.948Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Sep 04, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-04T12:45:42.970Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Sep 04, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-04T12:45:42.998Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Sep 04, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-04T12:45:43.014Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Sep 04, 2021 12:45:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-04T12:45:43.036Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Sep 04, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-04T12:45:43.060Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Sep 04, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-04T12:45:43.084Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Sep 04, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-04T12:45:43.108Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Sep 04, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-04T12:45:43.129Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Sep 04, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-04T12:45:43.145Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Sep 04, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-04T12:45:43.167Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Sep 04, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-04T12:45:43.200Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Sep 04, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-04T12:45:43.222Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Sep 04, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-04T12:45:43.247Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Sep 04, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-04T12:45:43.269Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Sep 04, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-04T12:45:43.293Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Sep 04, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-04T12:45:43.308Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Sep 04, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-04T12:45:43.330Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Sep 04, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-04T12:45:43.703Z: Starting 5 ****s in us-central1-a...
Sep 04, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-04T12:46:01.751Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Sep 04, 2021 12:46:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-04T12:46:30.313Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Sep 04, 2021 12:47:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-04T12:46:59.280Z: Workers have started successfully.
Sep 04, 2021 12:47:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-04T12:46:59.330Z: Workers have started successfully.
Sep 04, 2021 2:27:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-04T14:27:44.761Z: Staged package cldrdata-k07I6K9W3X5KTQbcDIEsqM0LXyM18f0eR6IaJw-P_xk.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/cldrdata-k07I6K9W3X5KTQbcDIEsqM0LXyM18f0eR6IaJw-P_xk.jar' is inaccessible.
Sep 04, 2021 2:27:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-04T14:27:45.073Z: Staged package dnsns-RGhCDg3GVOQVC2r6ka2N0hmI4eqQH6VobuoAnQ74MnE.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/dnsns-RGhCDg3GVOQVC2r6ka2N0hmI4eqQH6VobuoAnQ74MnE.jar' is inaccessible.
Sep 04, 2021 2:27:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-04T14:27:45.162Z: Staged package flatbuffers-java-1.12.0-P4wIi03QSphYch8uFiUIyU2w3Yb5YeMG7mPvLtqHG_c.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/flatbuffers-java-1.12.0-P4wIi03QSphYch8uFiUIyU2w3Yb5YeMG7mPvLtqHG_c.jar' is inaccessible.
Sep 04, 2021 2:27:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-04T14:27:46.414Z: Staged package icedtea-sound-Bdoi2wYa757-fzq5vconCy4SSQ22ZaOq7yuC98fKPs8.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/icedtea-sound-Bdoi2wYa757-fzq5vconCy4SSQ22ZaOq7yuC98fKPs8.jar' is inaccessible.
Sep 04, 2021 2:27:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-04T14:27:46.481Z: Staged package jaccess-CMbK-IOdQPLKHEqCuDnE-yBk-VpbtVT-hgjbHRUGO78.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/jaccess-CMbK-IOdQPLKHEqCuDnE-yBk-VpbtVT-hgjbHRUGO78.jar' is inaccessible.
Sep 04, 2021 2:27:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-04T14:27:46.742Z: Staged package joda-time-2.10.10-3Y58khhaZ40be5M_MSCbYgPI_6kemIBHWhvgNGuWF-M.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/joda-time-2.10.10-3Y58khhaZ40be5M_MSCbYgPI_6kemIBHWhvgNGuWF-M.jar' is inaccessible.
Sep 04, 2021 2:27:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-04T14:27:46.947Z: Staged package localedata-ae5Z0L6ak4922fztWeWy7ajiWXdG3ubNrwerJRFoFj0.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/localedata-ae5Z0L6ak4922fztWeWy7ajiWXdG3ubNrwerJRFoFj0.jar' is inaccessible.
Sep 04, 2021 2:27:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-04T14:27:47.031Z: Staged package nashorn-XHtz_UehGpYcLTOrATrTnMNVUgEVa_ttoWkPxnVfqTo.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/nashorn-XHtz_UehGpYcLTOrATrTnMNVUgEVa_ttoWkPxnVfqTo.jar' is inaccessible.
Sep 04, 2021 2:27:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-09-04T14:27:47.912Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Sep 04, 2021 4:00:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-04T16:00:26.864Z: Cancel request is committed for workflow job: 2021-09-04_05_45_30-7711831206780935778.
Sep 04, 2021 4:00:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-04T16:00:26.891Z: Cleaning up.
Sep 04, 2021 4:00:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-04T16:00:26.960Z: Stopping **** pool...
Sep 04, 2021 4:00:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-04T16:00:27.007Z: Stopping **** pool...
Sep 04, 2021 4:02:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-04T16:02:48.233Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Sep 04, 2021 4:02:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-04T16:02:48.261Z: Worker pool stopped.
Sep 04, 2021 4:02:53 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-09-04_05_45_30-7711831206780935778 finished with status CANCELLED.
Load test results for test (ID): 2735176e-8870-4568-874c-a302a4e1cca0 and timestamp: 2021-09-04T12:45:25.508000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11549.099
dataflow_v2_java11_total_bytes_count             3.23299942E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210904124334
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:0bef8b98fb24f275fb9c1157ba1474668e10ebf39d19978394346fee5f45bbae
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210904124334]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:0bef8b98fb24f275fb9c1157ba1474668e10ebf39d19978394346fee5f45bbae]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210904124334] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:0bef8b98fb24f275fb9c1157ba1474668e10ebf39d19978394346fee5f45bbae])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:0bef8b98fb24f275fb9c1157ba1474668e10ebf39d19978394346fee5f45bbae
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:0bef8b98fb24f275fb9c1157ba1474668e10ebf39d19978394346fee5f45bbae
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:0bef8b98fb24f275fb9c1157ba1474668e10ebf39d19978394346fee5f45bbae].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 36s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/7idrvdcwxghlq

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #78

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/78/display/redirect?page=changes>

Changes:

[Steve Niemitz] [BEAM-12767] Add another unit test for PipelineOptions deserialization

[aydar.zaynutdinov] [BEAM-11873] Add support for writes with returning values in JdbcIO

[Andrew Pilloud] s/org.apache.beam.vendor.calcite.v1_20_0/org.apache.beam.vendor.calcite.v1_26_0/g

[Andrew Pilloud] [BEAM-9379] Update to vendored Calcite to 1.26.0

[Andrew Pilloud] Fix flattened rows

[Andrew Pilloud] Fix DDL

[Andrew Pilloud] Handle BeamRelNode in RelSubset

[Andrew Pilloud] Fix BeamIOPushDown

[Andrew Pilloud] [BEAM-9190] Update BeamBigQuerySqlDialect

[Andrew Pilloud] Remap IN to Search

[Andrew Pilloud] [BEAM-9379] Use byte[] instead of ByteString for (VAR)BINARY in UDFs.

[Andrew Pilloud] [BEAM-9379] Update UDF NULL type mismatch test since there is stricter

[Andrew Pilloud] Fix ZetaSQL window function mapping

[Andrew Pilloud] Fix Bigtable tests that depend on SQL types

[Andrew Pilloud] Workaround CALCITE-4759 in JoinPushThroughJoinRule

[Andrew Pilloud] Disable nested bytes tests, sorry!

[Andrew Pilloud] SqlLine is rotting, Just CAST types for now

[Andrew Pilloud] Update CHANGES.md

[Andrew Pilloud] Up spotbug stack size

[Andrew Pilloud] Fix BeamMatchRel copy

[Andrew Pilloud] partitionKey everywhere

[Andrew Pilloud] Make it functional

[Andrew Pilloud] Update CreateFunction

[Andrew Pilloud] No tpcds dependency

[Andrew Pilloud] Fix default time types

[Ankur Goenka] fixing release date

[Andrew Pilloud] [BEAM-12680] Calcite SqlTransform no longer experimental

[noreply] [BEAM-12823] TestStream Support in Samza Portable Runner (#15421)


------------------------------------------
[...truncated 48.20 KB...]
8555e663f65b: Waiting
d00da3cd7763: Waiting
799760671c38: Preparing
799760671c38: Waiting
d402f4f1b906: Waiting
00ef5416d927: Waiting
3891808a925b: Waiting
cbd1847397a6: Pushed
93d8904b4346: Pushed
dcc60227efea: Pushed
b24966251fc9: Pushed
ce552a63504e: Pushed
90a3717dea9a: Pushed
ae7896080c89: Pushed
a736df8d5d8a: Pushed
3891808a925b: Layer already exists
d402f4f1b906: Layer already exists
00ef5416d927: Layer already exists
8555e663f65b: Layer already exists
d00da3cd7763: Layer already exists
4e61e63529c2: Layer already exists
799760671c38: Layer already exists
d459ac429b69: Pushed
efe909e58895: Pushed
7ef68bd23097: Pushed
7c8d8b857bee: Pushed
20210903124334: digest: sha256:32234880a5257393275e5a13bbd8b69d5627be9b45e07ce70f389530435c806d size: 4311

> Task :sdks:java:testing:load-tests:run
Sep 03, 2021 12:46:06 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Sep 03, 2021 12:46:06 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 191 files. Enable logging at DEBUG level to see which files will be staged.
Sep 03, 2021 12:46:07 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Sep 03, 2021 12:46:07 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Sep 03, 2021 12:46:10 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Sep 03, 2021 12:46:10 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <112622 bytes, hash af91d0cf048a02657fed8ce508db571cdf75562e10530f09e7961d362ee2130c> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-r5HQzwSKAmV_7YzlCNtXHN91Vi4QUw8J55YdNi7iEww.pb
Sep 03, 2021 12:46:12 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 191 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Sep 03, 2021 12:46:12 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 191 files cached, 0 files newly uploaded in 0 seconds
Sep 03, 2021 12:46:12 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Sep 03, 2021 12:46:12 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7afc4db9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1a1f5f71, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@63884e4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@524270b8, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4acb7ecc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2a4f5433, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6812fa3a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@29149030, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@38b8b6c0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@68868328, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@173a6728, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1a22e0ef, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@67514bdd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3b920bdc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7f5538a1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@72fd8a3c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1f9f381d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@34780cd9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@623ebac7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7c8c70d6]
Sep 03, 2021 12:46:12 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Sep 03, 2021 12:46:12 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Sep 03, 2021 12:46:12 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Sep 03, 2021 12:46:12 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Sep 03, 2021 12:46:12 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@75120e58, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@48976e6d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2a367e93, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7f6874f2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1a6dc589, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@697a34af, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@70211df5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4c5228e7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@38e7ed69, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@806996, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@78b612c6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@257e0827, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@22752544, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@21ba2445, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@69d23296, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c3820bb, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@376c7d7d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4784efd9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3fba233d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@427ae189]
Sep 03, 2021 12:46:12 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Sep 03, 2021 12:46:12 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Sep 03, 2021 12:46:12 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Sep 03, 2021 12:46:12 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Sep 03, 2021 12:46:12 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Sep 03, 2021 12:46:12 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Sep 03, 2021 12:46:12 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Sep 03, 2021 12:46:12 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Sep 03, 2021 12:46:12 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Sep 03, 2021 12:46:12 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Sep 03, 2021 12:46:12 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Sep 03, 2021 12:46:12 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.34.0-SNAPSHOT
Sep 03, 2021 12:46:13 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-09-03_05_46_12-16976215867526938241?project=apache-beam-testing
Sep 03, 2021 12:46:13 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-09-03_05_46_12-16976215867526938241
Sep 03, 2021 12:46:13 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-09-03_05_46_12-16976215867526938241
Sep 03, 2021 12:46:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-09-03T12:46:24.131Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-09-2ohq. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Sep 03, 2021 12:46:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-03T12:46:28.548Z: Worker configuration: e2-standard-2 in us-central1-a.
Sep 03, 2021 12:46:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-03T12:46:29.337Z: Expanding SplittableParDo operations into optimizable parts.
Sep 03, 2021 12:46:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-03T12:46:29.356Z: Expanding CollectionToSingleton operations into optimizable parts.
Sep 03, 2021 12:46:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-03T12:46:29.427Z: Expanding CoGroupByKey operations into optimizable parts.
Sep 03, 2021 12:46:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-03T12:46:29.499Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Sep 03, 2021 12:46:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-03T12:46:29.521Z: Expanding GroupByKey operations into streaming Read/Write steps
Sep 03, 2021 12:46:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-03T12:46:29.575Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Sep 03, 2021 12:46:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-03T12:46:29.687Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Sep 03, 2021 12:46:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-03T12:46:29.722Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Sep 03, 2021 12:46:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-03T12:46:29.754Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Sep 03, 2021 12:46:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-03T12:46:29.782Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Sep 03, 2021 12:46:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-03T12:46:29.807Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Sep 03, 2021 12:46:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-03T12:46:29.832Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Sep 03, 2021 12:46:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-03T12:46:29.864Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Sep 03, 2021 12:46:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-03T12:46:29.889Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Sep 03, 2021 12:46:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-03T12:46:29.932Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Sep 03, 2021 12:46:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-03T12:46:29.948Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Sep 03, 2021 12:46:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-03T12:46:29.974Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Sep 03, 2021 12:46:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-03T12:46:30.008Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Sep 03, 2021 12:46:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-03T12:46:30.042Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Sep 03, 2021 12:46:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-03T12:46:30.065Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Sep 03, 2021 12:46:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-03T12:46:30.097Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Sep 03, 2021 12:46:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-03T12:46:30.128Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Sep 03, 2021 12:46:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-03T12:46:30.164Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Sep 03, 2021 12:46:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-03T12:46:30.194Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Sep 03, 2021 12:46:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-03T12:46:30.217Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Sep 03, 2021 12:46:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-03T12:46:30.252Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Sep 03, 2021 12:46:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-03T12:46:30.282Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Sep 03, 2021 12:46:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-03T12:46:30.312Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Sep 03, 2021 12:46:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-03T12:46:30.340Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Sep 03, 2021 12:46:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-03T12:46:30.712Z: Starting 5 ****s in us-central1-a...
Sep 03, 2021 12:46:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-03T12:46:58.600Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Sep 03, 2021 12:47:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-03T12:47:14.442Z: Autoscaling: Raised the number of ****s to 1 so that the pipeline can catch up with its backlog and keep up with its input rate.
Sep 03, 2021 12:47:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-03T12:47:14.478Z: Resized **** pool to 1, though goal was 5.  This could be a quota issue.
Sep 03, 2021 12:47:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-03T12:47:24.748Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Sep 03, 2021 12:47:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-03T12:47:52.761Z: Workers have started successfully.
Sep 03, 2021 12:47:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-03T12:47:52.796Z: Workers have started successfully.
Sep 03, 2021 2:13:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-03T14:13:31.587Z: Staged package beam-vendor-grpc-1_36_0-0.2-cBdYf-JPXEZ1XnZqk0m4-VlBsCKU_yOoV0qLHUFCl2c.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-grpc-1_36_0-0.2-cBdYf-JPXEZ1XnZqk0m4-VlBsCKU_yOoV0qLHUFCl2c.jar' is inaccessible.
Sep 03, 2021 2:13:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-09-03T14:13:34.633Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Sep 03, 2021 2:16:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-09-03T14:16:34.525Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Sep 03, 2021 2:19:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-03T14:19:31.531Z: Staged package beam-vendor-grpc-1_36_0-0.2-cBdYf-JPXEZ1XnZqk0m4-VlBsCKU_yOoV0qLHUFCl2c.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-grpc-1_36_0-0.2-cBdYf-JPXEZ1XnZqk0m4-VlBsCKU_yOoV0qLHUFCl2c.jar' is inaccessible.
Sep 03, 2021 2:19:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-09-03T14:19:34.439Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Sep 03, 2021 2:22:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-09-03T14:22:34.194Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Sep 03, 2021 2:25:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-03T14:25:31.835Z: Staged package beam-vendor-grpc-1_36_0-0.2-cBdYf-JPXEZ1XnZqk0m4-VlBsCKU_yOoV0qLHUFCl2c.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-grpc-1_36_0-0.2-cBdYf-JPXEZ1XnZqk0m4-VlBsCKU_yOoV0qLHUFCl2c.jar' is inaccessible.
Sep 03, 2021 2:25:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-09-03T14:25:34.477Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Sep 03, 2021 2:28:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-09-03T14:28:34.175Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Sep 03, 2021 2:31:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-03T14:31:31.514Z: Staged package beam-vendor-grpc-1_36_0-0.2-cBdYf-JPXEZ1XnZqk0m4-VlBsCKU_yOoV0qLHUFCl2c.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-grpc-1_36_0-0.2-cBdYf-JPXEZ1XnZqk0m4-VlBsCKU_yOoV0qLHUFCl2c.jar' is inaccessible.
Sep 03, 2021 2:31:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-09-03T14:31:34.276Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Sep 03, 2021 4:00:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-03T16:00:38.701Z: Cancel request is committed for workflow job: 2021-09-03_05_46_12-16976215867526938241.
Sep 03, 2021 4:00:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-03T16:00:38.744Z: Cleaning up.
Sep 03, 2021 4:00:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-03T16:00:38.809Z: Stopping **** pool...
Sep 03, 2021 4:00:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-03T16:00:38.859Z: Stopping **** pool...
Sep 03, 2021 4:03:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-03T16:03:12.716Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Sep 03, 2021 4:03:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-03T16:03:12.742Z: Worker pool stopped.
Sep 03, 2021 4:03:18 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-09-03_05_46_12-16976215867526938241 finished with status CANCELLED.
Load test results for test (ID): bcf151dc-c01a-4d95-9d1f-cb49973589ba and timestamp: 2021-09-03T12:46:07.243000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11513.842
dataflow_v2_java11_total_bytes_count              2.5381647E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210903124334
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:32234880a5257393275e5a13bbd8b69d5627be9b45e07ce70f389530435c806d
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210903124334]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:32234880a5257393275e5a13bbd8b69d5627be9b45e07ce70f389530435c806d]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210903124334] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:32234880a5257393275e5a13bbd8b69d5627be9b45e07ce70f389530435c806d])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:32234880a5257393275e5a13bbd8b69d5627be9b45e07ce70f389530435c806d
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:32234880a5257393275e5a13bbd8b69d5627be9b45e07ce70f389530435c806d
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:32234880a5257393275e5a13bbd8b69d5627be9b45e07ce70f389530435c806d].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 20m 3s
101 actionable tasks: 72 executed, 27 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/rh53tfhycxrzw

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #77

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/77/display/redirect?page=changes>

Changes:

[baeminbo] Fix apiclient_test unittest not to fail with no credentials

[noreply] [BEAM-10913] - Adding new Grafana dashboard to monitor GAs post-commit

[aromanenko.dev] Remove a template line

[Steve Niemitz] [BEAM-12767] Fix handling display data in pipeline serialization

[kawaigin] Updated Linux golden screenshots for notebook integration tests.


------------------------------------------
[...truncated 49.75 KB...]
05103deb4558: Waiting
062f9311b72d: Waiting
a881cfa23a78: Waiting
9889ce9dc2b0: Waiting
1de5678f791d: Waiting
c953594230a2: Pushed
8f7a95a2fe9b: Pushed
dfb1d85d1b56: Pushed
1de5678f791d: Pushed
575b7bedee3e: Pushed
f1177be33f24: Pushed
2f863c88b2af: Pushed
4f6c106ebd53: Pushed
db15c90c0fb2: Layer already exists
80013b6a5075: Pushed
f0a78adddb27: Pushed
a62f4b6a21e8: Layer already exists
062f9311b72d: Layer already exists
9889ce9dc2b0: Layer already exists
21b17a30443e: Layer already exists
05103deb4558: Layer already exists
a881cfa23a78: Layer already exists
839df4029537: Pushed
e5637c6a3547: Pushed
20210902124342: digest: sha256:e4d0fb396672406793530cd45982f3abe648de9bf7206620c67ca72de29fcf16 size: 4311

> Task :sdks:java:testing:load-tests:run
Sep 02, 2021 12:46:37 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Sep 02, 2021 12:46:37 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 191 files. Enable logging at DEBUG level to see which files will be staged.
Sep 02, 2021 12:46:38 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Sep 02, 2021 12:46:38 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Sep 02, 2021 12:46:40 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Sep 02, 2021 12:46:40 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <112622 bytes, hash 50600cd8eae0c780183c9855075e877a6c0e0ed62c0d9a38aaf04d86aeb9cd82> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-UGAM2Orgx4AYPJhVB16HemwODtYsDZo4qvBNhq65zYI.pb
Sep 02, 2021 12:46:42 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 191 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Sep 02, 2021 12:46:42 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 191 files cached, 0 files newly uploaded in 0 seconds
Sep 02, 2021 12:46:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Sep 02, 2021 12:46:42 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1f346ad2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@46a145ba, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7ab34619, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@ae2db25, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@363c4251, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7afc4db9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1a1f5f71, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@63884e4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@524270b8, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4acb7ecc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2a4f5433, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6812fa3a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@29149030, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@38b8b6c0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@68868328, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@173a6728, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1a22e0ef, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@67514bdd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3b920bdc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7f5538a1]
Sep 02, 2021 12:46:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Sep 02, 2021 12:46:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Sep 02, 2021 12:46:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Sep 02, 2021 12:46:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Sep 02, 2021 12:46:42 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7d78f3d5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@a4b5ce3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@f5b6e78, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4b4eced1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@71926a36, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@216e9ca3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@75120e58, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@48976e6d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2a367e93, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7f6874f2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1a6dc589, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@697a34af, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@70211df5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4c5228e7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@38e7ed69, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@806996, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@78b612c6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@257e0827, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@22752544, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@21ba2445]
Sep 02, 2021 12:46:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Sep 02, 2021 12:46:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Sep 02, 2021 12:46:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Sep 02, 2021 12:46:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Sep 02, 2021 12:46:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Sep 02, 2021 12:46:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Sep 02, 2021 12:46:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Sep 02, 2021 12:46:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Sep 02, 2021 12:46:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Sep 02, 2021 12:46:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Sep 02, 2021 12:46:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Sep 02, 2021 12:46:43 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.34.0-SNAPSHOT
Sep 02, 2021 12:46:43 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-09-02_05_46_43-16590442448644662931?project=apache-beam-testing
Sep 02, 2021 12:46:43 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-09-02_05_46_43-16590442448644662931
Sep 02, 2021 12:46:43 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-09-02_05_46_43-16590442448644662931
Sep 02, 2021 12:46:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-09-02T12:46:52.367Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-09-zg4v. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Sep 02, 2021 12:46:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-02T12:46:57.280Z: Worker configuration: e2-standard-2 in us-central1-a.
Sep 02, 2021 12:46:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-02T12:46:57.991Z: Expanding SplittableParDo operations into optimizable parts.
Sep 02, 2021 12:46:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-02T12:46:58.022Z: Expanding CollectionToSingleton operations into optimizable parts.
Sep 02, 2021 12:46:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-02T12:46:58.088Z: Expanding CoGroupByKey operations into optimizable parts.
Sep 02, 2021 12:46:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-02T12:46:58.169Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Sep 02, 2021 12:46:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-02T12:46:58.205Z: Expanding GroupByKey operations into streaming Read/Write steps
Sep 02, 2021 12:46:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-02T12:46:58.263Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Sep 02, 2021 12:46:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-02T12:46:58.353Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Sep 02, 2021 12:46:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-02T12:46:58.404Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Sep 02, 2021 12:46:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-02T12:46:58.451Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Sep 02, 2021 12:46:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-02T12:46:58.484Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Sep 02, 2021 12:46:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-02T12:46:58.519Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Sep 02, 2021 12:46:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-02T12:46:58.554Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Sep 02, 2021 12:46:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-02T12:46:58.584Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Sep 02, 2021 12:46:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-02T12:46:58.620Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Sep 02, 2021 12:46:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-02T12:46:58.646Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Sep 02, 2021 12:46:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-02T12:46:58.677Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Sep 02, 2021 12:46:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-02T12:46:58.698Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Sep 02, 2021 12:47:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-02T12:46:58.728Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Sep 02, 2021 12:47:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-02T12:46:58.762Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Sep 02, 2021 12:47:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-02T12:46:58.794Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Sep 02, 2021 12:47:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-02T12:46:58.827Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Sep 02, 2021 12:47:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-02T12:46:58.854Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Sep 02, 2021 12:47:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-02T12:46:58.885Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Sep 02, 2021 12:47:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-02T12:46:58.917Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Sep 02, 2021 12:47:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-02T12:46:58.951Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Sep 02, 2021 12:47:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-02T12:46:58.973Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Sep 02, 2021 12:47:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-02T12:46:59.006Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Sep 02, 2021 12:47:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-02T12:46:59.036Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Sep 02, 2021 12:47:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-02T12:46:59.070Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Sep 02, 2021 12:47:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-02T12:46:59.472Z: Starting 5 ****s in us-central1-a...
Sep 02, 2021 12:47:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-02T12:47:08.424Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Sep 02, 2021 12:47:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-02T12:47:39.287Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Sep 02, 2021 12:48:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-02T12:48:10.090Z: Workers have started successfully.
Sep 02, 2021 12:48:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-02T12:48:10.122Z: Workers have started successfully.
Sep 02, 2021 4:00:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-02T16:00:28.514Z: Cancel request is committed for workflow job: 2021-09-02_05_46_43-16590442448644662931.
Sep 02, 2021 4:00:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-02T16:00:28.586Z: Cleaning up.
Sep 02, 2021 4:00:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-02T16:00:28.658Z: Stopping **** pool...
Sep 02, 2021 4:00:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-02T16:00:28.704Z: Stopping **** pool...
Sep 02, 2021 4:02:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-02T16:02:49.680Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Sep 02, 2021 4:02:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-02T16:02:49.718Z: Worker pool stopped.
Sep 02, 2021 4:02:54 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-09-02_05_46_43-16590442448644662931 finished with status CANCELLED.
Load test results for test (ID): 9986c180-3701-4274-975d-5d3acea884a7 and timestamp: 2021-09-02T12:46:37.964000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11474.881
dataflow_v2_java11_total_bytes_count             2.32144866E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210902124342
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:e4d0fb396672406793530cd45982f3abe648de9bf7206620c67ca72de29fcf16
Deleted: sha256:1460b1bab3b162b0980f003e56d3fb27cc180d96347c946104514cb61bb73a5f
Deleted: sha256:d1911753b0480ef323e537e1536d0b61c9a2934a37a0f3609cbe80b9d1f4740f
Deleted: sha256:8c15d3e4ed96103f0703066102bbe33555de1339ea033734e330787ccd38a665
Deleted: sha256:9abe72867d080759bcef666feefada10b3faf5bf5270d0231cf9f7f3bafcbb9c
Deleted: sha256:53071cf80e387b96ca0ebe5c62ec896cd64f66e316ca2e75e0e42498cd3fa15d
Deleted: sha256:86212cc17ac3659642f766d7d9708c481061ca051523d678b916d40a2ee972d7
Deleted: sha256:f8fae6697db9e0ab71c8c1b7e59c659534486f500fc69d22ce9e1711f7c27b90
Deleted: sha256:857d89924e5826447ff60c6cce56cf08b9609c5feeb604e804f087e0a5760aa9
Deleted: sha256:e80a45d27d8b5fd69f9a54956458855bd578051ea0d44007fc13e8f2254771a5
Deleted: sha256:d1b70dbf0a81a8c491ec403f4188fe98c18645af9a7f8204f914029322013699
Deleted: sha256:384159bd607faebbb47ec9dd78ab7f1947634e15a214edbaacb8c5e3e32eb46e
Deleted: sha256:1d2d60dac388276b221def437b3ff456360ad8ac2c4659a0418ef2bc435daabd
Deleted: sha256:74db1f967095cc5b80b5905dba33b5266108f7e545db12342e6c0a965d2fa0bf
Deleted: sha256:1c889e4ba5110fe6f6b31ffe22e9fdc924394c30ababce8e48c74a9f8c9d5bf1
Deleted: sha256:dfba54daad83cd1f6400e648197ece127abbcc7f5b3db5b64f62d315e464c92c
Deleted: sha256:e7f6a15bbc16302714d8978fbdd761d4f1622e3be4339e3abb44b98134704451
Deleted: sha256:0b502efa37a83cedd36e9bf2744c46a049522553a20394e8ddf00966a9346364
Deleted: sha256:471c3eb42768ea5b4cbca993f82e900b46ecd8f42ccc2f1212ebd34dc91ebe8b
Deleted: sha256:0b4ee0b4aebd249fe248fa10664042100389cf87dc03ee11e086d8d2bf889543
Deleted: sha256:32a1447bab4c207bb5e0b21349d858d8d776387589b6a2c1be3db730aaace9fe
Deleted: sha256:52155632746b0eb070a31f74587b7d36d6fd4d0654672d12a1477fd9eedcf11d
Deleted: sha256:8ddc3345de28322ee7e10e2bbffe58e79ad292f086be0f9804f06ad6c5f64cca
Deleted: sha256:0a64dface855551480a10a7faf5693cb988eccc48623333ab4f58ed3bd1a2cf1
Deleted: sha256:c369f665b6c3a492f3fc9a7667b2b303e0ae253980b0b47f0d42b301f7502b9b
Deleted: sha256:f31c22d591ebf4ae4a563e93fe8c3d855e2c7c20814ab63bd45d173617782bc2
Deleted: sha256:de8c70b64e82d57c2ed0519579e56d040d18db1987fad2c1610d6a310f138441
Deleted: sha256:31d915ce66f8553f87f03175c2b10e45a7ad3de78a9f33f254beca528026c138
Deleted: sha256:60a08c2d3b3b15d1283627112fb48c47e3a03c9b37e74a4a5e26c8b3c377eec4
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210902124342]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:e4d0fb396672406793530cd45982f3abe648de9bf7206620c67ca72de29fcf16]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210902124342] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:e4d0fb396672406793530cd45982f3abe648de9bf7206620c67ca72de29fcf16])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:e4d0fb396672406793530cd45982f3abe648de9bf7206620c67ca72de29fcf16
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:e4d0fb396672406793530cd45982f3abe648de9bf7206620c67ca72de29fcf16
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:e4d0fb396672406793530cd45982f3abe648de9bf7206620c67ca72de29fcf16].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 35s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/tn67b7x4g7wzu

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #76

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/76/display/redirect?page=changes>

Changes:

[emilyye] sync nltk, orjson for Python image

[noreply] Allow `google-auth < 3`

[samuelw] [BEAM-12776] Change closing to happen in background in parallel for

[Luke Cwik] [BEAM-12802] Refactor DataStreamsDecoder so that it becomes aware of the

[noreply] Fix typo in BigQuery documentation

[ajamato] [BEAM-11994] Instantiate a new ServiceCallMetric before each request to

[Ankur Goenka] Remove duplicate 2.33.0 section

[Steve Niemitz] [BEAM-12767] Improve PipelineOption parsing UX

[noreply] add python spark example in documentation (#15426)

[noreply] Add per-batch metrics to JdbcIO.write (#15429)


------------------------------------------
[...truncated 50.92 KB...]
05103deb4558: Layer already exists
a881cfa23a78: Layer already exists
46553dbf9a6a: Pushed
825b5c390d9a: Pushed
f96bb7a1c058: Pushed
20210901124331: digest: sha256:e9de3c0384410542c5a5defa5e66c022340c088ca41ffa8c8eb23306f158beba size: 4311

> Task :sdks:java:testing:load-tests:run
Sep 01, 2021 12:45:21 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Sep 01, 2021 12:45:21 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 191 files. Enable logging at DEBUG level to see which files will be staged.
Sep 01, 2021 12:45:22 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Sep 01, 2021 12:45:22 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Sep 01, 2021 12:45:25 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Sep 01, 2021 12:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <112622 bytes, hash 942ff591edb74b2d891525494d5c7cd9d7ebe325e7da6d5146bd451cd4dc9eb5> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-lC_1ke23Sy2JFSVJTVx82dfr4yXn2m1RRr1FHNTcnrU.pb
Sep 01, 2021 12:45:27 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 191 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Sep 01, 2021 12:45:27 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 191 files cached, 0 files newly uploaded in 0 seconds
Sep 01, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Sep 01, 2021 12:45:27 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1f346ad2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@46a145ba, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7ab34619, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@ae2db25, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@363c4251, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7afc4db9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1a1f5f71, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@63884e4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@524270b8, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4acb7ecc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2a4f5433, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6812fa3a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@29149030, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@38b8b6c0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@68868328, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@173a6728, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1a22e0ef, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@67514bdd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3b920bdc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7f5538a1]
Sep 01, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Sep 01, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Sep 01, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Sep 01, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Sep 01, 2021 12:45:27 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@a4b5ce3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@f5b6e78, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4b4eced1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@71926a36, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@216e9ca3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@75120e58, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@48976e6d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2a367e93, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7f6874f2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1a6dc589, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@697a34af, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@70211df5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4c5228e7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@38e7ed69, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@806996, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@78b612c6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@257e0827, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@22752544, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@21ba2445, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@69d23296]
Sep 01, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Sep 01, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Sep 01, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Sep 01, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Sep 01, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Sep 01, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Sep 01, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Sep 01, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Sep 01, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Sep 01, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Sep 01, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Sep 01, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.34.0-SNAPSHOT
Sep 01, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-09-01_05_45_27-7406839887615416035?project=apache-beam-testing
Sep 01, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-09-01_05_45_27-7406839887615416035
Sep 01, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-09-01_05_45_27-7406839887615416035
Sep 01, 2021 12:45:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-09-01T12:45:35.772Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-09-3eql. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Sep 01, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-01T12:45:39.524Z: Worker configuration: e2-standard-2 in us-central1-a.
Sep 01, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-01T12:45:40.160Z: Expanding SplittableParDo operations into optimizable parts.
Sep 01, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-01T12:45:40.181Z: Expanding CollectionToSingleton operations into optimizable parts.
Sep 01, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-01T12:45:40.248Z: Expanding CoGroupByKey operations into optimizable parts.
Sep 01, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-01T12:45:40.319Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Sep 01, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-01T12:45:40.340Z: Expanding GroupByKey operations into streaming Read/Write steps
Sep 01, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-01T12:45:40.409Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Sep 01, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-01T12:45:40.508Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Sep 01, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-01T12:45:40.538Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Sep 01, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-01T12:45:40.575Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Sep 01, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-01T12:45:40.594Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Sep 01, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-01T12:45:40.616Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Sep 01, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-01T12:45:40.637Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Sep 01, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-01T12:45:40.665Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Sep 01, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-01T12:45:40.700Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Sep 01, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-01T12:45:40.732Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Sep 01, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-01T12:45:40.778Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Sep 01, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-01T12:45:40.805Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Sep 01, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-01T12:45:40.827Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Sep 01, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-01T12:45:40.854Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Sep 01, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-01T12:45:40.888Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Sep 01, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-01T12:45:40.922Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Sep 01, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-01T12:45:40.954Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Sep 01, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-01T12:45:40.990Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Sep 01, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-01T12:45:41.021Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Sep 01, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-01T12:45:41.055Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Sep 01, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-01T12:45:41.085Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Sep 01, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-01T12:45:41.110Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Sep 01, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-01T12:45:41.166Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Sep 01, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-01T12:45:41.187Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Sep 01, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-01T12:45:41.541Z: Starting 5 ****s in us-central1-a...
Sep 01, 2021 12:45:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-01T12:45:59.189Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Sep 01, 2021 12:46:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-01T12:46:26.997Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Sep 01, 2021 12:46:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-01T12:46:55.773Z: Workers have started successfully.
Sep 01, 2021 12:46:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-01T12:46:55.805Z: Workers have started successfully.
Sep 01, 2021 12:51:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-01T12:51:41.706Z: generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:1089
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:1089
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:1089
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:1089
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:1089
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:1089
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:1089
Sep 01, 2021 12:52:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-01T12:52:50.729Z: generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:1089
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:1089
Sep 01, 2021 12:52:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-01T12:52:52.787Z: generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:1089
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:1089
Sep 01, 2021 4:00:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-01T16:00:26.277Z: Cancel request is committed for workflow job: 2021-09-01_05_45_27-7406839887615416035.
Sep 01, 2021 4:00:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-01T16:00:26.339Z: Cleaning up.
Sep 01, 2021 4:00:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-01T16:00:26.400Z: Stopping **** pool...
Sep 01, 2021 4:00:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-01T16:00:26.445Z: Stopping **** pool...
Sep 01, 2021 4:02:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-01T16:02:44.075Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Sep 01, 2021 4:02:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-01T16:02:44.106Z: Worker pool stopped.
Sep 01, 2021 4:02:50 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-09-01_05_45_27-7406839887615416035 finished with status CANCELLED.
Load test results for test (ID): 619ef2be-01db-4759-837f-40d3f5c2590b and timestamp: 2021-09-01T12:45:22.205000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11549.051
dataflow_v2_java11_total_bytes_count             2.49502246E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210901124331
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:e9de3c0384410542c5a5defa5e66c022340c088ca41ffa8c8eb23306f158beba
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210901124331]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:e9de3c0384410542c5a5defa5e66c022340c088ca41ffa8c8eb23306f158beba]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210901124331] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:e9de3c0384410542c5a5defa5e66c022340c088ca41ffa8c8eb23306f158beba])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:e9de3c0384410542c5a5defa5e66c022340c088ca41ffa8c8eb23306f158beba
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:e9de3c0384410542c5a5defa5e66c022340c088ca41ffa8c8eb23306f158beba
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:e9de3c0384410542c5a5defa5e66c022340c088ca41ffa8c8eb23306f158beba].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 36s
101 actionable tasks: 73 executed, 26 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/jasppfcaup67w

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #75

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/75/display/redirect?page=changes>

Changes:

[Andrew Pilloud] Docs are built in build_release_candidate.sh

[Andrew Pilloud] Add pypy to email

[Andrew Pilloud] Update CHANGES.md along with website

[Ankur Goenka] Fix Change log for Dataframe preview

[Kyle Weaver] Revert "Merge pull request #15271 Decreasing peak memory usage for

[noreply] [BEAM-11097] Refactor Side Input opening to abstract away from ParDo

[Kyle Weaver] [BEAM-12820] Fix null check error

[kawaigin] [BEAM-10708] Added beam_sql magics

[noreply] Merge pull request #15007 from [BEAM-12428] Implement


------------------------------------------
[...truncated 49.06 KB...]
The push refers to repository [us.gcr.io/apache-beam-testing/java-postcommit-it/java]
d4835d6665d2: Preparing
6cbb157804e4: Preparing
b3d7ddc28005: Preparing
52a0415f1003: Preparing
3cbb6ebbc1c0: Preparing
9bbfbc762e3f: Preparing
7a82a697e8ce: Preparing
124558ad7f96: Preparing
f98885472d7d: Preparing
0f1c176a278b: Preparing
b818b01328d6: Preparing
ce9509e99a00: Preparing
db15c90c0fb2: Preparing
a62f4b6a21e8: Preparing
062f9311b72d: Preparing
9889ce9dc2b0: Preparing
21b17a30443e: Preparing
05103deb4558: Preparing
a881cfa23a78: Preparing
ce9509e99a00: Waiting
db15c90c0fb2: Waiting
7a82a697e8ce: Waiting
21b17a30443e: Waiting
a62f4b6a21e8: Waiting
a881cfa23a78: Waiting
9bbfbc762e3f: Waiting
124558ad7f96: Waiting
9889ce9dc2b0: Waiting
062f9311b72d: Waiting
0f1c176a278b: Waiting
05103deb4558: Waiting
b818b01328d6: Waiting
b3d7ddc28005: Pushed
6cbb157804e4: Pushed
3cbb6ebbc1c0: Pushed
d4835d6665d2: Pushed
9bbfbc762e3f: Pushed
52a0415f1003: Pushed
124558ad7f96: Pushed
f98885472d7d: Pushed
db15c90c0fb2: Layer already exists
a62f4b6a21e8: Layer already exists
b818b01328d6: Pushed
062f9311b72d: Layer already exists
9889ce9dc2b0: Layer already exists
7a82a697e8ce: Pushed
05103deb4558: Layer already exists
a881cfa23a78: Layer already exists
21b17a30443e: Layer already exists
ce9509e99a00: Pushed
0f1c176a278b: Pushed
20210831124331: digest: sha256:e0beb387c6d244d80177c853180f0226a358a52c6b4a776b251d7b45c3c0ef10 size: 4311

> Task :sdks:java:testing:load-tests:run
Aug 31, 2021 12:45:13 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Aug 31, 2021 12:45:13 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 191 files. Enable logging at DEBUG level to see which files will be staged.
Aug 31, 2021 12:45:14 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Aug 31, 2021 12:45:14 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Aug 31, 2021 12:45:16 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Aug 31, 2021 12:45:16 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <112622 bytes, hash b86d27b17ff3a3e571408b67831dd56ba12dc5ef9cb02920382bbccc39d3e9bf> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-uG0nsX_zo-VxQItngx3Va6Etxe-csCkgOCu8zDnT6b8.pb
Aug 31, 2021 12:45:18 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 191 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Aug 31, 2021 12:45:18 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 191 files cached, 0 files newly uploaded in 0 seconds
Aug 31, 2021 12:45:18 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Aug 31, 2021 12:45:18 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6e041285, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@267dc982, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@439b15f2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3aa41da1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@74fab04a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c83468e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@272ce069, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3d7fb838, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1c26273d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3a37a501, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7a583586, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3a26ec8d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@17143b3b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@100c8b75, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2bc378f7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@268cbb86, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@10f7918f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@64d4f7c7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@54e02f6a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@17f3eefb]
Aug 31, 2021 12:45:18 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Aug 31, 2021 12:45:18 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Aug 31, 2021 12:45:18 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Aug 31, 2021 12:45:18 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Aug 31, 2021 12:45:18 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58f437b0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@20f6f88c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4277127c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4c7e978c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@354e7004, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@22ebccb9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5875de6a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@608fe01f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2687725a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5a75ec37, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2c05ff9d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@33956d1a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2e1ddc90, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5b7c8930, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58867cd5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@40e32762, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5a8ab2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3f50b680, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@54db056b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@d5af0a5]
Aug 31, 2021 12:45:18 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Aug 31, 2021 12:45:18 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Aug 31, 2021 12:45:18 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Aug 31, 2021 12:45:18 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Aug 31, 2021 12:45:18 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Aug 31, 2021 12:45:18 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Aug 31, 2021 12:45:18 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Aug 31, 2021 12:45:18 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Aug 31, 2021 12:45:18 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Aug 31, 2021 12:45:18 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Aug 31, 2021 12:45:18 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Aug 31, 2021 12:45:18 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.34.0-SNAPSHOT
Aug 31, 2021 12:45:19 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-08-31_05_45_18-9487515673359331582?project=apache-beam-testing
Aug 31, 2021 12:45:19 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-08-31_05_45_18-9487515673359331582
Aug 31, 2021 12:45:19 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-08-31_05_45_18-9487515673359331582
Aug 31, 2021 12:45:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-08-31T12:45:24.815Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-08-2vr3. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Aug 31, 2021 12:45:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-31T12:45:27.785Z: Worker configuration: e2-standard-2 in us-central1-a.
Aug 31, 2021 12:45:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-31T12:45:28.546Z: Expanding SplittableParDo operations into optimizable parts.
Aug 31, 2021 12:45:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-31T12:45:28.576Z: Expanding CollectionToSingleton operations into optimizable parts.
Aug 31, 2021 12:45:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-31T12:45:28.640Z: Expanding CoGroupByKey operations into optimizable parts.
Aug 31, 2021 12:45:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-31T12:45:28.712Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Aug 31, 2021 12:45:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-31T12:45:28.756Z: Expanding GroupByKey operations into streaming Read/Write steps
Aug 31, 2021 12:45:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-31T12:45:28.825Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Aug 31, 2021 12:45:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-31T12:45:28.918Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Aug 31, 2021 12:45:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-31T12:45:28.947Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Aug 31, 2021 12:45:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-31T12:45:28.981Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Aug 31, 2021 12:45:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-31T12:45:29.007Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Aug 31, 2021 12:45:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-31T12:45:29.040Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Aug 31, 2021 12:45:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-31T12:45:29.072Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Aug 31, 2021 12:45:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-31T12:45:29.100Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Aug 31, 2021 12:45:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-31T12:45:29.138Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Aug 31, 2021 12:45:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-31T12:45:29.192Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Aug 31, 2021 12:45:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-31T12:45:29.217Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Aug 31, 2021 12:45:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-31T12:45:29.245Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Aug 31, 2021 12:45:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-31T12:45:29.280Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Aug 31, 2021 12:45:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-31T12:45:29.314Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Aug 31, 2021 12:45:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-31T12:45:29.336Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Aug 31, 2021 12:45:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-31T12:45:29.361Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Aug 31, 2021 12:45:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-31T12:45:29.388Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Aug 31, 2021 12:45:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-31T12:45:29.427Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Aug 31, 2021 12:45:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-31T12:45:29.450Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Aug 31, 2021 12:45:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-31T12:45:29.474Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Aug 31, 2021 12:45:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-31T12:45:29.498Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Aug 31, 2021 12:45:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-31T12:45:29.525Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Aug 31, 2021 12:45:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-31T12:45:29.549Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Aug 31, 2021 12:45:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-31T12:45:29.568Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Aug 31, 2021 12:45:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-31T12:45:29.968Z: Starting 5 ****s in us-central1-a...
Aug 31, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-31T12:45:38.229Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Aug 31, 2021 12:46:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-31T12:46:13.773Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Aug 31, 2021 12:46:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-31T12:46:43.669Z: Workers have started successfully.
Aug 31, 2021 12:46:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-31T12:46:43.693Z: Workers have started successfully.
Aug 31, 2021 4:00:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-31T16:00:27.593Z: Cancel request is committed for workflow job: 2021-08-31_05_45_18-9487515673359331582.
Aug 31, 2021 4:00:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-31T16:00:28.559Z: Cleaning up.
Aug 31, 2021 4:00:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-31T16:00:28.701Z: Stopping **** pool...
Aug 31, 2021 4:00:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-31T16:00:28.763Z: Stopping **** pool...
Aug 31, 2021 4:02:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-31T16:02:42.686Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Aug 31, 2021 4:02:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-31T16:02:42.723Z: Worker pool stopped.
Aug 31, 2021 4:02:50 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-08-31_05_45_18-9487515673359331582 finished with status CANCELLED.
Load test results for test (ID): a594dff7-4ad8-4e37-a55a-65713fca2bb0 and timestamp: 2021-08-31T12:45:14.251000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11551.033
dataflow_v2_java11_total_bytes_count             2.67027777E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210831124331
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:e0beb387c6d244d80177c853180f0226a358a52c6b4a776b251d7b45c3c0ef10
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210831124331]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:e0beb387c6d244d80177c853180f0226a358a52c6b4a776b251d7b45c3c0ef10]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210831124331] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:e0beb387c6d244d80177c853180f0226a358a52c6b4a776b251d7b45c3c0ef10])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:e0beb387c6d244d80177c853180f0226a358a52c6b4a776b251d7b45c3c0ef10
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:e0beb387c6d244d80177c853180f0226a358a52c6b4a776b251d7b45c3c0ef10
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:e0beb387c6d244d80177c853180f0226a358a52c6b4a776b251d7b45c3c0ef10].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 35s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/k676qyuf22wiw

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #74

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/74/display/redirect>

Changes:


------------------------------------------
[...truncated 49.59 KB...]
f729de08df97: Preparing
e6834376748e: Preparing
7c9239b9bead: Preparing
d7364e1b488d: Preparing
3ab1c51218bf: Preparing
905d465e29a5: Preparing
05b6dee50863: Preparing
37358d759e3c: Preparing
cc1f494738e7: Preparing
760405a5fbed: Preparing
b03c2d665fc4: Preparing
8010309ee463: Preparing
db15c90c0fb2: Preparing
a62f4b6a21e8: Preparing
062f9311b72d: Preparing
9889ce9dc2b0: Preparing
21b17a30443e: Preparing
05103deb4558: Preparing
a881cfa23a78: Preparing
905d465e29a5: Waiting
05b6dee50863: Waiting
cc1f494738e7: Waiting
37358d759e3c: Waiting
760405a5fbed: Waiting
b03c2d665fc4: Waiting
8010309ee463: Waiting
9889ce9dc2b0: Waiting
21b17a30443e: Waiting
062f9311b72d: Waiting
05103deb4558: Waiting
a881cfa23a78: Waiting
db15c90c0fb2: Waiting
a62f4b6a21e8: Waiting
3ab1c51218bf: Pushed
7c9239b9bead: Pushed
e6834376748e: Pushed
905d465e29a5: Pushed
f729de08df97: Pushed
d7364e1b488d: Pushed
37358d759e3c: Pushed
cc1f494738e7: Pushed
db15c90c0fb2: Layer already exists
a62f4b6a21e8: Layer already exists
062f9311b72d: Layer already exists
b03c2d665fc4: Pushed
8010309ee463: Pushed
9889ce9dc2b0: Layer already exists
21b17a30443e: Layer already exists
a881cfa23a78: Layer already exists
05103deb4558: Layer already exists
05b6dee50863: Pushed
760405a5fbed: Pushed
20210830124335: digest: sha256:29cbbe18054d4a74b404b566a3dfc8005d2fc22989ea59230566b367d77de3ef size: 4311

> Task :sdks:java:testing:load-tests:run
Aug 30, 2021 12:45:26 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Aug 30, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 191 files. Enable logging at DEBUG level to see which files will be staged.
Aug 30, 2021 12:45:27 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Aug 30, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Aug 30, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Aug 30, 2021 12:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <112622 bytes, hash 203dcc4767992360f556e26874b8531cf524b1b9fdfaded046840ed23f7e7a46> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-ID3MR2eZI2D1VuJodLhTHPUksbn9-t7QRoQO0j9-ekY.pb
Aug 30, 2021 12:45:31 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 191 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Aug 30, 2021 12:45:32 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 191 files cached, 0 files newly uploaded in 0 seconds
Aug 30, 2021 12:45:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Aug 30, 2021 12:45:32 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12d5c30e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@b887730, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26586b74, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@52f57666, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6e041285, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@267dc982, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@439b15f2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3aa41da1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@74fab04a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c83468e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@272ce069, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3d7fb838, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1c26273d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3a37a501, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7a583586, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3a26ec8d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@17143b3b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@100c8b75, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2bc378f7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@268cbb86]
Aug 30, 2021 12:45:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Aug 30, 2021 12:45:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Aug 30, 2021 12:45:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Aug 30, 2021 12:45:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Aug 30, 2021 12:45:32 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@20d87335, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2a8a4e0c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26c89563, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3bd6ba24, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58f437b0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@20f6f88c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4277127c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4c7e978c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@354e7004, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@22ebccb9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5875de6a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@608fe01f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2687725a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5a75ec37, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2c05ff9d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@33956d1a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2e1ddc90, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5b7c8930, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58867cd5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@40e32762]
Aug 30, 2021 12:45:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Aug 30, 2021 12:45:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Aug 30, 2021 12:45:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Aug 30, 2021 12:45:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Aug 30, 2021 12:45:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Aug 30, 2021 12:45:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Aug 30, 2021 12:45:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Aug 30, 2021 12:45:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Aug 30, 2021 12:45:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Aug 30, 2021 12:45:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Aug 30, 2021 12:45:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Aug 30, 2021 12:45:32 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.34.0-SNAPSHOT
Aug 30, 2021 12:45:33 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-08-30_05_45_32-11879196848889758170?project=apache-beam-testing
Aug 30, 2021 12:45:33 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-08-30_05_45_32-11879196848889758170
Aug 30, 2021 12:45:33 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-08-30_05_45_32-11879196848889758170
Aug 30, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-08-30T12:45:40.475Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-08-ebod. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Aug 30, 2021 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-30T12:45:45.164Z: Worker configuration: e2-standard-2 in us-central1-a.
Aug 30, 2021 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-30T12:45:45.908Z: Expanding SplittableParDo operations into optimizable parts.
Aug 30, 2021 12:45:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-30T12:45:45.992Z: Expanding CollectionToSingleton operations into optimizable parts.
Aug 30, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-30T12:45:46.071Z: Expanding CoGroupByKey operations into optimizable parts.
Aug 30, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-30T12:45:46.151Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Aug 30, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-30T12:45:46.177Z: Expanding GroupByKey operations into streaming Read/Write steps
Aug 30, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-30T12:45:46.234Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Aug 30, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-30T12:45:46.323Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Aug 30, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-30T12:45:46.353Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Aug 30, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-30T12:45:46.386Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Aug 30, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-30T12:45:46.424Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Aug 30, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-30T12:45:46.455Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Aug 30, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-30T12:45:46.488Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Aug 30, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-30T12:45:46.512Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Aug 30, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-30T12:45:46.567Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Aug 30, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-30T12:45:46.614Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Aug 30, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-30T12:45:46.649Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Aug 30, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-30T12:45:46.674Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Aug 30, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-30T12:45:46.707Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Aug 30, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-30T12:45:46.740Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Aug 30, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-30T12:45:46.768Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Aug 30, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-30T12:45:46.799Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Aug 30, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-30T12:45:46.826Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Aug 30, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-30T12:45:46.849Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Aug 30, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-30T12:45:46.881Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Aug 30, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-30T12:45:46.918Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Aug 30, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-30T12:45:46.950Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Aug 30, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-30T12:45:46.978Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Aug 30, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-30T12:45:47.015Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Aug 30, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-30T12:45:47.049Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Aug 30, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-30T12:45:47.432Z: Starting 5 ****s in us-central1-a...
Aug 30, 2021 12:46:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-30T12:45:58.833Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Aug 30, 2021 12:46:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-30T12:46:30.442Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Aug 30, 2021 12:47:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-30T12:47:03.470Z: Workers have started successfully.
Aug 30, 2021 12:47:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-30T12:47:03.499Z: Workers have started successfully.
Aug 30, 2021 4:00:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-30T16:00:25.731Z: Cancel request is committed for workflow job: 2021-08-30_05_45_32-11879196848889758170.
Aug 30, 2021 4:00:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-30T16:00:25.812Z: Cleaning up.
Aug 30, 2021 4:00:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-30T16:00:25.887Z: Stopping **** pool...
Aug 30, 2021 4:00:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-30T16:00:25.939Z: Stopping **** pool...
Aug 30, 2021 4:02:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-30T16:02:49.810Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Aug 30, 2021 4:02:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-30T16:02:49.845Z: Worker pool stopped.
Aug 30, 2021 4:02:55 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-08-30_05_45_32-11879196848889758170 finished with status CANCELLED.
Load test results for test (ID): 5f89fbef-274b-4e72-a1fb-4db5385b02a7 and timestamp: 2021-08-30T12:45:27.507000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11521.935
dataflow_v2_java11_total_bytes_count             3.00369656E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210830124335
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:29cbbe18054d4a74b404b566a3dfc8005d2fc22989ea59230566b367d77de3ef
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210830124335]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:29cbbe18054d4a74b404b566a3dfc8005d2fc22989ea59230566b367d77de3ef]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210830124335] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:29cbbe18054d4a74b404b566a3dfc8005d2fc22989ea59230566b367d77de3ef])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:29cbbe18054d4a74b404b566a3dfc8005d2fc22989ea59230566b367d77de3ef
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:29cbbe18054d4a74b404b566a3dfc8005d2fc22989ea59230566b367d77de3ef
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:29cbbe18054d4a74b404b566a3dfc8005d2fc22989ea59230566b367d77de3ef].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 40s
101 actionable tasks: 72 executed, 27 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/f4yo2rohw32mi

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #73

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/73/display/redirect>

Changes:


------------------------------------------
[...truncated 49.23 KB...]
2d8d17d36a1d: Preparing
e1950f3e6b4b: Preparing
2960a27d1046: Preparing
e9c8f24db807: Preparing
05ef00c99485: Preparing
db15c90c0fb2: Preparing
a62f4b6a21e8: Preparing
062f9311b72d: Preparing
9889ce9dc2b0: Preparing
21b17a30443e: Preparing
05103deb4558: Preparing
a881cfa23a78: Preparing
59843649ed12: Waiting
a62f4b6a21e8: Waiting
062f9311b72d: Waiting
25b282da8709: Waiting
9889ce9dc2b0: Waiting
2960a27d1046: Waiting
2d8d17d36a1d: Waiting
21b17a30443e: Waiting
05103deb4558: Waiting
e1950f3e6b4b: Waiting
e9c8f24db807: Waiting
a881cfa23a78: Waiting
db15c90c0fb2: Waiting
78c3c1a1da29: Pushed
e9368af969f7: Pushed
cd7bb8ff962c: Pushed
59843649ed12: Pushed
c8319dd5ad9a: Pushed
a9088aab0bcd: Pushed
2d8d17d36a1d: Pushed
e1950f3e6b4b: Pushed
e9c8f24db807: Pushed
db15c90c0fb2: Layer already exists
25b282da8709: Pushed
062f9311b72d: Layer already exists
a62f4b6a21e8: Layer already exists
9889ce9dc2b0: Layer already exists
21b17a30443e: Layer already exists
05103deb4558: Layer already exists
a881cfa23a78: Layer already exists
05ef00c99485: Pushed
2960a27d1046: Pushed
20210829124333: digest: sha256:8201bded74457f2b179a12ccbc72d7f262df9180f2bb2c7f5cff76a63f7d0eb2 size: 4311

> Task :sdks:java:testing:load-tests:run
Aug 29, 2021 12:45:21 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Aug 29, 2021 12:45:21 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 191 files. Enable logging at DEBUG level to see which files will be staged.
Aug 29, 2021 12:45:22 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Aug 29, 2021 12:45:22 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Aug 29, 2021 12:45:25 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Aug 29, 2021 12:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <112622 bytes, hash c28672f39e5292efa97a75fcc2f5b9a59b3d76ab6acdce1acc8a2aa51216e223> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-woZy855Sku-penX8wvW5pZs9dqtqzc4azIoqpRIW4iM.pb
Aug 29, 2021 12:45:27 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 191 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Aug 29, 2021 12:45:27 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 191 files cached, 0 files newly uploaded in 0 seconds
Aug 29, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Aug 29, 2021 12:45:27 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@267dc982, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@439b15f2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3aa41da1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@74fab04a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c83468e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@272ce069, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3d7fb838, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1c26273d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3a37a501, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7a583586, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3a26ec8d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@17143b3b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@100c8b75, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2bc378f7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@268cbb86, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@10f7918f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@64d4f7c7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@54e02f6a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@17f3eefb, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ba46e63]
Aug 29, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Aug 29, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Aug 29, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Aug 29, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Aug 29, 2021 12:45:27 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@20f6f88c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4277127c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4c7e978c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@354e7004, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@22ebccb9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5875de6a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@608fe01f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2687725a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5a75ec37, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2c05ff9d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@33956d1a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2e1ddc90, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5b7c8930, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58867cd5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@40e32762, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5a8ab2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3f50b680, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@54db056b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@d5af0a5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5981f4a6]
Aug 29, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Aug 29, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Aug 29, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Aug 29, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Aug 29, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Aug 29, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Aug 29, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Aug 29, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Aug 29, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Aug 29, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Aug 29, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Aug 29, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.34.0-SNAPSHOT
Aug 29, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-08-29_05_45_28-1478507726918055493?project=apache-beam-testing
Aug 29, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-08-29_05_45_28-1478507726918055493
Aug 29, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-08-29_05_45_28-1478507726918055493
Aug 29, 2021 12:45:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-08-29T12:45:34.989Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-08-2dy5. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Aug 29, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-29T12:45:38.638Z: Worker configuration: e2-standard-2 in us-central1-a.
Aug 29, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-29T12:45:39.388Z: Expanding SplittableParDo operations into optimizable parts.
Aug 29, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-29T12:45:39.417Z: Expanding CollectionToSingleton operations into optimizable parts.
Aug 29, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-29T12:45:39.481Z: Expanding CoGroupByKey operations into optimizable parts.
Aug 29, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-29T12:45:39.539Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Aug 29, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-29T12:45:39.570Z: Expanding GroupByKey operations into streaming Read/Write steps
Aug 29, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-29T12:45:39.652Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Aug 29, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-29T12:45:39.753Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Aug 29, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-29T12:45:39.778Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Aug 29, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-29T12:45:39.810Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Aug 29, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-29T12:45:39.832Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Aug 29, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-29T12:45:39.865Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Aug 29, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-29T12:45:39.898Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Aug 29, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-29T12:45:39.922Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Aug 29, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-29T12:45:39.952Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Aug 29, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-29T12:45:39.986Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Aug 29, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-29T12:45:40.017Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Aug 29, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-29T12:45:40.052Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Aug 29, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-29T12:45:40.087Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Aug 29, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-29T12:45:40.110Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Aug 29, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-29T12:45:40.143Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Aug 29, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-29T12:45:40.175Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Aug 29, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-29T12:45:40.209Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Aug 29, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-29T12:45:40.242Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Aug 29, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-29T12:45:40.274Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Aug 29, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-29T12:45:40.302Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Aug 29, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-29T12:45:40.324Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Aug 29, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-29T12:45:40.347Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Aug 29, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-29T12:45:40.402Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Aug 29, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-29T12:45:40.436Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Aug 29, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-29T12:45:40.830Z: Starting 5 ****s in us-central1-a...
Aug 29, 2021 12:46:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-29T12:46:02.567Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Aug 29, 2021 12:46:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-29T12:46:24.496Z: Autoscaling: Raised the number of ****s to 4 so that the pipeline can catch up with its backlog and keep up with its input rate.
Aug 29, 2021 12:46:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-29T12:46:24.516Z: Resized **** pool to 4, though goal was 5.  This could be a quota issue.
Aug 29, 2021 12:46:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-29T12:46:54.732Z: Workers have started successfully.
Aug 29, 2021 12:46:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-29T12:46:54.758Z: Workers have started successfully.
Aug 29, 2021 12:47:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-29T12:47:05.424Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Aug 29, 2021 4:00:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-29T16:00:25.544Z: Cancel request is committed for workflow job: 2021-08-29_05_45_28-1478507726918055493.
Aug 29, 2021 4:00:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-29T16:00:25.596Z: Cleaning up.
Aug 29, 2021 4:00:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-29T16:00:25.665Z: Stopping **** pool...
Aug 29, 2021 4:00:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-29T16:00:25.707Z: Stopping **** pool...
Aug 29, 2021 4:02:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-29T16:02:44.435Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Aug 29, 2021 4:02:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-29T16:02:44.502Z: Worker pool stopped.
Aug 29, 2021 4:02:51 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-08-29_05_45_28-1478507726918055493 finished with status CANCELLED.
Load test results for test (ID): 7b942949-3da1-4f45-bbc0-2e099e0c8c9d and timestamp: 2021-08-29T12:45:21.934000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11553.381
dataflow_v2_java11_total_bytes_count             2.95107567E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210829124333
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:8201bded74457f2b179a12ccbc72d7f262df9180f2bb2c7f5cff76a63f7d0eb2
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210829124333]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:8201bded74457f2b179a12ccbc72d7f262df9180f2bb2c7f5cff76a63f7d0eb2]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210829124333] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:8201bded74457f2b179a12ccbc72d7f262df9180f2bb2c7f5cff76a63f7d0eb2])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:0cbc6c32ca4f132acadedf169a3d2f9ce617386d5b301ae2521258226c91cd82
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:0cbc6c32ca4f132acadedf169a3d2f9ce617386d5b301ae2521258226c91cd82
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:0cbc6c32ca4f132acadedf169a3d2f9ce617386d5b301ae2521258226c91cd82].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:8201bded74457f2b179a12ccbc72d7f262df9180f2bb2c7f5cff76a63f7d0eb2
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:8201bded74457f2b179a12ccbc72d7f262df9180f2bb2c7f5cff76a63f7d0eb2
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:8201bded74457f2b179a12ccbc72d7f262df9180f2bb2c7f5cff76a63f7d0eb2].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 35s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/awp2oesfb57tw

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #72

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/72/display/redirect?page=changes>

Changes:

[randomstep] [BEAM-12706] Bump Apache Arrow to 5.0.0

[aromanenko.dev] [BEAM-12270] TPC-DS: Add schema projection for Parquet source

[randomstep] [BEAM-12706] Bump Apache Arrow to 5.0.0

[noreply] [BEAM-12810] Reverting PR-15185 (#15402)


------------------------------------------
[...truncated 49.26 KB...]
95ccfc6dfa27: Preparing
0f47db8df84a: Preparing
0d144fee2724: Preparing
0926e4249010: Preparing
db15c90c0fb2: Preparing
a62f4b6a21e8: Preparing
062f9311b72d: Preparing
9889ce9dc2b0: Preparing
21b17a30443e: Preparing
05103deb4558: Preparing
a881cfa23a78: Preparing
a1435351be11: Waiting
703fd6850b35: Waiting
062f9311b72d: Waiting
95ccfc6dfa27: Waiting
9889ce9dc2b0: Waiting
21b17a30443e: Waiting
db15c90c0fb2: Waiting
0f47db8df84a: Waiting
fe3f2cc1fc03: Waiting
a62f4b6a21e8: Waiting
05103deb4558: Waiting
0926e4249010: Waiting
a881cfa23a78: Waiting
0d144fee2724: Waiting
a35f2e815c5b: Pushed
2912d8cd41de: Pushed
d1f8c9823775: Pushed
f3289642dff8: Pushed
fe3f2cc1fc03: Pushed
73122cb8f52b: Pushed
703fd6850b35: Pushed
95ccfc6dfa27: Pushed
db15c90c0fb2: Layer already exists
a62f4b6a21e8: Layer already exists
062f9311b72d: Layer already exists
9889ce9dc2b0: Layer already exists
21b17a30443e: Layer already exists
0d144fee2724: Pushed
05103deb4558: Layer already exists
0926e4249010: Pushed
a881cfa23a78: Layer already exists
a1435351be11: Pushed
0f47db8df84a: Pushed
20210828124336: digest: sha256:1e7fc6382fb1b7b16a76a2bba4141d4fc8844f319c501e72032fdbbd669e04c3 size: 4311

> Task :sdks:java:testing:load-tests:run
Aug 28, 2021 12:45:42 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Aug 28, 2021 12:45:42 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 191 files. Enable logging at DEBUG level to see which files will be staged.
Aug 28, 2021 12:45:43 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Aug 28, 2021 12:45:43 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Aug 28, 2021 12:45:45 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Aug 28, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <112622 bytes, hash 3eb8cbafe77fdfaa5fa0ef540a4f5eae87af41ab64ba95cf0ee8cad35b491427> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-PrjLr-d_36pfoO9UCk9eroevQatkupXPDujK01tJFCc.pb
Aug 28, 2021 12:45:47 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 191 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Aug 28, 2021 12:45:47 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 191 files cached, 0 files newly uploaded in 0 seconds
Aug 28, 2021 12:45:47 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Aug 28, 2021 12:45:47 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12d5c30e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@b887730, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26586b74, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@52f57666, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6e041285, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@267dc982, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@439b15f2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3aa41da1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@74fab04a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c83468e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@272ce069, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3d7fb838, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1c26273d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3a37a501, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7a583586, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3a26ec8d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@17143b3b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@100c8b75, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2bc378f7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@268cbb86]
Aug 28, 2021 12:45:47 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Aug 28, 2021 12:45:47 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Aug 28, 2021 12:45:47 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Aug 28, 2021 12:45:47 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Aug 28, 2021 12:45:47 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@20d87335, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2a8a4e0c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26c89563, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3bd6ba24, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58f437b0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@20f6f88c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4277127c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4c7e978c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@354e7004, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@22ebccb9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5875de6a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@608fe01f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2687725a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5a75ec37, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2c05ff9d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@33956d1a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2e1ddc90, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5b7c8930, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58867cd5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@40e32762]
Aug 28, 2021 12:45:47 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Aug 28, 2021 12:45:47 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Aug 28, 2021 12:45:47 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Aug 28, 2021 12:45:47 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Aug 28, 2021 12:45:47 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Aug 28, 2021 12:45:47 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Aug 28, 2021 12:45:47 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Aug 28, 2021 12:45:47 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Aug 28, 2021 12:45:47 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Aug 28, 2021 12:45:47 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Aug 28, 2021 12:45:47 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Aug 28, 2021 12:45:48 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.34.0-SNAPSHOT
Aug 28, 2021 12:45:48 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-08-28_05_45_48-8144314990321223725?project=apache-beam-testing
Aug 28, 2021 12:45:48 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-08-28_05_45_48-8144314990321223725
Aug 28, 2021 12:45:48 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-08-28_05_45_48-8144314990321223725
Aug 28, 2021 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-08-28T12:45:54.031Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-08-nz9d. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Aug 28, 2021 12:45:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-28T12:45:57.843Z: Worker configuration: e2-standard-2 in us-central1-a.
Aug 28, 2021 12:45:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-28T12:45:58.682Z: Expanding SplittableParDo operations into optimizable parts.
Aug 28, 2021 12:45:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-28T12:45:58.705Z: Expanding CollectionToSingleton operations into optimizable parts.
Aug 28, 2021 12:45:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-28T12:45:58.767Z: Expanding CoGroupByKey operations into optimizable parts.
Aug 28, 2021 12:45:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-28T12:45:58.859Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Aug 28, 2021 12:45:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-28T12:45:58.896Z: Expanding GroupByKey operations into streaming Read/Write steps
Aug 28, 2021 12:45:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-28T12:45:58.975Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Aug 28, 2021 12:45:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-28T12:45:59.071Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Aug 28, 2021 12:45:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-28T12:45:59.132Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Aug 28, 2021 12:45:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-28T12:45:59.166Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Aug 28, 2021 12:45:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-28T12:45:59.203Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Aug 28, 2021 12:45:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-28T12:45:59.225Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Aug 28, 2021 12:45:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-28T12:45:59.267Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Aug 28, 2021 12:45:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-28T12:45:59.301Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Aug 28, 2021 12:45:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-28T12:45:59.339Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Aug 28, 2021 12:45:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-28T12:45:59.368Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Aug 28, 2021 12:45:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-28T12:45:59.389Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Aug 28, 2021 12:45:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-28T12:45:59.426Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Aug 28, 2021 12:45:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-28T12:45:59.471Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Aug 28, 2021 12:45:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-28T12:45:59.513Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Aug 28, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-28T12:45:59.550Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Aug 28, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-28T12:45:59.592Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Aug 28, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-28T12:45:59.621Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Aug 28, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-28T12:45:59.650Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Aug 28, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-28T12:45:59.685Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Aug 28, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-28T12:45:59.718Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Aug 28, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-28T12:45:59.758Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Aug 28, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-28T12:45:59.792Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Aug 28, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-28T12:45:59.831Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Aug 28, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-28T12:45:59.859Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Aug 28, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-28T12:46:00.235Z: Starting 5 ****s in us-central1-a...
Aug 28, 2021 12:46:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-28T12:46:15.170Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Aug 28, 2021 12:46:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-28T12:46:54.253Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Aug 28, 2021 12:47:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-28T12:47:26.353Z: Workers have started successfully.
Aug 28, 2021 12:47:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-28T12:47:26.388Z: Workers have started successfully.
Aug 28, 2021 1:07:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-28T13:07:02.801Z: Staged package kafka-clients-2.4.1-or4Vrm01S3aXE64MuzGE8iWqFvyYIFseskr9n8KjYNk.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/kafka-clients-2.4.1-or4Vrm01S3aXE64MuzGE8iWqFvyYIFseskr9n8KjYNk.jar' is inaccessible.
Aug 28, 2021 1:07:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-28T13:07:02.909Z: Staged package lz4-java-1.6.0-0ilUWqKx1SA8h2YUvbz_yswwNpf0-PJvdk4dbB7S5BY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/lz4-java-1.6.0-0ilUWqKx1SA8h2YUvbz_yswwNpf0-PJvdk4dbB7S5BY.jar' is inaccessible.
Aug 28, 2021 1:07:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-08-28T13:07:03.602Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Aug 28, 2021 1:10:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-08-28T13:10:04.105Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Aug 28, 2021 4:00:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-28T16:00:24.790Z: Cancel request is committed for workflow job: 2021-08-28_05_45_48-8144314990321223725.
Aug 28, 2021 4:00:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-28T16:00:24.818Z: Cleaning up.
Aug 28, 2021 4:00:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-28T16:00:24.867Z: Stopping **** pool...
Aug 28, 2021 4:00:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-28T16:00:24.933Z: Stopping **** pool...
Aug 28, 2021 4:02:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-28T16:02:48.396Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Aug 28, 2021 4:02:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-28T16:02:48.433Z: Worker pool stopped.
Aug 28, 2021 4:02:53 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-08-28_05_45_48-8144314990321223725 finished with status CANCELLED.
Load test results for test (ID): 14c229e5-7b1d-4206-860c-b4e1448334ff and timestamp: 2021-08-28T12:45:43.301000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11475.899
dataflow_v2_java11_total_bytes_count                1.88853E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210828124336
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:1e7fc6382fb1b7b16a76a2bba4141d4fc8844f319c501e72032fdbbd669e04c3
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210828124336]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:1e7fc6382fb1b7b16a76a2bba4141d4fc8844f319c501e72032fdbbd669e04c3]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210828124336] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:1e7fc6382fb1b7b16a76a2bba4141d4fc8844f319c501e72032fdbbd669e04c3])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:1e7fc6382fb1b7b16a76a2bba4141d4fc8844f319c501e72032fdbbd669e04c3
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:1e7fc6382fb1b7b16a76a2bba4141d4fc8844f319c501e72032fdbbd669e04c3
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:1e7fc6382fb1b7b16a76a2bba4141d4fc8844f319c501e72032fdbbd669e04c3].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 35s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/2imue4zvckr2i

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #71

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/71/display/redirect?page=changes>

Changes:

[Ahmet Altay] Clean unused methods out of apiclient.py

[Ahmet Altay] fix

[Ahmet Altay] lint

[Ahmet Altay] fix

[baeminbo] [BEAM-12751] Set clientRequestId for Dataflow python job creation

[Ankur Goenka] Add a blogpost for Apache Beam 2.32.0

[Ankur Goenka] Updating date for beam 2.32.0 blog post

[Ankur Goenka] Fixing author name

[Kyle Weaver] [BEAM-12320] Sickbay testSQLSelectsArrayAttributes.

[noreply] [BEAM-12742] SamzaTimerInternalsFactory#deleteTimer(TimerData) does not

[Daniel Oliveira] [GoSDK Infra] Bugfix: Parallelism ignored when using endpoint flag.

[noreply] Merge pull request #15183 from [BEAM-11983] Java Datastore - Implement

[noreply] Merge pull request #15328 from [BEAM-11987] Python Datastore - Implement


------------------------------------------
[...truncated 49.06 KB...]
The push refers to repository [us.gcr.io/apache-beam-testing/java-postcommit-it/java]
c31993b8bccd: Preparing
e16206cc676f: Preparing
38596e8dc1ae: Preparing
365eb72607dc: Preparing
cdeec8afd103: Preparing
208dd7d57401: Preparing
462d5732118b: Preparing
c2da805a8b2f: Preparing
5410b80a80e0: Preparing
b1ff5e7a29de: Preparing
a92aa3fd36d4: Preparing
585b8ba34bdb: Preparing
db15c90c0fb2: Preparing
a62f4b6a21e8: Preparing
062f9311b72d: Preparing
9889ce9dc2b0: Preparing
21b17a30443e: Preparing
05103deb4558: Preparing
a881cfa23a78: Preparing
585b8ba34bdb: Waiting
b1ff5e7a29de: Waiting
462d5732118b: Waiting
a92aa3fd36d4: Waiting
5410b80a80e0: Waiting
c2da805a8b2f: Waiting
db15c90c0fb2: Waiting
21b17a30443e: Waiting
05103deb4558: Waiting
a881cfa23a78: Waiting
062f9311b72d: Waiting
9889ce9dc2b0: Waiting
a62f4b6a21e8: Waiting
e16206cc676f: Pushed
38596e8dc1ae: Pushed
cdeec8afd103: Pushed
c31993b8bccd: Pushed
365eb72607dc: Pushed
208dd7d57401: Pushed
c2da805a8b2f: Pushed
5410b80a80e0: Pushed
db15c90c0fb2: Layer already exists
a62f4b6a21e8: Layer already exists
a92aa3fd36d4: Pushed
062f9311b72d: Layer already exists
9889ce9dc2b0: Layer already exists
21b17a30443e: Layer already exists
05103deb4558: Layer already exists
a881cfa23a78: Layer already exists
462d5732118b: Pushed
585b8ba34bdb: Pushed
b1ff5e7a29de: Pushed
20210827124332: digest: sha256:b19c09ff6bf71db252f8e51a7a7c431ee4e073c9b68960e3284a4cd71f3f56d9 size: 4311

> Task :sdks:java:testing:load-tests:run
Aug 27, 2021 12:45:17 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Aug 27, 2021 12:45:17 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 191 files. Enable logging at DEBUG level to see which files will be staged.
Aug 27, 2021 12:45:18 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Aug 27, 2021 12:45:18 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Aug 27, 2021 12:45:20 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Aug 27, 2021 12:45:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <112622 bytes, hash a0ad02ecb56bfd49dadd01743e16632acf006f0cfcb94e4663d36a0f755d67d9> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-oK0C7LVr_Una3QF0PhZjKs8Abwz8uU5GY9NqD3VdZ9k.pb
Aug 27, 2021 12:45:21 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 191 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Aug 27, 2021 12:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 191 files cached, 0 files newly uploaded in 0 seconds
Aug 27, 2021 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Aug 27, 2021 12:45:22 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12d5c30e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@b887730, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26586b74, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@52f57666, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6e041285, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@267dc982, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@439b15f2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3aa41da1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@74fab04a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c83468e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@272ce069, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3d7fb838, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1c26273d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3a37a501, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7a583586, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3a26ec8d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@17143b3b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@100c8b75, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2bc378f7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@268cbb86]
Aug 27, 2021 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Aug 27, 2021 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Aug 27, 2021 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Aug 27, 2021 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Aug 27, 2021 12:45:22 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@20d87335, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2a8a4e0c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26c89563, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3bd6ba24, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58f437b0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@20f6f88c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4277127c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4c7e978c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@354e7004, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@22ebccb9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5875de6a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@608fe01f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2687725a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5a75ec37, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2c05ff9d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@33956d1a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2e1ddc90, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5b7c8930, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58867cd5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@40e32762]
Aug 27, 2021 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Aug 27, 2021 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Aug 27, 2021 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Aug 27, 2021 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Aug 27, 2021 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Aug 27, 2021 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Aug 27, 2021 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Aug 27, 2021 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Aug 27, 2021 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Aug 27, 2021 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Aug 27, 2021 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Aug 27, 2021 12:45:22 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.34.0-SNAPSHOT
Aug 27, 2021 12:45:23 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-08-27_05_45_22-3632248430096267814?project=apache-beam-testing
Aug 27, 2021 12:45:23 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-08-27_05_45_22-3632248430096267814
Aug 27, 2021 12:45:23 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-08-27_05_45_22-3632248430096267814
Aug 27, 2021 12:45:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-08-27T12:45:31.047Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-08-tyg4. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Aug 27, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-27T12:45:36.058Z: Worker configuration: e2-standard-2 in us-central1-a.
Aug 27, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-27T12:45:36.707Z: Expanding SplittableParDo operations into optimizable parts.
Aug 27, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-27T12:45:36.743Z: Expanding CollectionToSingleton operations into optimizable parts.
Aug 27, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-27T12:45:36.824Z: Expanding CoGroupByKey operations into optimizable parts.
Aug 27, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-27T12:45:36.888Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Aug 27, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-27T12:45:36.920Z: Expanding GroupByKey operations into streaming Read/Write steps
Aug 27, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-27T12:45:36.989Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Aug 27, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-27T12:45:37.088Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Aug 27, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-27T12:45:37.141Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Aug 27, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-27T12:45:37.171Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Aug 27, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-27T12:45:37.197Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Aug 27, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-27T12:45:37.229Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Aug 27, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-27T12:45:37.262Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Aug 27, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-27T12:45:37.301Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Aug 27, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-27T12:45:37.341Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Aug 27, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-27T12:45:37.372Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Aug 27, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-27T12:45:37.402Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Aug 27, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-27T12:45:37.441Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Aug 27, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-27T12:45:37.475Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Aug 27, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-27T12:45:37.508Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Aug 27, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-27T12:45:37.543Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Aug 27, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-27T12:45:37.577Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Aug 27, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-27T12:45:37.604Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Aug 27, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-27T12:45:37.640Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Aug 27, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-27T12:45:37.671Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Aug 27, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-27T12:45:37.716Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Aug 27, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-27T12:45:37.739Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Aug 27, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-27T12:45:37.773Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Aug 27, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-27T12:45:37.805Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Aug 27, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-27T12:45:37.834Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Aug 27, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-27T12:45:38.231Z: Starting 5 ****s in us-central1-a...
Aug 27, 2021 12:46:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-27T12:46:05.389Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Aug 27, 2021 12:46:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-27T12:46:20.517Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Aug 27, 2021 12:46:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-27T12:46:51.903Z: Workers have started successfully.
Aug 27, 2021 12:46:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-27T12:46:51.935Z: Workers have started successfully.
Aug 27, 2021 4:00:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-27T16:00:40.139Z: Cancel request is committed for workflow job: 2021-08-27_05_45_22-3632248430096267814.
Aug 27, 2021 4:00:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-27T16:00:48.141Z: Cleaning up.
Aug 27, 2021 4:00:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-27T16:00:48.221Z: Stopping **** pool...
Aug 27, 2021 4:00:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-27T16:00:48.267Z: Stopping **** pool...
Aug 27, 2021 4:03:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-27T16:03:25.625Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Aug 27, 2021 4:03:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-27T16:03:25.659Z: Worker pool stopped.
Aug 27, 2021 4:03:31 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-08-27_05_45_22-3632248430096267814 finished with status CANCELLED.
Load test results for test (ID): 97b03f92-c8f8-4014-9435-2fa64f812c22 and timestamp: 2021-08-27T12:45:17.930000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11585.005
dataflow_v2_java11_total_bytes_count             2.15865183E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210827124332
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:b19c09ff6bf71db252f8e51a7a7c431ee4e073c9b68960e3284a4cd71f3f56d9
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210827124332]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:b19c09ff6bf71db252f8e51a7a7c431ee4e073c9b68960e3284a4cd71f3f56d9]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210827124332] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:b19c09ff6bf71db252f8e51a7a7c431ee4e073c9b68960e3284a4cd71f3f56d9])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:b19c09ff6bf71db252f8e51a7a7c431ee4e073c9b68960e3284a4cd71f3f56d9
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:b19c09ff6bf71db252f8e51a7a7c431ee4e073c9b68960e3284a4cd71f3f56d9
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:b19c09ff6bf71db252f8e51a7a7c431ee4e073c9b68960e3284a4cd71f3f56d9].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 20m 15s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/fdibiew664j4m

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #70

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/70/display/redirect?page=changes>

Changes:

[Ankur Goenka] Update Beam website to release 2.32.0

[noreply] Update website/www/site/content/en/get-started/downloads.md

[Jan Lukavský] [BEAM-12704] Failing test for Flink primitive Read

[Pablo Estrada] Informative error for BigDecimal conversion in JdbcIO.

[Kyle Weaver] [BEAM-12764] Revert "Merge pull request #15165 from [BEAM-12593] Verify

[Kyle Weaver] [BEAM-12733] Sickbay RecommendationAICatalogItemIT.createCatalogItem

[Kyle Weaver] [BEAM-12683] Sickbay RecommendationAIIT.test_create_catalog_item

[Kyle Weaver] [BEAM-12733] Sickbay RecommendationAIPredictIT.predict

[Jan Lukavský] [BEAM-12704] Primitive Read working on Flink

[noreply] [BEAM-12778] Prevent unnecessary dry run requests to BQ (#15356)

[Udi Meiri] Moving to 2.34.0-SNAPSHOT on master branch.

[noreply] Add 2.34.0 section to CHANGES.md


------------------------------------------
[...truncated 49.17 KB...]

See: https://cloud.google.com/container-registry/docs/support/deprecation-notices#gcloud-docker

The push refers to repository [us.gcr.io/apache-beam-testing/java-postcommit-it/java]
99fb969845f2: Preparing
05d163e76e20: Preparing
0713018df284: Preparing
eeb3ae663931: Preparing
c84a539df940: Preparing
6b8c729e0c28: Preparing
743737de48b6: Preparing
4a9a66b540b4: Preparing
ae1adaf1bd89: Preparing
7f4839d1d776: Preparing
bac508f5a3f5: Preparing
16fe3cd2967a: Preparing
db15c90c0fb2: Preparing
a62f4b6a21e8: Preparing
062f9311b72d: Preparing
9889ce9dc2b0: Preparing
21b17a30443e: Preparing
05103deb4558: Preparing
a881cfa23a78: Preparing
9889ce9dc2b0: Waiting
05103deb4558: Waiting
21b17a30443e: Waiting
a62f4b6a21e8: Waiting
16fe3cd2967a: Waiting
a881cfa23a78: Waiting
062f9311b72d: Waiting
4a9a66b540b4: Waiting
6b8c729e0c28: Waiting
db15c90c0fb2: Waiting
0713018df284: Pushed
05d163e76e20: Pushed
c84a539df940: Pushed
99fb969845f2: Pushed
6b8c729e0c28: Pushed
eeb3ae663931: Pushed
4a9a66b540b4: Pushed
ae1adaf1bd89: Pushed
db15c90c0fb2: Layer already exists
a62f4b6a21e8: Layer already exists
bac508f5a3f5: Pushed
062f9311b72d: Layer already exists
21b17a30443e: Layer already exists
9889ce9dc2b0: Layer already exists
743737de48b6: Pushed
05103deb4558: Layer already exists
16fe3cd2967a: Pushed
a881cfa23a78: Layer already exists
7f4839d1d776: Pushed
20210826124335: digest: sha256:5cd4ead7d45724eacc00e48dc02aa7908a003febdf446416ef77a7b0330c703c size: 4311

> Task :sdks:java:testing:load-tests:run
Aug 26, 2021 12:45:26 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Aug 26, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 191 files. Enable logging at DEBUG level to see which files will be staged.
Aug 26, 2021 12:45:28 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Aug 26, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Aug 26, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Aug 26, 2021 12:45:31 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <112622 bytes, hash d0c0dc1db6830a5980d2430749074a332610a73f9a880ae3530415f622434b97> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-0MDcHbaDClmA0kMHSQdKMyYQpz-aiArjUwQV9iJDS5c.pb
Aug 26, 2021 12:45:33 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 191 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Aug 26, 2021 12:45:33 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 191 files cached, 0 files newly uploaded in 0 seconds
Aug 26, 2021 12:45:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Aug 26, 2021 12:45:33 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12d5c30e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@b887730, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26586b74, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@52f57666, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6e041285, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@267dc982, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@439b15f2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3aa41da1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@74fab04a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c83468e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@272ce069, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3d7fb838, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1c26273d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3a37a501, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7a583586, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3a26ec8d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@17143b3b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@100c8b75, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2bc378f7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@268cbb86]
Aug 26, 2021 12:45:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Aug 26, 2021 12:45:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Aug 26, 2021 12:45:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Aug 26, 2021 12:45:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Aug 26, 2021 12:45:33 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@20d87335, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2a8a4e0c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26c89563, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3bd6ba24, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58f437b0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@20f6f88c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4277127c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4c7e978c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@354e7004, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@22ebccb9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5875de6a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@608fe01f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2687725a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5a75ec37, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2c05ff9d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@33956d1a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2e1ddc90, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5b7c8930, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58867cd5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@40e32762]
Aug 26, 2021 12:45:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Aug 26, 2021 12:45:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Aug 26, 2021 12:45:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Aug 26, 2021 12:45:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Aug 26, 2021 12:45:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Aug 26, 2021 12:45:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Aug 26, 2021 12:45:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Aug 26, 2021 12:45:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Aug 26, 2021 12:45:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Aug 26, 2021 12:45:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Aug 26, 2021 12:45:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Aug 26, 2021 12:45:33 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.34.0-SNAPSHOT
Aug 26, 2021 12:45:34 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-08-26_05_45_34-12250510447000782227?project=apache-beam-testing
Aug 26, 2021 12:45:34 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-08-26_05_45_34-12250510447000782227
Aug 26, 2021 12:45:34 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-08-26_05_45_34-12250510447000782227
Aug 26, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-08-26T12:45:41.231Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-08-bg1j. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Aug 26, 2021 12:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-26T12:45:45.549Z: Worker configuration: e2-standard-2 in us-central1-b.
Aug 26, 2021 12:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-26T12:45:46.185Z: Expanding SplittableParDo operations into optimizable parts.
Aug 26, 2021 12:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-26T12:45:46.216Z: Expanding CollectionToSingleton operations into optimizable parts.
Aug 26, 2021 12:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-26T12:45:46.283Z: Expanding CoGroupByKey operations into optimizable parts.
Aug 26, 2021 12:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-26T12:45:46.344Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Aug 26, 2021 12:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-26T12:45:46.373Z: Expanding GroupByKey operations into streaming Read/Write steps
Aug 26, 2021 12:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-26T12:45:46.439Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Aug 26, 2021 12:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-26T12:45:46.547Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Aug 26, 2021 12:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-26T12:45:46.573Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Aug 26, 2021 12:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-26T12:45:46.616Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Aug 26, 2021 12:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-26T12:45:46.649Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Aug 26, 2021 12:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-26T12:45:46.671Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Aug 26, 2021 12:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-26T12:45:46.705Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Aug 26, 2021 12:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-26T12:45:46.754Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Aug 26, 2021 12:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-26T12:45:46.788Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Aug 26, 2021 12:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-26T12:45:46.821Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Aug 26, 2021 12:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-26T12:45:46.856Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Aug 26, 2021 12:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-26T12:45:46.891Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Aug 26, 2021 12:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-26T12:45:46.912Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Aug 26, 2021 12:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-26T12:45:46.944Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Aug 26, 2021 12:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-26T12:45:46.976Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Aug 26, 2021 12:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-26T12:45:47.013Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Aug 26, 2021 12:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-26T12:45:47.043Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Aug 26, 2021 12:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-26T12:45:47.073Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Aug 26, 2021 12:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-26T12:45:47.104Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Aug 26, 2021 12:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-26T12:45:47.127Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Aug 26, 2021 12:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-26T12:45:47.160Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Aug 26, 2021 12:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-26T12:45:47.195Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Aug 26, 2021 12:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-26T12:45:47.216Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Aug 26, 2021 12:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-26T12:45:47.249Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Aug 26, 2021 12:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-26T12:45:47.579Z: Starting 5 ****s in us-central1-b...
Aug 26, 2021 12:46:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-26T12:46:02.213Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Aug 26, 2021 12:46:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-26T12:46:28.729Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Aug 26, 2021 12:46:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-26T12:46:57.865Z: Workers have started successfully.
Aug 26, 2021 12:46:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-26T12:46:57.894Z: Workers have started successfully.
Aug 26, 2021 4:00:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-26T16:00:25.694Z: Cancel request is committed for workflow job: 2021-08-26_05_45_34-12250510447000782227.
Aug 26, 2021 4:00:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-26T16:00:25.762Z: Cleaning up.
Aug 26, 2021 4:00:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-26T16:00:25.826Z: Stopping **** pool...
Aug 26, 2021 4:00:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-26T16:00:25.886Z: Stopping **** pool...
Aug 26, 2021 4:02:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-26T16:02:37.851Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Aug 26, 2021 4:02:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-26T16:02:37.885Z: Worker pool stopped.
Aug 26, 2021 4:02:44 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-08-26_05_45_34-12250510447000782227 finished with status CANCELLED.
Load test results for test (ID): 2be58d18-db50-46ff-8106-406d73400bfe and timestamp: 2021-08-26T12:45:27.667000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11546.875
dataflow_v2_java11_total_bytes_count             2.78789077E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210826124335
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:5cd4ead7d45724eacc00e48dc02aa7908a003febdf446416ef77a7b0330c703c
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210826124335]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:5cd4ead7d45724eacc00e48dc02aa7908a003febdf446416ef77a7b0330c703c]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210826124335] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:5cd4ead7d45724eacc00e48dc02aa7908a003febdf446416ef77a7b0330c703c])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:5cd4ead7d45724eacc00e48dc02aa7908a003febdf446416ef77a7b0330c703c
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:5cd4ead7d45724eacc00e48dc02aa7908a003febdf446416ef77a7b0330c703c
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:5cd4ead7d45724eacc00e48dc02aa7908a003febdf446416ef77a7b0330c703c].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 28s
101 actionable tasks: 72 executed, 27 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/lku2cyqqozpj6

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #69

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/69/display/redirect?page=changes>

Changes:

[ihor.indyk] Decreasing peak memory usage for

[ihor.indyk] Changing the default `merge_accumulators_batch_size`

[noreply] [Go SDK] Go SDK Exits Experimental (#15374)

[noreply] [BEAM-12724][BEAM-12349] Support user timers in Samza portable runner

[noreply] Clarify PCollection immutability. (#15227)

[noreply] [BEAM-11218] ptest allows to obtain a pipeline result (#15364)

[Kyle Weaver] Remove trailing whitespace

[Robert Burke] [BEAM-9615] Report Row as Known URN

[noreply] Merge pull request #15357 from [BEAM-12781] Add memoization of


------------------------------------------
[...truncated 54.36 KB...]
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Aug 25, 2021 12:45:27 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58f437b0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@20f6f88c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4277127c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4c7e978c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@354e7004, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@22ebccb9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5875de6a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@608fe01f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2687725a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5a75ec37, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2c05ff9d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@33956d1a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2e1ddc90, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5b7c8930, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58867cd5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@40e32762, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5a8ab2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3f50b680, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@54db056b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@d5af0a5]
Aug 25, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Aug 25, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Aug 25, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Aug 25, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Aug 25, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Aug 25, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Aug 25, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Aug 25, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Aug 25, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Aug 25, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Aug 25, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Aug 25, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.33.0-SNAPSHOT
Aug 25, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-08-25_05_45_27-5858850660834944658?project=apache-beam-testing
Aug 25, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-08-25_05_45_27-5858850660834944658
Aug 25, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-08-25_05_45_27-5858850660834944658
Aug 25, 2021 12:45:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-08-25T12:45:36.225Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-08-4q2t. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Aug 25, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-25T12:45:41.362Z: Worker configuration: e2-standard-2 in us-central1-a.
Aug 25, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-25T12:45:42.026Z: Expanding SplittableParDo operations into optimizable parts.
Aug 25, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-25T12:45:42.094Z: Expanding CollectionToSingleton operations into optimizable parts.
Aug 25, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-25T12:45:42.173Z: Expanding CoGroupByKey operations into optimizable parts.
Aug 25, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-25T12:45:42.252Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Aug 25, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-25T12:45:42.293Z: Expanding GroupByKey operations into streaming Read/Write steps
Aug 25, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-25T12:45:42.359Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Aug 25, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-25T12:45:42.534Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Aug 25, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-25T12:45:42.582Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Aug 25, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-25T12:45:42.616Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Aug 25, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-25T12:45:42.645Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Aug 25, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-25T12:45:42.669Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Aug 25, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-25T12:45:42.737Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Aug 25, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-25T12:45:42.773Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Aug 25, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-25T12:45:42.808Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Aug 25, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-25T12:45:42.840Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Aug 25, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-25T12:45:42.879Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Aug 25, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-25T12:45:42.913Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Aug 25, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-25T12:45:42.948Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Aug 25, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-25T12:45:42.995Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Aug 25, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-25T12:45:43.024Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Aug 25, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-25T12:45:43.054Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Aug 25, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-25T12:45:43.077Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Aug 25, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-25T12:45:43.099Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Aug 25, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-25T12:45:43.121Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Aug 25, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-25T12:45:43.156Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Aug 25, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-25T12:45:43.192Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Aug 25, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-25T12:45:43.230Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Aug 25, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-25T12:45:43.264Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Aug 25, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-25T12:45:43.300Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Aug 25, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-25T12:45:43.724Z: Starting 5 ****s in us-central1-a...
Aug 25, 2021 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-25T12:45:55.533Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Aug 25, 2021 12:46:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-25T12:46:35.005Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Aug 25, 2021 12:47:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-25T12:47:32.411Z: Workers have started successfully.
Aug 25, 2021 12:47:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-25T12:47:32.445Z: Workers have started successfully.
Aug 25, 2021 3:15:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-25T15:15:47.689Z: Staged package slf4j-api-1.7.30-zboHlk0btAoHYUhcax6ML4_Z6x0ZxTkorA1_lRAQXFc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/slf4j-api-1.7.30-zboHlk0btAoHYUhcax6ML4_Z6x0ZxTkorA1_lRAQXFc.jar' is inaccessible.
Aug 25, 2021 3:15:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-25T15:15:47.780Z: Staged package slf4j-jdk14-1.7.30-4PnbBJN49kZ5QXcUVJlSMyhft633LEZ-ZdryXmc6y6g.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/slf4j-jdk14-1.7.30-4PnbBJN49kZ5QXcUVJlSMyhft633LEZ-ZdryXmc6y6g.jar' is inaccessible.
Aug 25, 2021 3:15:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-25T15:15:47.926Z: Staged package xz-1.5-hvMPqHdfo6Ys2znR7XimAZFkwQWIZASNQsvuJE4m6EA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/xz-1.5-hvMPqHdfo6Ys2znR7XimAZFkwQWIZASNQsvuJE4m6EA.jar' is inaccessible.
Aug 25, 2021 3:15:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-08-25T15:15:48.003Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Aug 25, 2021 3:18:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-08-25T15:18:47.677Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Aug 25, 2021 3:21:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-25T15:21:47.728Z: Staged package slf4j-api-1.7.30-zboHlk0btAoHYUhcax6ML4_Z6x0ZxTkorA1_lRAQXFc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/slf4j-api-1.7.30-zboHlk0btAoHYUhcax6ML4_Z6x0ZxTkorA1_lRAQXFc.jar' is inaccessible.
Aug 25, 2021 3:21:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-25T15:21:47.767Z: Staged package slf4j-jdk14-1.7.30-4PnbBJN49kZ5QXcUVJlSMyhft633LEZ-ZdryXmc6y6g.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/slf4j-jdk14-1.7.30-4PnbBJN49kZ5QXcUVJlSMyhft633LEZ-ZdryXmc6y6g.jar' is inaccessible.
Aug 25, 2021 3:21:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-25T15:21:47.899Z: Staged package xz-1.5-hvMPqHdfo6Ys2znR7XimAZFkwQWIZASNQsvuJE4m6EA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/xz-1.5-hvMPqHdfo6Ys2znR7XimAZFkwQWIZASNQsvuJE4m6EA.jar' is inaccessible.
Aug 25, 2021 3:21:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-08-25T15:21:47.967Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Aug 25, 2021 3:24:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-08-25T15:24:47.674Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Aug 25, 2021 3:27:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-25T15:27:47.795Z: Staged package slf4j-api-1.7.30-zboHlk0btAoHYUhcax6ML4_Z6x0ZxTkorA1_lRAQXFc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/slf4j-api-1.7.30-zboHlk0btAoHYUhcax6ML4_Z6x0ZxTkorA1_lRAQXFc.jar' is inaccessible.
Aug 25, 2021 3:27:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-25T15:27:47.841Z: Staged package slf4j-jdk14-1.7.30-4PnbBJN49kZ5QXcUVJlSMyhft633LEZ-ZdryXmc6y6g.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/slf4j-jdk14-1.7.30-4PnbBJN49kZ5QXcUVJlSMyhft633LEZ-ZdryXmc6y6g.jar' is inaccessible.
Aug 25, 2021 3:27:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-25T15:27:48.031Z: Staged package xz-1.5-hvMPqHdfo6Ys2znR7XimAZFkwQWIZASNQsvuJE4m6EA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/xz-1.5-hvMPqHdfo6Ys2znR7XimAZFkwQWIZASNQsvuJE4m6EA.jar' is inaccessible.
Aug 25, 2021 3:27:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-08-25T15:27:48.129Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Aug 25, 2021 3:30:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-08-25T15:30:48.639Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Aug 25, 2021 3:33:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-25T15:33:48.943Z: Staged package slf4j-api-1.7.30-zboHlk0btAoHYUhcax6ML4_Z6x0ZxTkorA1_lRAQXFc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/slf4j-api-1.7.30-zboHlk0btAoHYUhcax6ML4_Z6x0ZxTkorA1_lRAQXFc.jar' is inaccessible.
Aug 25, 2021 3:33:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-25T15:33:48.982Z: Staged package slf4j-jdk14-1.7.30-4PnbBJN49kZ5QXcUVJlSMyhft633LEZ-ZdryXmc6y6g.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/slf4j-jdk14-1.7.30-4PnbBJN49kZ5QXcUVJlSMyhft633LEZ-ZdryXmc6y6g.jar' is inaccessible.
Aug 25, 2021 3:33:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-25T15:33:49.126Z: Staged package xz-1.5-hvMPqHdfo6Ys2znR7XimAZFkwQWIZASNQsvuJE4m6EA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/xz-1.5-hvMPqHdfo6Ys2znR7XimAZFkwQWIZASNQsvuJE4m6EA.jar' is inaccessible.
Aug 25, 2021 3:33:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-08-25T15:33:49.177Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Aug 25, 2021 3:36:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-08-25T15:36:47.952Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Aug 25, 2021 3:39:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-25T15:39:47.113Z: Staged package slf4j-api-1.7.30-zboHlk0btAoHYUhcax6ML4_Z6x0ZxTkorA1_lRAQXFc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/slf4j-api-1.7.30-zboHlk0btAoHYUhcax6ML4_Z6x0ZxTkorA1_lRAQXFc.jar' is inaccessible.
Aug 25, 2021 3:39:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-25T15:39:47.161Z: Staged package slf4j-jdk14-1.7.30-4PnbBJN49kZ5QXcUVJlSMyhft633LEZ-ZdryXmc6y6g.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/slf4j-jdk14-1.7.30-4PnbBJN49kZ5QXcUVJlSMyhft633LEZ-ZdryXmc6y6g.jar' is inaccessible.
Aug 25, 2021 3:39:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-25T15:39:47.278Z: Staged package xz-1.5-hvMPqHdfo6Ys2znR7XimAZFkwQWIZASNQsvuJE4m6EA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/xz-1.5-hvMPqHdfo6Ys2znR7XimAZFkwQWIZASNQsvuJE4m6EA.jar' is inaccessible.
Aug 25, 2021 3:39:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-08-25T15:39:47.340Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Aug 25, 2021 3:42:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-08-25T15:42:47.819Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Aug 25, 2021 3:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-25T15:45:47.474Z: Staged package slf4j-api-1.7.30-zboHlk0btAoHYUhcax6ML4_Z6x0ZxTkorA1_lRAQXFc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/slf4j-api-1.7.30-zboHlk0btAoHYUhcax6ML4_Z6x0ZxTkorA1_lRAQXFc.jar' is inaccessible.
Aug 25, 2021 3:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-25T15:45:47.508Z: Staged package slf4j-jdk14-1.7.30-4PnbBJN49kZ5QXcUVJlSMyhft633LEZ-ZdryXmc6y6g.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/slf4j-jdk14-1.7.30-4PnbBJN49kZ5QXcUVJlSMyhft633LEZ-ZdryXmc6y6g.jar' is inaccessible.
Aug 25, 2021 3:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-25T15:45:47.634Z: Staged package xz-1.5-hvMPqHdfo6Ys2znR7XimAZFkwQWIZASNQsvuJE4m6EA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/xz-1.5-hvMPqHdfo6Ys2znR7XimAZFkwQWIZASNQsvuJE4m6EA.jar' is inaccessible.
Aug 25, 2021 3:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-08-25T15:45:47.696Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Aug 25, 2021 3:48:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-08-25T15:48:47.799Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Aug 25, 2021 3:51:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-25T15:51:48.341Z: Staged package slf4j-api-1.7.30-zboHlk0btAoHYUhcax6ML4_Z6x0ZxTkorA1_lRAQXFc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/slf4j-api-1.7.30-zboHlk0btAoHYUhcax6ML4_Z6x0ZxTkorA1_lRAQXFc.jar' is inaccessible.
Aug 25, 2021 3:51:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-25T15:51:48.380Z: Staged package slf4j-jdk14-1.7.30-4PnbBJN49kZ5QXcUVJlSMyhft633LEZ-ZdryXmc6y6g.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/slf4j-jdk14-1.7.30-4PnbBJN49kZ5QXcUVJlSMyhft633LEZ-ZdryXmc6y6g.jar' is inaccessible.
Aug 25, 2021 3:51:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-25T15:51:48.521Z: Staged package xz-1.5-hvMPqHdfo6Ys2znR7XimAZFkwQWIZASNQsvuJE4m6EA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/xz-1.5-hvMPqHdfo6Ys2znR7XimAZFkwQWIZASNQsvuJE4m6EA.jar' is inaccessible.
Aug 25, 2021 3:51:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-08-25T15:51:48.568Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Aug 25, 2021 3:54:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-08-25T15:54:47.699Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Aug 25, 2021 3:57:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-25T15:57:47.382Z: Staged package slf4j-api-1.7.30-zboHlk0btAoHYUhcax6ML4_Z6x0ZxTkorA1_lRAQXFc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/slf4j-api-1.7.30-zboHlk0btAoHYUhcax6ML4_Z6x0ZxTkorA1_lRAQXFc.jar' is inaccessible.
Aug 25, 2021 3:57:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-25T15:57:47.421Z: Staged package slf4j-jdk14-1.7.30-4PnbBJN49kZ5QXcUVJlSMyhft633LEZ-ZdryXmc6y6g.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/slf4j-jdk14-1.7.30-4PnbBJN49kZ5QXcUVJlSMyhft633LEZ-ZdryXmc6y6g.jar' is inaccessible.
Aug 25, 2021 3:57:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-25T15:57:47.571Z: Staged package xz-1.5-hvMPqHdfo6Ys2znR7XimAZFkwQWIZASNQsvuJE4m6EA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/xz-1.5-hvMPqHdfo6Ys2znR7XimAZFkwQWIZASNQsvuJE4m6EA.jar' is inaccessible.
Aug 25, 2021 3:57:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-08-25T15:57:47.618Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Aug 25, 2021 4:00:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-25T16:00:38.442Z: Cancel request is committed for workflow job: 2021-08-25_05_45_27-5858850660834944658.
Aug 25, 2021 4:00:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-25T16:00:38.468Z: Cleaning up.
Aug 25, 2021 4:00:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-25T16:00:38.553Z: Stopping **** pool...
Aug 25, 2021 4:00:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-25T16:00:38.609Z: Stopping **** pool...
Aug 25, 2021 4:03:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-25T16:03:02.896Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Aug 25, 2021 4:03:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-25T16:03:02.937Z: Worker pool stopped.
Aug 25, 2021 4:03:08 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-08-25_05_45_27-5858850660834944658 finished with status CANCELLED.
Load test results for test (ID): ad4e9baa-a3c8-4280-8a3f-767401ad2001 and timestamp: 2021-08-25T12:45:22.977000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11492.092
dataflow_v2_java11_total_bytes_count             3.13801542E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210825124333
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:f1f8e9ce6657bd645443d1e5ff7e120de3b2b6caa28cc9c7741ad872cdba093c
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210825124333]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:f1f8e9ce6657bd645443d1e5ff7e120de3b2b6caa28cc9c7741ad872cdba093c]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210825124333] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:f1f8e9ce6657bd645443d1e5ff7e120de3b2b6caa28cc9c7741ad872cdba093c])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:f1f8e9ce6657bd645443d1e5ff7e120de3b2b6caa28cc9c7741ad872cdba093c
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:f1f8e9ce6657bd645443d1e5ff7e120de3b2b6caa28cc9c7741ad872cdba093c
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:f1f8e9ce6657bd645443d1e5ff7e120de3b2b6caa28cc9c7741ad872cdba093c].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:aaad17f254e187846e932e141c5c463aacc03c6b9f003c6a887b44cc32bcd303
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:aaad17f254e187846e932e141c5c463aacc03c6b9f003c6a887b44cc32bcd303
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:aaad17f254e187846e932e141c5c463aacc03c6b9f003c6a887b44cc32bcd303].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 56s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/lxd5aj3azpifm

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #68

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/68/display/redirect?page=changes>

Changes:

[Andrew Pilloud] [BEAM-9379] Remove Java11 specific SQL timeout

[noreply] [BEAM-8571] Release Go SDK containers, and use them. (#15365)

[noreply] [BEAM-6374] Elide collecting unnecessary pcollection metrics (#15358)


------------------------------------------
[...truncated 50.04 KB...]
b19f17003e5a: Waiting
3ea4ac94ce6b: Pushed
97fee357ec38: Pushed
f0af1d760717: Pushed
434c1ba94934: Pushed
23d1c58dfd75: Pushed
1d5e8ebd07fc: Pushed
8d5401f65eb6: Pushed
084602bcd9c7: Pushed
b19f17003e5a: Layer already exists
75f6a0e6e441: Layer already exists
c2e2307780ac: Layer already exists
7d890913ab69: Layer already exists
c600ad69fb64: Pushed
1235daf38153: Layer already exists
23d0c9484ded: Pushed
62d14713f2e9: Layer already exists
c2ddc1bc2645: Layer already exists
d0ff60ccb51f: Pushed
ef4a45eb9910: Pushed
20210824124333: digest: sha256:e598a61a4b83d95609ba9ad85b86a6d6810e3606897cd299f5e6d5f81ee8be22 size: 4310

> Task :sdks:java:testing:load-tests:run
Aug 24, 2021 12:45:18 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Aug 24, 2021 12:45:19 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 191 files. Enable logging at DEBUG level to see which files will be staged.
Aug 24, 2021 12:45:19 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Aug 24, 2021 12:45:19 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Aug 24, 2021 12:45:21 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Aug 24, 2021 12:45:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <112622 bytes, hash bbd73da18c2bfe9d9e751d9e4e22acf98d7b5dc5cb52c13c4d22199ba0f73d6d> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-u9c9oYwr_p2edR2eTiKs-Y17XcXLUsE8TSIZm6D3PW0.pb
Aug 24, 2021 12:45:23 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 191 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Aug 24, 2021 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 191 files cached, 0 files newly uploaded in 0 seconds
Aug 24, 2021 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Aug 24, 2021 12:45:24 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12d5c30e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@b887730, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26586b74, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@52f57666, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6e041285, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@267dc982, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@439b15f2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3aa41da1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@74fab04a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c83468e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@272ce069, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3d7fb838, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1c26273d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3a37a501, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7a583586, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3a26ec8d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@17143b3b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@100c8b75, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2bc378f7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@268cbb86]
Aug 24, 2021 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Aug 24, 2021 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Aug 24, 2021 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Aug 24, 2021 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Aug 24, 2021 12:45:24 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@20d87335, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2a8a4e0c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26c89563, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3bd6ba24, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58f437b0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@20f6f88c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4277127c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4c7e978c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@354e7004, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@22ebccb9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5875de6a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@608fe01f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2687725a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5a75ec37, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2c05ff9d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@33956d1a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2e1ddc90, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5b7c8930, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58867cd5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@40e32762]
Aug 24, 2021 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Aug 24, 2021 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Aug 24, 2021 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Aug 24, 2021 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Aug 24, 2021 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Aug 24, 2021 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Aug 24, 2021 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Aug 24, 2021 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Aug 24, 2021 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Aug 24, 2021 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Aug 24, 2021 12:45:24 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Aug 24, 2021 12:45:24 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.33.0-SNAPSHOT
Aug 24, 2021 12:45:25 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-08-24_05_45_24-6103816021493748310?project=apache-beam-testing
Aug 24, 2021 12:45:25 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-08-24_05_45_24-6103816021493748310
Aug 24, 2021 12:45:25 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-08-24_05_45_24-6103816021493748310
Aug 24, 2021 12:45:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-08-24T12:45:32.582Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-08-9inc. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Aug 24, 2021 12:45:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-24T12:45:37.899Z: Worker configuration: e2-standard-2 in us-central1-a.
Aug 24, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-24T12:45:38.576Z: Expanding SplittableParDo operations into optimizable parts.
Aug 24, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-24T12:45:38.620Z: Expanding CollectionToSingleton operations into optimizable parts.
Aug 24, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-24T12:45:38.688Z: Expanding CoGroupByKey operations into optimizable parts.
Aug 24, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-24T12:45:38.786Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Aug 24, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-24T12:45:38.829Z: Expanding GroupByKey operations into streaming Read/Write steps
Aug 24, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-24T12:45:38.907Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Aug 24, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-24T12:45:39.023Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Aug 24, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-24T12:45:39.055Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Aug 24, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-24T12:45:39.091Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Aug 24, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-24T12:45:39.123Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Aug 24, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-24T12:45:39.159Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Aug 24, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-24T12:45:39.188Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Aug 24, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-24T12:45:39.228Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Aug 24, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-24T12:45:39.272Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Aug 24, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-24T12:45:39.364Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Aug 24, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-24T12:45:39.419Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Aug 24, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-24T12:45:39.456Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Aug 24, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-24T12:45:39.500Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Aug 24, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-24T12:45:39.536Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Aug 24, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-24T12:45:39.573Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Aug 24, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-24T12:45:39.613Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Aug 24, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-24T12:45:39.642Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Aug 24, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-24T12:45:39.676Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Aug 24, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-24T12:45:39.713Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Aug 24, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-24T12:45:39.753Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Aug 24, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-24T12:45:39.784Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Aug 24, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-24T12:45:39.823Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Aug 24, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-24T12:45:39.856Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Aug 24, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-24T12:45:39.892Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Aug 24, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-24T12:45:40.323Z: Starting 5 ****s in us-central1-a...
Aug 24, 2021 12:45:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-24T12:45:50.980Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Aug 24, 2021 12:46:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-24T12:46:25.316Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Aug 24, 2021 12:47:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-24T12:47:20.167Z: Workers have started successfully.
Aug 24, 2021 12:47:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-24T12:47:20.202Z: Workers have started successfully.
Aug 24, 2021 2:33:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-24T14:33:40.513Z: Staged package amazon-kinesis-producer-0.14.1-5YYT_oyDFV17VWs8Of43O7N08sRfHu0SuJeqVSFCuLg.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/amazon-kinesis-producer-0.14.1-5YYT_oyDFV17VWs8Of43O7N08sRfHu0SuJeqVSFCuLg.jar' is inaccessible.
Aug 24, 2021 2:33:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-08-24T14:33:44.471Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Aug 24, 2021 4:00:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-24T16:00:47.595Z: Cancel request is committed for workflow job: 2021-08-24_05_45_24-6103816021493748310.
Aug 24, 2021 4:00:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-24T16:00:47.646Z: Cleaning up.
Aug 24, 2021 4:00:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-24T16:00:47.701Z: Stopping **** pool...
Aug 24, 2021 4:00:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-24T16:00:47.752Z: Stopping **** pool...
Aug 24, 2021 4:03:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-24T16:03:00.623Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Aug 24, 2021 4:03:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-24T16:03:00.666Z: Worker pool stopped.
Aug 24, 2021 4:03:06 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-08-24_05_45_24-6103816021493748310 finished with status CANCELLED.
Load test results for test (ID): 060d2de7-ab57-4ffe-b865-28269ee9944d and timestamp: 2021-08-24T12:45:19.599000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11564.041
dataflow_v2_java11_total_bytes_count             2.21424885E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210824124333
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:e598a61a4b83d95609ba9ad85b86a6d6810e3606897cd299f5e6d5f81ee8be22
Deleted: sha256:fd772bef1b512192f4fdd84c349bda5e98658d7530ccf1a7731fd63fa484077e
Deleted: sha256:e625a9fca2a47661232be43c717c8e1f81f5232690cea56d5d3cb44873fb595c
Deleted: sha256:55b135f0ba24fb8eb48d092764f6c8baa1bf4c9a95f6b8066f85a2b1a082aeee
Deleted: sha256:a8b1419def86131bdffb9f4e35ca1d947564402bc6aa5cf1822fd5d9d6a767c5
Deleted: sha256:32fae4ff9b5a40580626c0ac61d1b57f369ed60c79d40bce5124ce7d0a772f73
Deleted: sha256:e4b7c091e7ceacfd2a5ac19df8426f027585b1e33ee1dcde8ef47a1752fc9660
Deleted: sha256:0702e28a835747334a53b72b5da263d29e25a24026452e334b5759cb30e8b722
Deleted: sha256:d062bbc974efeb196145ab71e2cb4431b679d371e88fe90729f77bfd6b9bd824
Deleted: sha256:2d1344a22b872dc808ebb3d321835807cce537a2c68ae0e78c5bd42a2b4914e2
Deleted: sha256:8eabfb1b158a4c71fba9fa94fa25fe2b2c5aac0b2660fa571764b57558091ec9
Deleted: sha256:ef86fb9a6a45fcbbfa62d3a700907b54f9987a8e07d5a3ba6636c18aa399fdf6
Deleted: sha256:ba02b4854e2eef28ac96bf49673d416973e96340d4db5a7efcd5f84c65b48a47
Deleted: sha256:f756d38c34fba784956a94b7dd9761a8ab5f3a03233879e70d3b49a0a7099b47
Deleted: sha256:66b2d87acd1d571edb9964ae30e8bf39ddb972a7d206546d512365f5034c9e7b
Deleted: sha256:ce9d59e5520c1947e7de4c286417631c1805dd015f360fae5c455e3231792c8f
Deleted: sha256:e2648c71a6a86eae2883604a0fec44f9f9c52d5ab49a54cfdea5fec4e6540cf5
Deleted: sha256:0a566f311c8ea03b5ca380e8d91591373a83f583e310441b2ef9e6d878645936
Deleted: sha256:6849bfe7a625932cf0e1df2d356291abcece79fe6c9f9bdb5e9e8631cda2330e
Deleted: sha256:4602c4103a92262b82ca1b3fde381696455dbc1bbf7528ff74785f29a24902cc
Deleted: sha256:2f6076bbdb9e6763ee22c69791f8f439b8af4d5fad42db423e455134cf155276
Deleted: sha256:655f31e2163bb0b3216f2cbfd373a7f147d73aa225f8455cdfcbd12cc44e73d6
Deleted: sha256:47118143cf4561ce9a90c648a7fe3058a0754050001e74bd9c340d966708a1ee
Deleted: sha256:0f44847e38cfc0625a9fc41bb57f30d56094d8dd59993fa7f1f21d68d4cef2ab
Deleted: sha256:a2b5918fa2dd42a4990ebc735274b8e7b2eca4f1581d8b85b77727811a5da5db
Deleted: sha256:d40f1889c8f9c50f071a5298b1bd375118e663b187eea40c992556874307c49f
Deleted: sha256:ee5aafe89b621eb147926eaa1b30d63414cfd5869bd8e0860619424297fa41b0
Deleted: sha256:e543ae85ff28fa03205d8c86eba36cab0c47486323a4cea2cbd6dced800ab40d
Deleted: sha256:85797e2b8869c2f86a042b71e95a52ae93ba4388b444c815bcb5add986e9c9ec
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210824124333]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:e598a61a4b83d95609ba9ad85b86a6d6810e3606897cd299f5e6d5f81ee8be22]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210824124333] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:e598a61a4b83d95609ba9ad85b86a6d6810e3606897cd299f5e6d5f81ee8be22])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:e598a61a4b83d95609ba9ad85b86a6d6810e3606897cd299f5e6d5f81ee8be22
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:e598a61a4b83d95609ba9ad85b86a6d6810e3606897cd299f5e6d5f81ee8be22
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:e598a61a4b83d95609ba9ad85b86a6d6810e3606897cd299f5e6d5f81ee8be22].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 58s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/zknzepm2yh5eg

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #67

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/67/display/redirect>

Changes:


------------------------------------------
[...truncated 49.79 KB...]
c2e2307780ac: Waiting
377ad4dbae2d: Pushed
6879bc0a1fa3: Pushed
975db93699d5: Pushed
5c3a30e49a10: Pushed
5da6f9924b4a: Pushed
c4823eba9f0b: Pushed
73eb75a92a35: Pushed
711fa27e728f: Pushed
b19f17003e5a: Layer already exists
75f6a0e6e441: Layer already exists
c2e2307780ac: Layer already exists
5e569e8acb08: Pushed
7d890913ab69: Layer already exists
9fe4e2681ddc: Pushed
1235daf38153: Layer already exists
62d14713f2e9: Layer already exists
c2ddc1bc2645: Layer already exists
f41f7eab8c26: Pushed
3e8b3b0bf5cb: Pushed
20210823124336: digest: sha256:b7ce138a4c88df32ea59abf150995f6ba9a67156734140617dfc5f7059c6af97 size: 4310

> Task :sdks:java:testing:load-tests:run
Aug 23, 2021 12:45:39 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Aug 23, 2021 12:45:39 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 191 files. Enable logging at DEBUG level to see which files will be staged.
Aug 23, 2021 12:45:40 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Aug 23, 2021 12:45:40 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Aug 23, 2021 12:45:42 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Aug 23, 2021 12:45:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <112622 bytes, hash 9789089a00c37cf552733f02570f615e8195a93989f38ca11e5b8501cdb34558> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-l4kImgDDfPVScz8CVw9hXoGVqTmJ84yhHluFAc2zRVg.pb
Aug 23, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 191 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Aug 23, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 191 files cached, 0 files newly uploaded in 0 seconds
Aug 23, 2021 12:45:45 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Aug 23, 2021 12:45:45 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12d5c30e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@b887730, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26586b74, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@52f57666, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6e041285, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@267dc982, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@439b15f2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3aa41da1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@74fab04a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c83468e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@272ce069, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3d7fb838, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1c26273d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3a37a501, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7a583586, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3a26ec8d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@17143b3b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@100c8b75, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2bc378f7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@268cbb86]
Aug 23, 2021 12:45:45 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Aug 23, 2021 12:45:45 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Aug 23, 2021 12:45:45 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Aug 23, 2021 12:45:45 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Aug 23, 2021 12:45:45 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2d4608a6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@20d87335, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2a8a4e0c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26c89563, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3bd6ba24, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58f437b0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@20f6f88c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4277127c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4c7e978c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@354e7004, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@22ebccb9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5875de6a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@608fe01f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2687725a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5a75ec37, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2c05ff9d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@33956d1a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2e1ddc90, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5b7c8930, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58867cd5]
Aug 23, 2021 12:45:45 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Aug 23, 2021 12:45:45 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Aug 23, 2021 12:45:45 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Aug 23, 2021 12:45:45 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Aug 23, 2021 12:45:45 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Aug 23, 2021 12:45:45 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Aug 23, 2021 12:45:45 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Aug 23, 2021 12:45:45 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Aug 23, 2021 12:45:45 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Aug 23, 2021 12:45:45 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Aug 23, 2021 12:45:45 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Aug 23, 2021 12:45:45 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.33.0-SNAPSHOT
Aug 23, 2021 12:45:46 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-08-23_05_45_45-5928918039248961525?project=apache-beam-testing
Aug 23, 2021 12:45:46 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-08-23_05_45_45-5928918039248961525
Aug 23, 2021 12:45:46 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-08-23_05_45_45-5928918039248961525
Aug 23, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-08-23T12:45:54.997Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-08-gnt9. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Aug 23, 2021 12:46:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-23T12:46:00.758Z: Worker configuration: e2-standard-2 in us-central1-a.
Aug 23, 2021 12:46:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-23T12:46:01.482Z: Expanding SplittableParDo operations into optimizable parts.
Aug 23, 2021 12:46:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-23T12:46:01.511Z: Expanding CollectionToSingleton operations into optimizable parts.
Aug 23, 2021 12:46:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-23T12:46:01.564Z: Expanding CoGroupByKey operations into optimizable parts.
Aug 23, 2021 12:46:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-23T12:46:01.726Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Aug 23, 2021 12:46:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-23T12:46:01.758Z: Expanding GroupByKey operations into streaming Read/Write steps
Aug 23, 2021 12:46:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-23T12:46:01.829Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Aug 23, 2021 12:46:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-23T12:46:01.931Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Aug 23, 2021 12:46:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-23T12:46:01.961Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Aug 23, 2021 12:46:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-23T12:46:01.999Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Aug 23, 2021 12:46:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-23T12:46:02.032Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Aug 23, 2021 12:46:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-23T12:46:02.063Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Aug 23, 2021 12:46:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-23T12:46:02.099Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Aug 23, 2021 12:46:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-23T12:46:02.131Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Aug 23, 2021 12:46:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-23T12:46:02.165Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Aug 23, 2021 12:46:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-23T12:46:02.193Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Aug 23, 2021 12:46:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-23T12:46:02.236Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Aug 23, 2021 12:46:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-23T12:46:02.263Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Aug 23, 2021 12:46:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-23T12:46:02.300Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Aug 23, 2021 12:46:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-23T12:46:02.328Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Aug 23, 2021 12:46:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-23T12:46:02.364Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Aug 23, 2021 12:46:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-23T12:46:02.387Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Aug 23, 2021 12:46:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-23T12:46:02.416Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Aug 23, 2021 12:46:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-23T12:46:02.438Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Aug 23, 2021 12:46:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-23T12:46:02.474Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Aug 23, 2021 12:46:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-23T12:46:02.497Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Aug 23, 2021 12:46:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-23T12:46:02.529Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Aug 23, 2021 12:46:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-23T12:46:02.559Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Aug 23, 2021 12:46:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-23T12:46:02.605Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Aug 23, 2021 12:46:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-23T12:46:02.633Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Aug 23, 2021 12:46:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-23T12:46:03.006Z: Starting 5 ****s in us-central1-a...
Aug 23, 2021 12:46:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-23T12:46:13.184Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Aug 23, 2021 12:46:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-23T12:46:37.364Z: Autoscaling: Raised the number of ****s to 1 so that the pipeline can catch up with its backlog and keep up with its input rate.
Aug 23, 2021 12:46:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-23T12:46:37.388Z: Resized **** pool to 1, though goal was 5.  This could be a quota issue.
Aug 23, 2021 12:46:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-23T12:46:47.626Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Aug 23, 2021 12:47:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-23T12:47:32.809Z: Workers have started successfully.
Aug 23, 2021 12:47:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-23T12:47:32.844Z: Workers have started successfully.
Aug 23, 2021 4:00:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-23T16:00:28.439Z: Cancel request is committed for workflow job: 2021-08-23_05_45_45-5928918039248961525.
Aug 23, 2021 4:00:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-23T16:00:28.532Z: Cleaning up.
Aug 23, 2021 4:00:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-23T16:00:28.611Z: Stopping **** pool...
Aug 23, 2021 4:00:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-23T16:00:28.695Z: Stopping **** pool...
Aug 23, 2021 4:02:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-23T16:02:49.271Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Aug 23, 2021 4:02:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-23T16:02:49.312Z: Worker pool stopped.
Aug 23, 2021 4:02:55 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-08-23_05_45_45-5928918039248961525 finished with status CANCELLED.
Load test results for test (ID): e8a57fdf-bdac-487c-b7a6-5142d6de9469 and timestamp: 2021-08-23T12:45:40.087000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11522.779
dataflow_v2_java11_total_bytes_count             2.25241781E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210823124336
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:b7ce138a4c88df32ea59abf150995f6ba9a67156734140617dfc5f7059c6af97
Deleted: sha256:ce51550d5797cc638206a95bd82ef7b9764f2618c2c562bbb05721f2fd64146b
Deleted: sha256:9414992d1b68b1d7945bd12411691d745847c697a4648806b116a2b97707683e
Deleted: sha256:3378a3e75b87dc86accd93a34eb3ee94a264d9aafab3c9bd0d270008c60a0d12
Deleted: sha256:00f7cbc75a76b94799e0df55881cc1cfcb3f62f953a673ca587a709bb656fc44
Deleted: sha256:ed26891b4a96fe36e5b3f5e440fbeffd86a7d07a68150f734c42028659f35c57
Deleted: sha256:fc0e8db9ea79335f5c4906fe434bf5068d2197303f0e2071375239740cbcb337
Deleted: sha256:5163f3f2817d7b220f15b1dc588ab32df48c35f1bc2408c6bfbe3b079028f5f7
Deleted: sha256:66f75ae4b094ac1a26a91621203fdd57d5e075d102520f4a57b7c3a2c665e93c
Deleted: sha256:e0d2ac984dd7d61a115aac10df84554a062921da0c2bc7553cf13f8cbcc41d60
Deleted: sha256:422e51ccec278f345ef282016c9d7802decdabc81f15b5e844317fa7831b4b6f
Deleted: sha256:cf290b8aadd4ea74e0087fe9029767c99fdb79f3fa2a3aee4b68603c78aec0b2
Deleted: sha256:797c82b477af206de28e83dc53283b8ec01ab8c9c8e9076208edc3a92ca0052b
Deleted: sha256:b10d5118502eab85a272dd525d34958e96b48b9a21be96dfaf84cb86d15c4739
Deleted: sha256:d25380aacc454f00df587eff6ab4f4593482bcd11fbcb854126035f2242f9225
Deleted: sha256:17b0002000c74cd0aafbd17820ab0ae51a8f7497a756743cb80682cc58e5b22e
Deleted: sha256:8b30bbd9c8c84fff73c929f9e0e38977492e672227d7090fa05ca491d4a1a47b
Deleted: sha256:2d2dde0010f68357472675a0d246422ceed8048cd630e2027cbee175e1090732
Deleted: sha256:a0f5854d88eaf11ee835a2262b5bb5797276e6d4608ef381d09d7a153e949e2e
Deleted: sha256:69b3dae843acdd4cb04b4f034a4a65e805694db76c488d9e172e3f1a629c4d7a
Deleted: sha256:049e6fac5f46ec267e5e01315ad9bf5c660cd17fa933b29ea7c358ebb54afdeb
Deleted: sha256:a2fd1603c7bd393a6450956c91046d570dc6af85dd423fd6f8dcb66cf3842285
Deleted: sha256:412e3439e6f8c42071de72ece2c1e4240ae163db112007e01104fe7427a0d2cc
Deleted: sha256:51856036bdafbf0f2d8cc3f56b2ada99b5fd03217507158d5911261cc0ec84f9
Deleted: sha256:486319b4970567b05b154baf8dc20d55cc235d1bfaa8e6dd3fd387a3f380ef3c
Deleted: sha256:2dfdb7c7fe0afb13e85b92882a5236939d1c757ad94929cf660a67728b78b150
Deleted: sha256:bca2185b74dc49335e4556af8bc4ba0e72002fa72b7c22bce3e9d22ce5e1c38d
Deleted: sha256:0a8ef708d5d512b14196248b1965bdd893afb840874f149ef1fd8e5522e15058
Deleted: sha256:cffe64390a479a8ecd433de14ad95a45ba66e4778c395a0607116ff4e65fe6c3
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210823124336]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:b7ce138a4c88df32ea59abf150995f6ba9a67156734140617dfc5f7059c6af97]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210823124336] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:b7ce138a4c88df32ea59abf150995f6ba9a67156734140617dfc5f7059c6af97])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:b7ce138a4c88df32ea59abf150995f6ba9a67156734140617dfc5f7059c6af97
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:b7ce138a4c88df32ea59abf150995f6ba9a67156734140617dfc5f7059c6af97
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:b7ce138a4c88df32ea59abf150995f6ba9a67156734140617dfc5f7059c6af97].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 39s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/vvligfpxvxhpy

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 - Build # 66 - Aborted!

Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 - Build # 66 - Aborted:

Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/66/ to view the results.

Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #65

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/65/display/redirect?page=changes>

Changes:

[samuelw] [BEAM-12780] StreamingDataflowWorker should only retry exceptions

[zhoufek] [BEAM-9487] Disable allowing unsafe triggers by default

[noreply] [BEAM-10917] Add support for BigQuery Read API in Python BEAM (#15185)

[ryanthompson591] Change filter to also retry on 408 errors.

[noreply] Avoid spamming lull logging (#15366)


------------------------------------------
[...truncated 48.99 KB...]
7d890913ab69: Waiting
c2ddc1bc2645: Waiting
d5ae9d8c899a: Waiting
e5b668488759: Waiting
1235daf38153: Waiting
f5c1383a6762: Pushed
1ddc9b629565: Pushed
2bba9d568e57: Pushed
f0e17738635e: Pushed
a9c1ee0d5132: Pushed
1794e1630dca: Pushed
d5ae9d8c899a: Pushed
25961f454464: Pushed
b19f17003e5a: Layer already exists
75f6a0e6e441: Layer already exists
c2e2307780ac: Layer already exists
7d890913ab69: Layer already exists
1235daf38153: Layer already exists
62d14713f2e9: Layer already exists
aa3b2f84eb8e: Pushed
c2ddc1bc2645: Layer already exists
e5b668488759: Pushed
5e4d8d30be0f: Pushed
b879b082a915: Pushed
20210821124332: digest: sha256:3251210857cf46d6fe0c344715d4b25913fa55fa161d5fe7009c83ea7004b1ec size: 4310

> Task :sdks:java:testing:load-tests:run
Aug 21, 2021 12:45:24 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Aug 21, 2021 12:45:25 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 191 files. Enable logging at DEBUG level to see which files will be staged.
Aug 21, 2021 12:45:25 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Aug 21, 2021 12:45:25 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Aug 21, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Aug 21, 2021 12:45:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <112684 bytes, hash e9cf351b581d4cb0f6c313c5b619882abd0f7bf23888caf7c080304b7a49933a> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-6c81G1gdTLD2wxPFthmIKr0Pe_I4iMr3wIAwS3pJkzo.pb
Aug 21, 2021 12:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 191 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Aug 21, 2021 12:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 191 files cached, 0 files newly uploaded in 0 seconds
Aug 21, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Aug 21, 2021 12:45:31 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12d5c30e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@b887730, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26586b74, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@52f57666, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6e041285, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@267dc982, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@439b15f2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3aa41da1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@74fab04a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c83468e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@272ce069, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3d7fb838, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1c26273d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3a37a501, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7a583586, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3a26ec8d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@17143b3b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@100c8b75, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2bc378f7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@268cbb86]
Aug 21, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Aug 21, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Aug 21, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Aug 21, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Aug 21, 2021 12:45:31 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@20d87335, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2a8a4e0c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26c89563, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3bd6ba24, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58f437b0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@20f6f88c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4277127c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4c7e978c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@354e7004, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@22ebccb9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5875de6a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@608fe01f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2687725a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5a75ec37, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2c05ff9d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@33956d1a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2e1ddc90, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5b7c8930, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58867cd5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@40e32762]
Aug 21, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Aug 21, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Aug 21, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Aug 21, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Aug 21, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Aug 21, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Aug 21, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Aug 21, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Aug 21, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Aug 21, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Aug 21, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Aug 21, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.33.0-SNAPSHOT
Aug 21, 2021 12:45:32 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-08-21_05_45_31-8609495736697665091?project=apache-beam-testing
Aug 21, 2021 12:45:32 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-08-21_05_45_31-8609495736697665091
Aug 21, 2021 12:45:32 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-08-21_05_45_31-8609495736697665091
Aug 21, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-08-21T12:45:41.077Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-08-e7nf. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Aug 21, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-21T12:45:45.823Z: Worker configuration: e2-standard-2 in us-central1-a.
Aug 21, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-21T12:45:46.500Z: Expanding SplittableParDo operations into optimizable parts.
Aug 21, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-21T12:45:46.530Z: Expanding CollectionToSingleton operations into optimizable parts.
Aug 21, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-21T12:45:46.599Z: Expanding CoGroupByKey operations into optimizable parts.
Aug 21, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-21T12:45:46.687Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Aug 21, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-21T12:45:46.718Z: Expanding GroupByKey operations into streaming Read/Write steps
Aug 21, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-21T12:45:46.789Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Aug 21, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-21T12:45:46.922Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Aug 21, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-21T12:45:46.967Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Aug 21, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-21T12:45:47.037Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Aug 21, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-21T12:45:47.074Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Aug 21, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-21T12:45:47.118Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Aug 21, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-21T12:45:47.144Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Aug 21, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-21T12:45:47.183Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Aug 21, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-21T12:45:47.216Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Aug 21, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-21T12:45:47.249Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Aug 21, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-21T12:45:47.282Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Aug 21, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-21T12:45:47.312Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Aug 21, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-21T12:45:47.349Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Aug 21, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-21T12:45:47.387Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Aug 21, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-21T12:45:47.407Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Aug 21, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-21T12:45:47.447Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Aug 21, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-21T12:45:47.473Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Aug 21, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-21T12:45:47.514Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Aug 21, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-21T12:45:47.550Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Aug 21, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-21T12:45:47.587Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Aug 21, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-21T12:45:47.620Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Aug 21, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-21T12:45:47.643Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Aug 21, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-21T12:45:47.678Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Aug 21, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-21T12:45:47.708Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Aug 21, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-21T12:45:48.098Z: Starting 5 ****s in us-central1-a...
Aug 21, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-21T12:46:00.140Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Aug 21, 2021 12:46:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-21T12:46:34.011Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Aug 21, 2021 12:47:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-21T12:47:29.165Z: Workers have started successfully.
Aug 21, 2021 12:47:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-21T12:47:29.203Z: Workers have started successfully.
Aug 21, 2021 4:00:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-21T16:00:32.240Z: Cancel request is committed for workflow job: 2021-08-21_05_45_31-8609495736697665091.
Aug 21, 2021 4:00:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-21T16:00:32.333Z: Cleaning up.
Aug 21, 2021 4:00:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-21T16:00:32.428Z: Stopping **** pool...
Aug 21, 2021 4:00:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-21T16:00:32.482Z: Stopping **** pool...
Aug 21, 2021 4:02:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-21T16:02:56.625Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Aug 21, 2021 4:02:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-21T16:02:56.694Z: Worker pool stopped.
Aug 21, 2021 4:03:02 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-08-21_05_45_31-8609495736697665091 finished with status CANCELLED.
Load test results for test (ID): 2407257e-4292-4dab-96c8-a6b3bd9fb6f0 and timestamp: 2021-08-21T12:45:25.311000000Z:
Exception in thread "main"                  Metric:                    Value:
java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)
dataflow_v2_java11_runtime_sec                 11551.496
dataflow_v2_java11_total_bytes_count             2.69195971E10

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210821124332
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:3251210857cf46d6fe0c344715d4b25913fa55fa161d5fe7009c83ea7004b1ec
Deleted: sha256:93322953fbad0e896c6dd6e0c1a29a50b236927358ea4dae5b6a848bc6af55a4
Deleted: sha256:0208d11cda6cba15fe9481bb9c645128a2b813f778fe0fd2cc6eb5fe1cd12426
Deleted: sha256:5d3aa3bf1830a7c672d109151175146289054196c5f804c9cd012811441027b5
Deleted: sha256:5ac4a8b8ea059f05c349c4d80b2345eb5cdf667f761c788564c45cc291e7c9fe
Deleted: sha256:5ba974e6d551764c0ffa4f5b5a8b3722d40bf839a9e722216aadc2f0341841f8
Deleted: sha256:7036897278ad1566eb070221c2af71977643c1cf8c1d993fb67aa420d9193bb2
Deleted: sha256:f19a4ab5cefe0c433a5a5016114f01095cbe3aa2a7e385aae28eb64878cff744
Deleted: sha256:8601a40efd30747207baa7f81242b13d8934ba6236bf42fdf7416bfeb9522cd1
Deleted: sha256:91e461070f45c1a5149f416310a8d5d0d631b4f3962134a0a06215624206bc74
Deleted: sha256:52dfb6bf393461af30fbeb4f1adfd9312b6347b46499cae83693584c7fd7b437
Deleted: sha256:78c9100171f38f9c2262859660919d25dfb1b7de208b48a6e7dfe3efd1ea9a2c
Deleted: sha256:643f5ee67b8e9c993040f79b73549bb8df12c4e5a095450637fdadd568292139
Deleted: sha256:0115bc6371145f1302a2c840dbff6ff9ffbebb3affe359bff5831e510f7a362f
Deleted: sha256:7f1b06df913bc34800c08ce32c8cddd323980ebe5d9dff6def36c536681c6338
Deleted: sha256:dc7ce8b4dd5c65801d9059861a9949e4cbf38192cbcb0bf54ed1969febae4027
Deleted: sha256:a3f195deb80c428ef52bbca4febb81ded383327d1f9fbff95db14419aa005654
Deleted: sha256:b2e56356f02e7cfcc478df95ddfc0033047e50d583ea525d764b46acb337a400
Deleted: sha256:191aa72634575f77444158428a442071b1de06d558d12cc20d2fb98a8f21a73d
Deleted: sha256:2d630603b2a9b3e9df9f385c995863c3a30dcd47422bc6d6e8f84a934a52c657
Deleted: sha256:6f029c03dcf216a60c362b5f34fc686c8e03842b13896f762221635966990716
Deleted: sha256:a599bece7ad6077a6091bd35b4e60c939449a969376c9f7f22447441f9c6f8b6
Deleted: sha256:e1869621995783e5bf9254dac6effaebea02a06865bfda492ee2c8ef0a6c796e
Deleted: sha256:a8e93d710e860d5bf2d47b0ee4786b1f50d6eb13a55637af2194c1bfb4031d70
Deleted: sha256:cd220277b20473bd6395fa7fcfe8267f45e9983f093e0790d4e9446c4b882058
Deleted: sha256:146b331712f2ff95975be4513ab33e2593291058097cc47f2dafb82b15f921c9
Deleted: sha256:750d888fcb722a86816376e70997229b329f3ba6c439fba7904b7bbe23cc08cd
Deleted: sha256:69f81ada2636a8203c2fec52f0dba46fa33c5ced8dfce4e659749f3b3c6bf6e2
Deleted: sha256:3187dd3b1b87e5607bd0576977bb154bfbab6c9de93aab839fba9563c2b16eae
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210821124332]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:3251210857cf46d6fe0c344715d4b25913fa55fa161d5fe7009c83ea7004b1ec]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210821124332] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:3251210857cf46d6fe0c344715d4b25913fa55fa161d5fe7009c83ea7004b1ec])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:3251210857cf46d6fe0c344715d4b25913fa55fa161d5fe7009c83ea7004b1ec
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:3251210857cf46d6fe0c344715d4b25913fa55fa161d5fe7009c83ea7004b1ec
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:3251210857cf46d6fe0c344715d4b25913fa55fa161d5fe7009c83ea7004b1ec].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 47s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/5nnbo3abwao62

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #64

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/64/display/redirect?page=changes>

Changes:

[Steve Niemitz] [BEAM-12754] Only call getValue once per field per row

[ryanthompson591] Removes test_bad_path

[noreply] Fix broken BQ Integration Test  (#15352)

[noreply] Change conflicting StateReader name from side to reader (#15348)

[noreply] [BEAM-3304] Go triggering support (#15239)


------------------------------------------
[...truncated 49.45 KB...]
809234d56701: Waiting
1235daf38153: Preparing
62d14713f2e9: Preparing
6c22941a478a: Waiting
c2ddc1bc2645: Preparing
31dfa412cd03: Waiting
80968bc6d054: Waiting
c65759ec9c45: Waiting
c3503ad266a4: Waiting
fbc09633f421: Waiting
b19f17003e5a: Waiting
c2ddc1bc2645: Waiting
62d14713f2e9: Waiting
c2e2307780ac: Waiting
75f6a0e6e441: Waiting
7d890913ab69: Waiting
1235daf38153: Waiting
48ab9f2e5ccd: Pushed
c3b95bf0be1e: Pushed
45d2a6d733b0: Pushed
5cc2600ee12a: Pushed
9fa213fd21b4: Pushed
809234d56701: Pushed
c3503ad266a4: Pushed
80968bc6d054: Pushed
b19f17003e5a: Layer already exists
75f6a0e6e441: Layer already exists
c2e2307780ac: Layer already exists
7d890913ab69: Layer already exists
31dfa412cd03: Pushed
1235daf38153: Layer already exists
62d14713f2e9: Layer already exists
c2ddc1bc2645: Layer already exists
c65759ec9c45: Pushed
fbc09633f421: Pushed
6c22941a478a: Pushed
20210820124331: digest: sha256:2554867c54eff75e911fbb95311d048c6e4c32984f02966ec4abb8a516b94f83 size: 4310

> Task :sdks:java:testing:load-tests:run
Aug 20, 2021 12:45:25 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Aug 20, 2021 12:45:25 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 191 files. Enable logging at DEBUG level to see which files will be staged.
Aug 20, 2021 12:45:26 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Aug 20, 2021 12:45:26 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Aug 20, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Aug 20, 2021 12:45:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <112622 bytes, hash 08ff4aa6b0afb9a42f6becb611622fc83e3242d5000df44ed119c6e2709c41ce> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-CP9KprCvuaQva-y2EWIvyD4yQtUADfRO0RnG4nCcQc4.pb
Aug 20, 2021 12:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 191 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Aug 20, 2021 12:45:30 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 191 files cached, 0 files newly uploaded in 0 seconds
Aug 20, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Aug 20, 2021 12:45:30 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@439b15f2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3aa41da1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@74fab04a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c83468e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@272ce069, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3d7fb838, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1c26273d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3a37a501, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7a583586, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3a26ec8d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@17143b3b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@100c8b75, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2bc378f7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@268cbb86, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@10f7918f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@64d4f7c7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@54e02f6a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@17f3eefb, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ba46e63, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@788ddc1f]
Aug 20, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Aug 20, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Aug 20, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Aug 20, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Aug 20, 2021 12:45:30 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4277127c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4c7e978c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@354e7004, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@22ebccb9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5875de6a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@608fe01f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2687725a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5a75ec37, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2c05ff9d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@33956d1a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2e1ddc90, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5b7c8930, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58867cd5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@40e32762, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5a8ab2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3f50b680, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@54db056b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@d5af0a5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5981f4a6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@63dfada0]
Aug 20, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Aug 20, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Aug 20, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Aug 20, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Aug 20, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Aug 20, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Aug 20, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Aug 20, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Aug 20, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Aug 20, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Aug 20, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Aug 20, 2021 12:45:30 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.33.0-SNAPSHOT
Aug 20, 2021 12:45:32 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-08-20_05_45_30-16385337836395962967?project=apache-beam-testing
Aug 20, 2021 12:45:32 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-08-20_05_45_30-16385337836395962967
Aug 20, 2021 12:45:32 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-08-20_05_45_30-16385337836395962967
Aug 20, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-08-20T12:45:48.576Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-08-x9v0. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Aug 20, 2021 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-20T12:45:53.193Z: Worker configuration: e2-standard-2 in us-central1-a.
Aug 20, 2021 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-20T12:45:54.021Z: Expanding SplittableParDo operations into optimizable parts.
Aug 20, 2021 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-20T12:45:54.044Z: Expanding CollectionToSingleton operations into optimizable parts.
Aug 20, 2021 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-20T12:45:54.145Z: Expanding CoGroupByKey operations into optimizable parts.
Aug 20, 2021 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-20T12:45:54.217Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Aug 20, 2021 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-20T12:45:54.248Z: Expanding GroupByKey operations into streaming Read/Write steps
Aug 20, 2021 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-20T12:45:54.323Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Aug 20, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-20T12:45:54.467Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Aug 20, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-20T12:45:54.534Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Aug 20, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-20T12:45:54.578Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Aug 20, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-20T12:45:54.638Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Aug 20, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-20T12:45:54.693Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Aug 20, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-20T12:45:54.730Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Aug 20, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-20T12:45:54.766Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Aug 20, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-20T12:45:54.831Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Aug 20, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-20T12:45:54.865Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Aug 20, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-20T12:45:54.900Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Aug 20, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-20T12:45:54.926Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Aug 20, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-20T12:45:54.964Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Aug 20, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-20T12:45:54.989Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Aug 20, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-20T12:45:55.019Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Aug 20, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-20T12:45:55.055Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Aug 20, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-20T12:45:55.106Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Aug 20, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-20T12:45:55.143Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Aug 20, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-20T12:45:55.173Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Aug 20, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-20T12:45:55.211Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Aug 20, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-20T12:45:55.250Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Aug 20, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-20T12:45:55.284Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Aug 20, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-20T12:45:55.328Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Aug 20, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-20T12:45:55.363Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Aug 20, 2021 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-20T12:45:55.985Z: Starting 5 ****s in us-central1-a...
Aug 20, 2021 12:46:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-20T12:46:22.811Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Aug 20, 2021 12:46:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-20T12:46:38.650Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Aug 20, 2021 12:47:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-20T12:47:32.171Z: Workers have started successfully.
Aug 20, 2021 12:47:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-20T12:47:32.211Z: Workers have started successfully.
Aug 20, 2021 3:54:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-20T15:54:57.523Z: Staged package failureaccess-1.0.1-oXHuTHNN0tqDfksWvp30Zhr6typBra8x64Tf2vk2yiY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/failureaccess-1.0.1-oXHuTHNN0tqDfksWvp30Zhr6typBra8x64Tf2vk2yiY.jar' is inaccessible.
Aug 20, 2021 3:54:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-20T15:54:58.021Z: Staged package google-cloud-bigquerystorage-1.21.1-nHDFkh4KVg5oRb7szB06ag7xZCJMMpmYG87RtFqNYoQ.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-cloud-bigquerystorage-1.21.1-nHDFkh4KVg5oRb7szB06ag7xZCJMMpmYG87RtFqNYoQ.jar' is inaccessible.
Aug 20, 2021 3:54:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-20T15:54:58.354Z: Staged package google-oauth-client-1.31.0-9fiaR9DCEOJ3VPxypLVvfxhohPwMB0pyoLY-Gc52QtA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-oauth-client-1.31.0-9fiaR9DCEOJ3VPxypLVvfxhohPwMB0pyoLY-Gc52QtA.jar' is inaccessible.
Aug 20, 2021 3:55:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-20T15:54:59.546Z: Staged package listenablefuture-9999.0-empty-to-avoid-conflict-with-guava-s3KgN9QjCqV_vv_e8w_WEj-cDC24XQrO0AyRuXTzP5k.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava-s3KgN9QjCqV_vv_e8w_WEj-cDC24XQrO0AyRuXTzP5k.jar' is inaccessible.
Aug 20, 2021 3:55:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-20T15:54:59.826Z: Staged package netty-tcnative-boringssl-static-2.0.33.Final-q3CryMTeke7zEAdYTla2hvWe2cxEwrM38_6C8EAtOH0.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-tcnative-boringssl-static-2.0.33.Final-q3CryMTeke7zEAdYTla2hvWe2cxEwrM38_6C8EAtOH0.jar' is inaccessible.
Aug 20, 2021 3:55:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-20T15:55:00.371Z: Staged package threetenbp-1.5.1-Q0LuBNhwQPcbCqkYjulgeA7y2nNOMqjUOlIqWAteDzs.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/threetenbp-1.5.1-Q0LuBNhwQPcbCqkYjulgeA7y2nNOMqjUOlIqWAteDzs.jar' is inaccessible.
Aug 20, 2021 3:55:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-08-20T15:55:00.447Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Aug 20, 2021 3:57:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-08-20T15:57:59.746Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Aug 20, 2021 4:00:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-20T16:00:34.151Z: Cancel request is committed for workflow job: 2021-08-20_05_45_30-16385337836395962967.
Aug 20, 2021 4:00:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-20T16:00:34.242Z: Cleaning up.
Aug 20, 2021 4:00:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-20T16:00:34.615Z: Stopping **** pool...
Aug 20, 2021 4:00:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-20T16:00:34.905Z: Stopping **** pool...
Aug 20, 2021 4:02:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-20T16:02:56.969Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Aug 20, 2021 4:02:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-20T16:02:57.027Z: Worker pool stopped.
Aug 20, 2021 4:03:04 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-08-20_05_45_30-16385337836395962967 finished with status CANCELLED.
Load test results for test (ID): 162c373f-c4b6-4165-af1a-d82f604fcfed and timestamp: 2021-08-20T12:45:26.188000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11478.693
dataflow_v2_java11_total_bytes_count             2.02519446E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210820124331
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:2554867c54eff75e911fbb95311d048c6e4c32984f02966ec4abb8a516b94f83
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210820124331]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:2554867c54eff75e911fbb95311d048c6e4c32984f02966ec4abb8a516b94f83]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210820124331] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:2554867c54eff75e911fbb95311d048c6e4c32984f02966ec4abb8a516b94f83])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:2554867c54eff75e911fbb95311d048c6e4c32984f02966ec4abb8a516b94f83
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:2554867c54eff75e911fbb95311d048c6e4c32984f02966ec4abb8a516b94f83
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:2554867c54eff75e911fbb95311d048c6e4c32984f02966ec4abb8a516b94f83].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 50s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/fjja7y7rclbkk

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #63

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/63/display/redirect?page=changes>

Changes:

[heejong] [BEAM-12735] Adding Python XLang examples to the RC validation script

[samuelw] [BEAM-12740] [BEAM-8268] Improve error handling and retries for GCS

[aromanenko.dev] [BEAM-12270] Add Parquet source support for TPD-DS benchmark

[heejong] update

[heejong] check xlang test outputs

[noreply] [BEAM-12742] Samza Runner does not properly delete modified timer

[noreply] [BEAM-3713] Cleanup, remove nosetest references (#15245)

[noreply] [BEAM-12772] Remove test implementation of SamzaIORegistrar (#15347)


------------------------------------------
[...truncated 49.18 KB...]
75f6a0e6e441: Waiting
a4d7f40bd5e1: Pushed
844aea564417: Pushed
d0c51e47841e: Pushed
e9f384cf2826: Pushed
42b320e153d2: Pushed
d364566d27d1: Pushed
7f8111a16a37: Pushed
81fffcf57fbe: Pushed
b19f17003e5a: Layer already exists
75f6a0e6e441: Layer already exists
c2e2307780ac: Layer already exists
79005165bb45: Pushed
7d890913ab69: Layer already exists
1235daf38153: Layer already exists
62d14713f2e9: Layer already exists
1fab7801fa97: Pushed
c2ddc1bc2645: Layer already exists
cdd7077abde3: Pushed
8dd10cea5b3d: Pushed
20210819124332: digest: sha256:52abf3063652313973e8ac32b334ce8c0e904a2ac060f94faf698adec380bdd0 size: 4310

> Task :sdks:java:testing:load-tests:run
Aug 19, 2021 12:45:22 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Aug 19, 2021 12:45:22 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 191 files. Enable logging at DEBUG level to see which files will be staged.
Aug 19, 2021 12:45:23 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Aug 19, 2021 12:45:23 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Aug 19, 2021 12:45:25 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Aug 19, 2021 12:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <112684 bytes, hash 7ff56fbcfa3c9820c0a20ca7dc74c07bbee4e71b219aa14ff5b495bf0817bc46> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-f_VvvPo8mCDAogyn3HTAe77k5xshmqFP9bSVvwgXvEY.pb
Aug 19, 2021 12:45:26 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 191 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Aug 19, 2021 12:45:27 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 191 files cached, 0 files newly uploaded in 0 seconds
Aug 19, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Aug 19, 2021 12:45:27 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3aa41da1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@74fab04a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c83468e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@272ce069, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3d7fb838, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1c26273d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3a37a501, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7a583586, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3a26ec8d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@17143b3b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@100c8b75, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2bc378f7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@268cbb86, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@10f7918f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@64d4f7c7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@54e02f6a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@17f3eefb, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ba46e63, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@788ddc1f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2dc3271b]
Aug 19, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Aug 19, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Aug 19, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Aug 19, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Aug 19, 2021 12:45:27 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4c7e978c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@354e7004, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@22ebccb9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5875de6a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@608fe01f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2687725a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5a75ec37, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2c05ff9d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@33956d1a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2e1ddc90, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5b7c8930, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58867cd5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@40e32762, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5a8ab2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3f50b680, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@54db056b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@d5af0a5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5981f4a6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@63dfada0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6f231ced]
Aug 19, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Aug 19, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Aug 19, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Aug 19, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Aug 19, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Aug 19, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Aug 19, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Aug 19, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Aug 19, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Aug 19, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Aug 19, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Aug 19, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.33.0-SNAPSHOT
Aug 19, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-08-19_05_45_27-17888583719528530001?project=apache-beam-testing
Aug 19, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-08-19_05_45_27-17888583719528530001
Aug 19, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-08-19_05_45_27-17888583719528530001
Aug 19, 2021 12:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-08-19T12:45:34.815Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-08-958d. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Aug 19, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-19T12:45:38.038Z: Worker configuration: e2-standard-2 in us-central1-a.
Aug 19, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-19T12:45:38.801Z: Expanding SplittableParDo operations into optimizable parts.
Aug 19, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-19T12:45:38.845Z: Expanding CollectionToSingleton operations into optimizable parts.
Aug 19, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-19T12:45:38.914Z: Expanding CoGroupByKey operations into optimizable parts.
Aug 19, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-19T12:45:38.965Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Aug 19, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-19T12:45:38.997Z: Expanding GroupByKey operations into streaming Read/Write steps
Aug 19, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-19T12:45:39.054Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Aug 19, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-19T12:45:39.157Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Aug 19, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-19T12:45:39.182Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Aug 19, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-19T12:45:39.208Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Aug 19, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-19T12:45:39.247Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Aug 19, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-19T12:45:39.268Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Aug 19, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-19T12:45:39.300Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Aug 19, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-19T12:45:39.335Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Aug 19, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-19T12:45:39.356Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Aug 19, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-19T12:45:39.389Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Aug 19, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-19T12:45:39.418Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Aug 19, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-19T12:45:39.464Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Aug 19, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-19T12:45:39.503Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Aug 19, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-19T12:45:39.533Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Aug 19, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-19T12:45:39.565Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Aug 19, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-19T12:45:39.587Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Aug 19, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-19T12:45:39.620Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Aug 19, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-19T12:45:39.654Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Aug 19, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-19T12:45:39.691Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Aug 19, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-19T12:45:39.721Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Aug 19, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-19T12:45:39.757Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Aug 19, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-19T12:45:39.834Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Aug 19, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-19T12:45:39.858Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Aug 19, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-19T12:45:39.880Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Aug 19, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-19T12:45:40.277Z: Starting 5 ****s in us-central1-a...
Aug 19, 2021 12:46:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-19T12:46:01.247Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Aug 19, 2021 12:46:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-19T12:46:30.138Z: Autoscaling: Raised the number of ****s to 4 so that the pipeline can catch up with its backlog and keep up with its input rate.
Aug 19, 2021 12:46:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-19T12:46:30.162Z: Resized **** pool to 4, though goal was 5.  This could be a quota issue.
Aug 19, 2021 12:46:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-19T12:46:50.685Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Aug 19, 2021 12:47:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-19T12:47:29.033Z: Workers have started successfully.
Aug 19, 2021 12:47:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-19T12:47:29.066Z: Workers have started successfully.
Aug 19, 2021 4:00:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-19T16:00:40.243Z: Cancel request is committed for workflow job: 2021-08-19_05_45_27-17888583719528530001.
Aug 19, 2021 4:00:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-19T16:00:40.320Z: Cleaning up.
Aug 19, 2021 4:00:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-19T16:00:40.390Z: Stopping **** pool...
Aug 19, 2021 4:00:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-19T16:00:40.457Z: Stopping **** pool...
Aug 19, 2021 4:02:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-19T16:02:58.724Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Aug 19, 2021 4:02:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-19T16:02:58.761Z: Worker pool stopped.
Aug 19, 2021 4:03:05 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-08-19_05_45_27-17888583719528530001 finished with status CANCELLED.
Load test results for test (ID): 63c6b5cd-a5c6-4772-81e4-f2b604477ed7 and timestamp: 2021-08-19T12:45:22.783000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11554.324
dataflow_v2_java11_total_bytes_count             2.75145426E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210819124332
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:52abf3063652313973e8ac32b334ce8c0e904a2ac060f94faf698adec380bdd0
Deleted: sha256:5e00d23d2cbdc0f5a868964f670a6e07bff4237a455db613adec0f21916cfa48
Deleted: sha256:6c5792bc5646f422bbec702198bcc07ad16588f3113fec9a7c543f70ebbf1425
Deleted: sha256:ec0a190155a112dfa36cc32c6dd4f435e740efad52598794d4e50d9abcc9fb7a
Deleted: sha256:0cab04f3a88834cb069d720878703bb56b8550a191d2c718e2b376a25e3fd25e
Deleted: sha256:839b3a89b00739de3c16cd690ddfd6eb06c06f41b15d71c438652f5b8caa113a
Deleted: sha256:00fb7cbff5c50e0b7527489c94496bd1f5e10a33add8c21ee38811ecce9fb5f0
Deleted: sha256:269c4b372b3df34876617a61270ccc6989f07cba1a708c5c839a11f330191efa
Deleted: sha256:850fdc7213edb358e27c92f0912393832111093bc051474caa8b9b891d1b95f4
Deleted: sha256:1387365bd25bc71d2aa50b01c760f772318ccbc77b980d44e7f79eca9892ea45
Deleted: sha256:6d0a81346ebe1eebbf816fb68ff731c7446b730f5e5dd19f438c9d51447c3c1c
Deleted: sha256:55c4e440a797ba87a0ac908bfbbc65f61499da2719995c205f36e54148ecbeab
Deleted: sha256:f09c942c995c0b7307e3a9014d082533bd2360718983e549a70a01fb5a2699d3
Deleted: sha256:115dbf690f7b239fd72e5d1fec0d6052279a793d0d7a8d613dd29a49839d74d4
Deleted: sha256:e1f50145946cf106aa87bb72ce167d348d6fafb496b87ebb0d99074f21e5143c
Deleted: sha256:a6e23eea0898104097e2c6bc042dbacc0b0f84a94bb284acbeb9c0654d21a509
Deleted: sha256:8e11b19b41c1e126d2f78010c6d968d590e0a4c343103cb6ef5c6fcb36140736
Deleted: sha256:4062cc0620d682e32b33ad6505920ca047df2df290a9e0df0facbdcd33ca3a83
Deleted: sha256:7e2001150fdf0fabef5c39eaf1590cc379a35034e6e551010c8283a5aff7c372
Deleted: sha256:40886774e18231615ac6fcf14b98006d0ded0536400e8dca812285fb1dba2cbf
Deleted: sha256:458d1f42b24624441be9441e4b8a0736e54ac7d2b205acd662fa0af516c0c769
Deleted: sha256:cea059be94ba96c3cbb4eed92484616e15fc66296525bb29f89d5e641220ef54
Deleted: sha256:4c8c3dc71cdd099ae51aa1b8c9deaa2d5779c98e35988bfbcc586c5709a68ff6
Deleted: sha256:d121a7032d9bcca6b56718c6e6771ff9370dd420e3bdc1e5057c1db12ff7d7c9
Deleted: sha256:6e02f3d94c2359e344e7a2d0230ddb17b752f6d521d456d1b2c22e1421cad944
Deleted: sha256:557b2926f8541d3c3ec0da655c29106749d34ed9a08cec5e8f66862999c2878a
Deleted: sha256:a92309ca388793eefea03ad66fa4c08d387ba6268c21dc80cd41e53131ed45d6
Deleted: sha256:93c0d7a68c057d44ca88f5cd7fe51ab43f295b90586392b11325ba88de9d5595
Deleted: sha256:c29a5ce813bf4d6c988b56dad0d2c68564de99f95d14d94103558b0c9ab2dc8c
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210819124332]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:52abf3063652313973e8ac32b334ce8c0e904a2ac060f94faf698adec380bdd0]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210819124332] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:52abf3063652313973e8ac32b334ce8c0e904a2ac060f94faf698adec380bdd0])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:52abf3063652313973e8ac32b334ce8c0e904a2ac060f94faf698adec380bdd0
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:52abf3063652313973e8ac32b334ce8c0e904a2ac060f94faf698adec380bdd0
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:52abf3063652313973e8ac32b334ce8c0e904a2ac060f94faf698adec380bdd0].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 54s
101 actionable tasks: 72 executed, 27 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/xhclrdagxgejo

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #62

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/62/display/redirect?page=changes>

Changes:

[ruane.jb] Fix package containing RestrictionProvider

[Jan Lukavský] [BEAM-12710] Disable TestStreamTest.testFirstElementLate for portable

[randomstep] BEAM-12635 Bump Apache Compress to 1.21

[noreply] Fix line length

[noreply] Fix whitespace.

[noreply] Take formatters suggestion

[aromanenko.dev] [BEAM-12429] Add support for S3 Bucket Key at the object level

[noreply] [BEAM-12755] Stop throwing exceptions during formatting and calculation

[noreply] [BEAM-12734] Autopopulate opts using Go flags. (#15311)

[noreply] [BEAM-12768] Make the test less strict and instead match on substring

[noreply] [BEAM-11088] Clean up incorrect comment (#15345)

[noreply] [BEAM-10955] Update Flink minor versions and enable testSavepointRest…

[daniel.o.programmer] [BEAM-5379] Go modules basic support and upgrade to Go 1.16.5.

[daniel.o.programmer] [BEAM-5379] Go Jenkins tests adjusted to use modules.

[daniel.o.programmer] [BEAM-5379] Add /v2/ to go module path and update references.

[daniel.o.programmer] [BEAM-5379] Fix Go SDK proto generation, and update protos.

[daniel.o.programmer] [BEAM-5379] Go module work fixup.

[noreply] Fix grpc data read thread block with finished instruction_id in


------------------------------------------
[...truncated 50.21 KB...]
732a914dbcb8: Waiting
8d04f5677728: Waiting
cc80467800d6: Waiting
eeb0f15f0e68: Waiting
ad7ae1b1fc97: Waiting
9654006c59e5: Pushed
9978939174b4: Pushed
201d7eb44cb6: Pushed
e89a22a914ec: Pushed
cc80467800d6: Pushed
d47cbc8165fb: Pushed
732a914dbcb8: Pushed
eeb0f15f0e68: Pushed
b19f17003e5a: Layer already exists
75f6a0e6e441: Layer already exists
c2e2307780ac: Layer already exists
ba528102801e: Pushed
7d890913ab69: Layer already exists
ad7ae1b1fc97: Pushed
1235daf38153: Layer already exists
62d14713f2e9: Layer already exists
c2ddc1bc2645: Layer already exists
0cc88b4c246a: Pushed
8d04f5677728: Pushed
20210818124334: digest: sha256:a578e1826618b22ff723faa0877f94d4355c62aaf498923b87ebb7fe04bdcf1c size: 4310

> Task :sdks:java:testing:load-tests:run
Aug 18, 2021 12:45:31 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Aug 18, 2021 12:45:32 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 191 files. Enable logging at DEBUG level to see which files will be staged.
Aug 18, 2021 12:45:32 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Aug 18, 2021 12:45:32 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Aug 18, 2021 12:45:35 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Aug 18, 2021 12:45:35 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <112684 bytes, hash 8b68370012832025ad5be73594fd86c65cf7026563e338458737d54bcb0f8459> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-i2g3ABKDICWtW-c1lP2Gxlz3AmVj4zhFhzfVS8sPhFk.pb
Aug 18, 2021 12:45:36 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 191 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Aug 18, 2021 12:45:37 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 191 files cached, 0 files newly uploaded in 0 seconds
Aug 18, 2021 12:45:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Aug 18, 2021 12:45:37 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12d5c30e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@b887730, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26586b74, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@52f57666, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6e041285, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@267dc982, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@439b15f2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3aa41da1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@74fab04a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c83468e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@272ce069, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3d7fb838, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1c26273d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3a37a501, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7a583586, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3a26ec8d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@17143b3b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@100c8b75, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2bc378f7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@268cbb86]
Aug 18, 2021 12:45:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Aug 18, 2021 12:45:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Aug 18, 2021 12:45:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Aug 18, 2021 12:45:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Aug 18, 2021 12:45:37 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2d4608a6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@20d87335, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2a8a4e0c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26c89563, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3bd6ba24, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58f437b0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@20f6f88c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4277127c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4c7e978c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@354e7004, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@22ebccb9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5875de6a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@608fe01f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2687725a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5a75ec37, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2c05ff9d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@33956d1a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2e1ddc90, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5b7c8930, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58867cd5]
Aug 18, 2021 12:45:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Aug 18, 2021 12:45:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Aug 18, 2021 12:45:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Aug 18, 2021 12:45:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Aug 18, 2021 12:45:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Aug 18, 2021 12:45:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Aug 18, 2021 12:45:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Aug 18, 2021 12:45:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Aug 18, 2021 12:45:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Aug 18, 2021 12:45:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Aug 18, 2021 12:45:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Aug 18, 2021 12:45:37 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.33.0-SNAPSHOT
Aug 18, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-08-18_05_45_37-7643709984619510344?project=apache-beam-testing
Aug 18, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-08-18_05_45_37-7643709984619510344
Aug 18, 2021 12:45:38 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-08-18_05_45_37-7643709984619510344
Aug 18, 2021 12:45:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-08-18T12:45:44.366Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-08-ex5x. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Aug 18, 2021 12:45:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-18T12:45:47.939Z: Worker configuration: e2-standard-2 in us-central1-a.
Aug 18, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-18T12:45:48.643Z: Expanding SplittableParDo operations into optimizable parts.
Aug 18, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-18T12:45:48.679Z: Expanding CollectionToSingleton operations into optimizable parts.
Aug 18, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-18T12:45:48.733Z: Expanding CoGroupByKey operations into optimizable parts.
Aug 18, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-18T12:45:48.809Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Aug 18, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-18T12:45:48.843Z: Expanding GroupByKey operations into streaming Read/Write steps
Aug 18, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-18T12:45:48.895Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Aug 18, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-18T12:45:48.975Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Aug 18, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-18T12:45:49.009Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Aug 18, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-18T12:45:49.042Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Aug 18, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-18T12:45:49.071Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Aug 18, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-18T12:45:49.097Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Aug 18, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-18T12:45:49.131Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Aug 18, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-18T12:45:49.165Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Aug 18, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-18T12:45:49.200Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Aug 18, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-18T12:45:49.232Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Aug 18, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-18T12:45:49.265Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Aug 18, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-18T12:45:49.300Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Aug 18, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-18T12:45:49.335Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Aug 18, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-18T12:45:49.371Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Aug 18, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-18T12:45:49.404Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Aug 18, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-18T12:45:49.440Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Aug 18, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-18T12:45:49.462Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Aug 18, 2021 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-18T12:45:49.493Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Aug 18, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-18T12:45:49.527Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Aug 18, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-18T12:45:49.563Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Aug 18, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-18T12:45:49.594Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Aug 18, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-18T12:45:49.616Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Aug 18, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-18T12:45:49.643Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Aug 18, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-18T12:45:49.675Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Aug 18, 2021 12:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-18T12:45:50.003Z: Starting 5 ****s in us-central1-a...
Aug 18, 2021 12:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-18T12:45:54.884Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Aug 18, 2021 12:46:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-18T12:46:42.056Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Aug 18, 2021 12:47:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-18T12:47:35.704Z: Workers have started successfully.
Aug 18, 2021 12:47:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-18T12:47:35.738Z: Workers have started successfully.
Aug 18, 2021 4:00:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-18T16:00:34.946Z: Cancel request is committed for workflow job: 2021-08-18_05_45_37-7643709984619510344.
Aug 18, 2021 4:00:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-18T16:00:35.074Z: Cleaning up.
Aug 18, 2021 4:00:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-18T16:00:35.156Z: Stopping **** pool...
Aug 18, 2021 4:00:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-18T16:00:35.210Z: Stopping **** pool...
Aug 18, 2021 4:03:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-18T16:03:04.054Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Aug 18, 2021 4:03:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-18T16:03:04.097Z: Worker pool stopped.
Aug 18, 2021 4:03:11 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-08-18_05_45_37-7643709984619510344 finished with status CANCELLED.
Load test results for test (ID): fd492f73-f10c-4a82-ab86-4d794e9ed569 and timestamp: 2021-08-18T12:45:32.562000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                  11522.52
dataflow_v2_java11_total_bytes_count             3.24367856E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210818124334
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:a578e1826618b22ff723faa0877f94d4355c62aaf498923b87ebb7fe04bdcf1c
Deleted: sha256:359e4384b215cb3f6a8e8f3a0cf6a2183675ee493a55b7d45e897f0658edcf12
Deleted: sha256:cfaa691fcd3a9440d0e10a5f643079d8494a761b4c96786f03bcaa5bfbe686f0
Deleted: sha256:f9bd1570ce1f52453ca180f4c7e598f66ae411a166537b7b6854e1fbda3decbe
Deleted: sha256:80dadfdf7cadb86e4bd17505ee692e8782ede09ef1423837eb23ec58cc26e371
Deleted: sha256:5673b7372210ea80d1b51fcda8c4a0f5176a86d77508cec4da7e56f3a3ce7612
Deleted: sha256:356f658faea43cb10815997491820b7916a60d031a9a75f46e610bcb275c3ec8
Deleted: sha256:0536f09093d6e6b735c28e87dd36001cb1dc821e28174c1d952cc78a4d8ee2d5
Deleted: sha256:573c6002da4c755dce21f34bdca1b089e17b84852685722cf81b3ee34855bdd2
Deleted: sha256:74dfb486ef0e22802b814abcc1210b81a395c08807c03b9e49f80dc9ccadcdcd
Deleted: sha256:ecbf37318854a6685f6d1997cc57005159970a84e7966302f1013c2f29ab8c82
Deleted: sha256:ba6d58c05180152316f36feed97ef64be85b7bb8cd5a5bc5dc455512a5bfba1e
Deleted: sha256:417865bcc30c98d0383b363b509fe8b54910af29db879f1fd9bca1d29f1e12ac
Deleted: sha256:9c158c2eac42e259d7ea4c1e2b8ed1a1f202e5f14d46d043850a72af10876f2f
Deleted: sha256:9f30b0fa9ff8b9471717f52c54eba4f259d52f91db6c8188c42f77644c3f7de8
Deleted: sha256:279dc1a4bd232cf091bda6b3a662112b256018288db8fe7d8321ba8273a7bc5a
Deleted: sha256:e8f65ecc6fa91bcae0e7c34bda39833fda09e1669d170fcdbe512b3dfebee6d0
Deleted: sha256:7bd350b479ba9160129bf3d3c3a3c0bdf1af6992d826d9d08cebc00f488773e0
Deleted: sha256:d39bc0e9bd45fdefeb96c8855294266b668a38b2f8a4807bb12c22b8ec7f0caa
Deleted: sha256:7278532b1ba79866d43d2462001add2eec1fbc6f36ec23893d0177efde8131fd
Deleted: sha256:0dfaa5e72c73ad4cf187bbdacd760684d88410c35547b2a529a479a8b72ec892
Deleted: sha256:2bb74b96f0a71e5408fdb8a29a5c898d73e997274e4f8904542f493ca63dc0ad
Deleted: sha256:d5f6205192f63805aff9b39976911382ed215e25f777217d43827c2cc19e0859
Deleted: sha256:9c0c92fca272f041345cc59919e20754be2901cebc83bf4cdc22c6d6b368eb9f
Deleted: sha256:ad9dbe6d958c408f1bff416259b54dddae9add51c2a0ead8fc4cbf12900e5510
Deleted: sha256:1fff2242234fcb1b44ab076cbec0174b26e592724a9f1d0225f13a4bacd3dbe1
Deleted: sha256:b853ef90e8dea4e33b7875f6f4dcabfe1a7bc5605f21b8da97dd9318e428f9a4
Deleted: sha256:264ccc8a07e96330580685d9a0bb6b181848aa7fbb0e2e84e162481d5e31f606
Deleted: sha256:90983e23a6e0c4ba1d00a8b88191458afb00e7213adcd5bd3f4fb0858ec168fd
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210818124334]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:a578e1826618b22ff723faa0877f94d4355c62aaf498923b87ebb7fe04bdcf1c]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210818124334] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:a578e1826618b22ff723faa0877f94d4355c62aaf498923b87ebb7fe04bdcf1c])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:a578e1826618b22ff723faa0877f94d4355c62aaf498923b87ebb7fe04bdcf1c
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:a578e1826618b22ff723faa0877f94d4355c62aaf498923b87ebb7fe04bdcf1c
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:a578e1826618b22ff723faa0877f94d4355c62aaf498923b87ebb7fe04bdcf1c].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 59s
101 actionable tasks: 72 executed, 27 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/y5url5cuyfb5k

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #61

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/61/display/redirect?page=changes>

Changes:

[Luke Cwik] [BEAM-12619] Swap LinkedBlockingQueue to ArrayBlockingQueue for minor

[Luke Cwik] fixup! Fix spotbugs warning

[Kyle Weaver] [BEAM-12609] Make type parameter part of PushdownProjector interface.

[Jeremy Quinn] Add AWS services as a runtime dependency to support S3

[pascal.gillet] [BEAM-12479] Fixes UnsupportedOperationException

[Andrew Pilloud] [BEAM-12759] ORDER BY then SELECT

[baeminbo] [BEAM-12504] Make CreateTransaction wait on input signal

[noreply] [BEAM-7745] Avoid uncached state fetches for streaming side-inputs

[Robert Burke] redundant build

[noreply] [BEAM-11088] TestStream utility and testing improvements (#15320)


------------------------------------------
[...truncated 65.64 KB...]
a64d3d8c3294: Preparing
d7019d9766f2: Preparing
d3d4b98fb03b: Preparing
e62e98114a38: Preparing
0ad5a314d1e0: Preparing
72e1f7a8ff27: Preparing
4d5cfb6531c2: Preparing
67a98aac0413: Preparing
3ca81366648c: Preparing
14a72816b6ec: Preparing
90330910d68d: Preparing
04698ebb8319: Preparing
f42aed5f7feb: Preparing
89819bafde36: Preparing
f3d5b8f65132: Preparing
ad83f0aa5c0a: Preparing
5a9a65095453: Preparing
4b0edb23340c: Preparing
afa3e488a0ee: Preparing
72e1f7a8ff27: Waiting
4d5cfb6531c2: Waiting
89819bafde36: Waiting
67a98aac0413: Waiting
3ca81366648c: Waiting
5a9a65095453: Waiting
14a72816b6ec: Waiting
f42aed5f7feb: Waiting
4b0edb23340c: Waiting
f3d5b8f65132: Waiting
afa3e488a0ee: Waiting
90330910d68d: Waiting
ad83f0aa5c0a: Waiting
04698ebb8319: Waiting
0ad5a314d1e0: Pushed
d7019d9766f2: Pushed
d3d4b98fb03b: Pushed
e62e98114a38: Pushed
a64d3d8c3294: Pushed
72e1f7a8ff27: Pushed
67a98aac0413: Pushed
3ca81366648c: Pushed
f42aed5f7feb: Layer already exists
89819bafde36: Layer already exists
f3d5b8f65132: Layer already exists
ad83f0aa5c0a: Layer already exists
5a9a65095453: Layer already exists
4b0edb23340c: Layer already exists
90330910d68d: Pushed
afa3e488a0ee: Layer already exists
04698ebb8319: Pushed
4d5cfb6531c2: Pushed
14a72816b6ec: Pushed
20210817124335: digest: sha256:4506e2a2b54f4e6614f56ac34f968edefc047ced6c373fa513663474ec259916 size: 4310

> Task :sdks:java:testing:load-tests:run
Aug 17, 2021 12:48:21 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Aug 17, 2021 12:48:21 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 191 files. Enable logging at DEBUG level to see which files will be staged.
Aug 17, 2021 12:48:22 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Aug 17, 2021 12:48:22 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Aug 17, 2021 12:48:24 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Aug 17, 2021 12:48:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <112622 bytes, hash 6d0e8521a8e9eaebf4f5a1707be83cc7635f96e8c6e4b2773906e3fe1a60e3c2> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-bQ6FIajp6uv09aFwe-g8x2NflujG5LJ3OQbj_hpg48I.pb
Aug 17, 2021 12:48:26 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 191 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Aug 17, 2021 12:48:26 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 191 files cached, 0 files newly uploaded in 0 seconds
Aug 17, 2021 12:48:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Aug 17, 2021 12:48:26 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12d5c30e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@b887730, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26586b74, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@52f57666, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6e041285, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@267dc982, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@439b15f2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3aa41da1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@74fab04a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c83468e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@272ce069, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3d7fb838, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1c26273d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3a37a501, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7a583586, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3a26ec8d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@17143b3b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@100c8b75, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2bc378f7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@268cbb86]
Aug 17, 2021 12:48:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Aug 17, 2021 12:48:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Aug 17, 2021 12:48:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Aug 17, 2021 12:48:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Aug 17, 2021 12:48:27 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@20d87335, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2a8a4e0c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26c89563, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3bd6ba24, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58f437b0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@20f6f88c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4277127c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4c7e978c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@354e7004, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@22ebccb9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5875de6a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@608fe01f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2687725a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5a75ec37, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2c05ff9d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@33956d1a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2e1ddc90, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5b7c8930, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58867cd5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@40e32762]
Aug 17, 2021 12:48:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Aug 17, 2021 12:48:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Aug 17, 2021 12:48:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Aug 17, 2021 12:48:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Aug 17, 2021 12:48:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Aug 17, 2021 12:48:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Aug 17, 2021 12:48:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Aug 17, 2021 12:48:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Aug 17, 2021 12:48:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Aug 17, 2021 12:48:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Aug 17, 2021 12:48:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Aug 17, 2021 12:48:27 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.33.0-SNAPSHOT
Aug 17, 2021 12:48:28 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-08-17_05_48_27-4021806797223042197?project=apache-beam-testing
Aug 17, 2021 12:48:28 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-08-17_05_48_27-4021806797223042197
Aug 17, 2021 12:48:28 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-08-17_05_48_27-4021806797223042197
Aug 17, 2021 12:48:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-08-17T12:48:34.377Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-08-w4im. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Aug 17, 2021 12:48:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-17T12:48:38.276Z: Worker configuration: e2-standard-2 in us-central1-a.
Aug 17, 2021 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-17T12:48:38.967Z: Expanding SplittableParDo operations into optimizable parts.
Aug 17, 2021 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-17T12:48:38.991Z: Expanding CollectionToSingleton operations into optimizable parts.
Aug 17, 2021 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-17T12:48:39.055Z: Expanding CoGroupByKey operations into optimizable parts.
Aug 17, 2021 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-17T12:48:39.139Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Aug 17, 2021 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-17T12:48:39.174Z: Expanding GroupByKey operations into streaming Read/Write steps
Aug 17, 2021 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-17T12:48:39.243Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Aug 17, 2021 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-17T12:48:39.371Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Aug 17, 2021 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-17T12:48:39.406Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Aug 17, 2021 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-17T12:48:39.440Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Aug 17, 2021 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-17T12:48:39.473Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Aug 17, 2021 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-17T12:48:39.509Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Aug 17, 2021 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-17T12:48:39.543Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Aug 17, 2021 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-17T12:48:39.564Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Aug 17, 2021 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-17T12:48:39.641Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Aug 17, 2021 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-17T12:48:39.668Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Aug 17, 2021 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-17T12:48:39.701Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Aug 17, 2021 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-17T12:48:39.731Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Aug 17, 2021 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-17T12:48:39.757Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Aug 17, 2021 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-17T12:48:39.792Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Aug 17, 2021 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-17T12:48:39.825Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Aug 17, 2021 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-17T12:48:39.858Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Aug 17, 2021 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-17T12:48:39.881Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Aug 17, 2021 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-17T12:48:39.910Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Aug 17, 2021 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-17T12:48:39.945Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Aug 17, 2021 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-17T12:48:39.978Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Aug 17, 2021 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-17T12:48:40.009Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Aug 17, 2021 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-17T12:48:40.043Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Aug 17, 2021 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-17T12:48:40.080Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Aug 17, 2021 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-17T12:48:40.133Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Aug 17, 2021 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-17T12:48:40.487Z: Starting 5 ****s in us-central1-a...
Aug 17, 2021 12:48:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-17T12:48:59.090Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Aug 17, 2021 12:49:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-17T12:49:26.273Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Aug 17, 2021 12:50:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-17T12:50:21.882Z: Workers have started successfully.
Aug 17, 2021 12:50:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-17T12:50:21.904Z: Workers have started successfully.
Aug 17, 2021 4:00:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-17T16:00:37.200Z: Cancel request is committed for workflow job: 2021-08-17_05_48_27-4021806797223042197.
Aug 17, 2021 4:00:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-17T16:00:37.271Z: Cleaning up.
Aug 17, 2021 4:00:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-17T16:00:37.350Z: Stopping **** pool...
Aug 17, 2021 4:00:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-17T16:00:37.415Z: Stopping **** pool...
Aug 17, 2021 4:02:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-17T16:02:56.008Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Aug 17, 2021 4:02:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-17T16:02:56.043Z: Worker pool stopped.
Aug 17, 2021 4:03:01 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-08-17_05_48_27-4021806797223042197 finished with status CANCELLED.
Load test results for test (ID): 3cb2e641-c699-4c27-b32d-07d5d5454111 and timestamp: 2021-08-17T12:48:22.296000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11379.959
dataflow_v2_java11_total_bytes_count             2.80407676E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210817124335
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:4506e2a2b54f4e6614f56ac34f968edefc047ced6c373fa513663474ec259916
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210817124335]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:4506e2a2b54f4e6614f56ac34f968edefc047ced6c373fa513663474ec259916]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210817124335] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:4506e2a2b54f4e6614f56ac34f968edefc047ced6c373fa513663474ec259916])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:4506e2a2b54f4e6614f56ac34f968edefc047ced6c373fa513663474ec259916
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:4506e2a2b54f4e6614f56ac34f968edefc047ced6c373fa513663474ec259916
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:4506e2a2b54f4e6614f56ac34f968edefc047ced6c373fa513663474ec259916].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 43s
108 actionable tasks: 77 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/tdl5htowy2zuk

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #60

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/60/display/redirect>

Changes:


------------------------------------------
[...truncated 66.05 KB...]
1e94fb1aac8a: Waiting
afa3e488a0ee: Waiting
ad83f0aa5c0a: Waiting
f42aed5f7feb: Waiting
5e05aba601cb: Waiting
89819bafde36: Waiting
4e5c73086214: Pushed
1cd17d835dc1: Pushed
293720a9893a: Pushed
b40532d2d46b: Pushed
cd8201afd6df: Pushed
ad96e035db19: Pushed
58447b274736: Pushed
e533213451a9: Pushed
f42aed5f7feb: Layer already exists
89819bafde36: Layer already exists
5e05aba601cb: Pushed
f3d5b8f65132: Layer already exists
ad83f0aa5c0a: Layer already exists
1e94fb1aac8a: Pushed
5a9a65095453: Layer already exists
4b0edb23340c: Layer already exists
afa3e488a0ee: Layer already exists
4f00500066b4: Pushed
e6a72e1e89a3: Pushed
20210816124333: digest: sha256:6dfcfeecfc5121b4f62f8fe459e2e6aa8900c2aabf1ca4193d2f9e7c9c221cdb size: 4310

> Task :sdks:java:testing:load-tests:run
Aug 16, 2021 12:49:44 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Aug 16, 2021 12:49:44 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 191 files. Enable logging at DEBUG level to see which files will be staged.
Aug 16, 2021 12:49:45 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Aug 16, 2021 12:49:45 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Aug 16, 2021 12:49:48 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Aug 16, 2021 12:49:48 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <112622 bytes, hash 4618474e7da28c2afa2a256036875f69837b22ad33b54f11d85685bb77d9b784> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-RhhHTn2ijCr6KiVgNodfaYN7Iq0ztU8R2FaFu3fZt4Q.pb
Aug 16, 2021 12:49:49 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 191 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Aug 16, 2021 12:49:50 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 191 files cached, 0 files newly uploaded in 0 seconds
Aug 16, 2021 12:49:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Aug 16, 2021 12:49:50 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12d5c30e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@b887730, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26586b74, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@52f57666, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6e041285, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@267dc982, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@439b15f2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3aa41da1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@74fab04a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c83468e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@272ce069, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3d7fb838, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1c26273d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3a37a501, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7a583586, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3a26ec8d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@17143b3b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@100c8b75, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2bc378f7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@268cbb86]
Aug 16, 2021 12:49:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Aug 16, 2021 12:49:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Aug 16, 2021 12:49:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Aug 16, 2021 12:49:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Aug 16, 2021 12:49:50 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@20d87335, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2a8a4e0c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26c89563, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3bd6ba24, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58f437b0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@20f6f88c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4277127c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4c7e978c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@354e7004, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@22ebccb9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5875de6a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@608fe01f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2687725a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5a75ec37, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2c05ff9d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@33956d1a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2e1ddc90, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5b7c8930, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58867cd5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@40e32762]
Aug 16, 2021 12:49:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Aug 16, 2021 12:49:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Aug 16, 2021 12:49:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Aug 16, 2021 12:49:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Aug 16, 2021 12:49:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Aug 16, 2021 12:49:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Aug 16, 2021 12:49:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Aug 16, 2021 12:49:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Aug 16, 2021 12:49:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Aug 16, 2021 12:49:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Aug 16, 2021 12:49:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Aug 16, 2021 12:49:50 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.33.0-SNAPSHOT
Aug 16, 2021 12:49:51 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-08-16_05_49_50-14415542787694677541?project=apache-beam-testing
Aug 16, 2021 12:49:51 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-08-16_05_49_50-14415542787694677541
Aug 16, 2021 12:49:51 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-08-16_05_49_50-14415542787694677541
Aug 16, 2021 12:49:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-08-16T12:49:57.790Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-08-g1oe. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Aug 16, 2021 12:50:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-16T12:50:02.097Z: Worker configuration: e2-standard-2 in us-central1-a.
Aug 16, 2021 12:50:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-16T12:50:02.796Z: Expanding SplittableParDo operations into optimizable parts.
Aug 16, 2021 12:50:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-16T12:50:02.834Z: Expanding CollectionToSingleton operations into optimizable parts.
Aug 16, 2021 12:50:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-16T12:50:02.952Z: Expanding CoGroupByKey operations into optimizable parts.
Aug 16, 2021 12:50:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-16T12:50:03.010Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Aug 16, 2021 12:50:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-16T12:50:03.030Z: Expanding GroupByKey operations into streaming Read/Write steps
Aug 16, 2021 12:50:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-16T12:50:03.085Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Aug 16, 2021 12:50:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-16T12:50:03.199Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Aug 16, 2021 12:50:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-16T12:50:03.243Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Aug 16, 2021 12:50:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-16T12:50:03.269Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Aug 16, 2021 12:50:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-16T12:50:03.296Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Aug 16, 2021 12:50:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-16T12:50:03.322Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Aug 16, 2021 12:50:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-16T12:50:03.356Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Aug 16, 2021 12:50:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-16T12:50:03.382Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Aug 16, 2021 12:50:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-16T12:50:03.406Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Aug 16, 2021 12:50:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-16T12:50:03.427Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Aug 16, 2021 12:50:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-16T12:50:03.455Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Aug 16, 2021 12:50:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-16T12:50:03.492Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Aug 16, 2021 12:50:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-16T12:50:03.525Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Aug 16, 2021 12:50:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-16T12:50:03.561Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Aug 16, 2021 12:50:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-16T12:50:03.585Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Aug 16, 2021 12:50:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-16T12:50:03.617Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Aug 16, 2021 12:50:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-16T12:50:03.650Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Aug 16, 2021 12:50:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-16T12:50:03.672Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Aug 16, 2021 12:50:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-16T12:50:03.701Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Aug 16, 2021 12:50:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-16T12:50:03.722Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Aug 16, 2021 12:50:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-16T12:50:03.758Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Aug 16, 2021 12:50:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-16T12:50:03.789Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Aug 16, 2021 12:50:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-16T12:50:03.823Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Aug 16, 2021 12:50:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-16T12:50:03.849Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Aug 16, 2021 12:50:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-16T12:50:04.174Z: Starting 5 ****s in us-central1-a...
Aug 16, 2021 12:50:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-16T12:50:30.916Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Aug 16, 2021 12:50:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-16T12:50:46.165Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Aug 16, 2021 12:51:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-16T12:51:39.609Z: Workers have started successfully.
Aug 16, 2021 12:51:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-16T12:51:39.625Z: Workers have started successfully.
Aug 16, 2021 4:00:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-16T16:00:55.418Z: Cancel request is committed for workflow job: 2021-08-16_05_49_50-14415542787694677541.
Aug 16, 2021 4:00:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-16T16:00:55.477Z: Cleaning up.
Aug 16, 2021 4:00:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-16T16:00:55.528Z: Stopping **** pool...
Aug 16, 2021 4:00:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-16T16:00:55.574Z: Stopping **** pool...
Aug 16, 2021 4:01:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-16T16:01:04.039Z: generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:1089
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:1089
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:1089
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:1089
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:1089
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:1089
Aug 16, 2021 4:03:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-16T16:03:16.249Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Aug 16, 2021 4:03:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-16T16:03:16.286Z: Worker pool stopped.
Aug 16, 2021 4:03:22 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-08-16_05_49_50-14415542787694677541 finished with status CANCELLED.
Aug 16, 2021 4:03:22 PM org.apache.beam.sdk.testutils.metrics.MetricsReader getCounterMetric
SEVERE: Failed to get metric totalBytes.count, from namespace co_gbk
Load test results for test (ID): b4d1c1af-ee9a-443b-82ef-58dd8dd0b5dd and timestamp: 2021-08-16T12:49:44.841000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                   679.715
dataflow_v2_java11_total_bytes_count                      -1.0
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210816124333
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:6dfcfeecfc5121b4f62f8fe459e2e6aa8900c2aabf1ca4193d2f9e7c9c221cdb
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210816124333]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:6dfcfeecfc5121b4f62f8fe459e2e6aa8900c2aabf1ca4193d2f9e7c9c221cdb]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210816124333] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:6dfcfeecfc5121b4f62f8fe459e2e6aa8900c2aabf1ca4193d2f9e7c9c221cdb])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:6dfcfeecfc5121b4f62f8fe459e2e6aa8900c2aabf1ca4193d2f9e7c9c221cdb
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:6dfcfeecfc5121b4f62f8fe459e2e6aa8900c2aabf1ca4193d2f9e7c9c221cdb
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:6dfcfeecfc5121b4f62f8fe459e2e6aa8900c2aabf1ca4193d2f9e7c9c221cdb].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 20m 7s
107 actionable tasks: 76 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/4yyllsqqwpyhi

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #59

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/59/display/redirect>

Changes:


------------------------------------------
[...truncated 65.24 KB...]
88b81850f9e3: Preparing
0afbe09869d5: Preparing
b8f59ee8509d: Preparing
a8a0cc030adc: Preparing
6e6b70d92b94: Preparing
1185e92dc5e6: Preparing
3dcf995042bd: Preparing
d60b9c79a264: Preparing
8a241d825bcd: Preparing
d645fe966cdd: Preparing
80ccc6c6ee6e: Preparing
f42aed5f7feb: Preparing
89819bafde36: Preparing
f3d5b8f65132: Preparing
ad83f0aa5c0a: Preparing
5a9a65095453: Preparing
4b0edb23340c: Preparing
afa3e488a0ee: Preparing
afa3e488a0ee: Waiting
f42aed5f7feb: Waiting
5a9a65095453: Waiting
89819bafde36: Waiting
6e6b70d92b94: Waiting
ad83f0aa5c0a: Waiting
4b0edb23340c: Waiting
f3d5b8f65132: Waiting
1185e92dc5e6: Waiting
8a241d825bcd: Waiting
d645fe966cdd: Waiting
d60b9c79a264: Waiting
3dcf995042bd: Waiting
88b81850f9e3: Pushed
a8a0cc030adc: Pushed
0afbe09869d5: Pushed
35d308762ee8: Pushed
b8f59ee8509d: Pushed
6e6b70d92b94: Pushed
3dcf995042bd: Pushed
d60b9c79a264: Pushed
f42aed5f7feb: Layer already exists
d645fe966cdd: Pushed
89819bafde36: Layer already exists
f3d5b8f65132: Layer already exists
ad83f0aa5c0a: Layer already exists
80ccc6c6ee6e: Pushed
5a9a65095453: Layer already exists
4b0edb23340c: Layer already exists
afa3e488a0ee: Layer already exists
1185e92dc5e6: Pushed
8a241d825bcd: Pushed
20210815124333: digest: sha256:24a962e431cee4be935c2e1d6ffb43301d7606b14065354818d8814c45c948a2 size: 4310

> Task :sdks:java:testing:load-tests:run
Aug 15, 2021 12:48:20 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Aug 15, 2021 12:48:20 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 191 files. Enable logging at DEBUG level to see which files will be staged.
Aug 15, 2021 12:48:21 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Aug 15, 2021 12:48:21 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Aug 15, 2021 12:48:24 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Aug 15, 2021 12:48:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <112622 bytes, hash c99ef094b428bd43fa8b7913e01c433c3647305f49c237bbc5c0acd94bf4c7c4> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-yZ7wlLQovUP6i3kT4BxDPDZHMF9Jwje7xcCs2Uv0x8Q.pb
Aug 15, 2021 12:48:26 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 191 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Aug 15, 2021 12:48:26 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 191 files cached, 0 files newly uploaded in 0 seconds
Aug 15, 2021 12:48:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Aug 15, 2021 12:48:26 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12d5c30e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@b887730, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26586b74, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@52f57666, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6e041285, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@267dc982, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@439b15f2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3aa41da1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@74fab04a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c83468e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@272ce069, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3d7fb838, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1c26273d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3a37a501, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7a583586, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3a26ec8d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@17143b3b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@100c8b75, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2bc378f7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@268cbb86]
Aug 15, 2021 12:48:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Aug 15, 2021 12:48:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Aug 15, 2021 12:48:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Aug 15, 2021 12:48:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Aug 15, 2021 12:48:26 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@20d87335, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2a8a4e0c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26c89563, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3bd6ba24, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58f437b0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@20f6f88c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4277127c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4c7e978c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@354e7004, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@22ebccb9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5875de6a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@608fe01f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2687725a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5a75ec37, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2c05ff9d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@33956d1a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2e1ddc90, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5b7c8930, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58867cd5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@40e32762]
Aug 15, 2021 12:48:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Aug 15, 2021 12:48:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Aug 15, 2021 12:48:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Aug 15, 2021 12:48:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Aug 15, 2021 12:48:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Aug 15, 2021 12:48:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Aug 15, 2021 12:48:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Aug 15, 2021 12:48:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Aug 15, 2021 12:48:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Aug 15, 2021 12:48:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Aug 15, 2021 12:48:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Aug 15, 2021 12:48:26 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.33.0-SNAPSHOT
Aug 15, 2021 12:48:27 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-08-15_05_48_27-11108425642055097136?project=apache-beam-testing
Aug 15, 2021 12:48:27 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-08-15_05_48_27-11108425642055097136
Aug 15, 2021 12:48:27 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-08-15_05_48_27-11108425642055097136
Aug 15, 2021 12:48:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-08-15T12:48:33.936Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-08-8tdg. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Aug 15, 2021 12:48:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-15T12:48:37.165Z: Worker configuration: e2-standard-2 in us-central1-a.
Aug 15, 2021 12:48:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-15T12:48:37.767Z: Expanding SplittableParDo operations into optimizable parts.
Aug 15, 2021 12:48:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-15T12:48:37.807Z: Expanding CollectionToSingleton operations into optimizable parts.
Aug 15, 2021 12:48:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-15T12:48:37.856Z: Expanding CoGroupByKey operations into optimizable parts.
Aug 15, 2021 12:48:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-15T12:48:37.926Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Aug 15, 2021 12:48:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-15T12:48:37.961Z: Expanding GroupByKey operations into streaming Read/Write steps
Aug 15, 2021 12:48:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-15T12:48:38.019Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Aug 15, 2021 12:48:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-15T12:48:38.112Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Aug 15, 2021 12:48:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-15T12:48:38.136Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Aug 15, 2021 12:48:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-15T12:48:38.161Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Aug 15, 2021 12:48:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-15T12:48:38.195Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Aug 15, 2021 12:48:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-15T12:48:38.220Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Aug 15, 2021 12:48:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-15T12:48:38.243Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Aug 15, 2021 12:48:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-15T12:48:38.269Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Aug 15, 2021 12:48:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-15T12:48:38.301Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Aug 15, 2021 12:48:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-15T12:48:38.349Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Aug 15, 2021 12:48:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-15T12:48:38.381Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Aug 15, 2021 12:48:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-15T12:48:38.403Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Aug 15, 2021 12:48:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-15T12:48:38.427Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Aug 15, 2021 12:48:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-15T12:48:38.485Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Aug 15, 2021 12:48:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-15T12:48:38.518Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Aug 15, 2021 12:48:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-15T12:48:38.543Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Aug 15, 2021 12:48:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-15T12:48:38.566Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Aug 15, 2021 12:48:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-15T12:48:38.597Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Aug 15, 2021 12:48:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-15T12:48:38.620Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Aug 15, 2021 12:48:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-15T12:48:38.647Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Aug 15, 2021 12:48:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-15T12:48:38.679Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Aug 15, 2021 12:48:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-15T12:48:38.707Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Aug 15, 2021 12:48:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-15T12:48:38.738Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Aug 15, 2021 12:48:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-15T12:48:38.766Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Aug 15, 2021 12:48:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-15T12:48:39.076Z: Starting 5 ****s in us-central1-a...
Aug 15, 2021 12:48:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-15T12:48:44.861Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Aug 15, 2021 12:49:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-15T12:49:20.777Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Aug 15, 2021 12:50:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-15T12:50:11.910Z: Workers have started successfully.
Aug 15, 2021 12:50:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-15T12:50:11.933Z: Workers have started successfully.
Aug 15, 2021 4:00:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-15T16:00:52.211Z: Cancel request is committed for workflow job: 2021-08-15_05_48_27-11108425642055097136.
Aug 15, 2021 4:00:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-15T16:00:52.308Z: Cleaning up.
Aug 15, 2021 4:00:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-15T16:00:52.361Z: Stopping **** pool...
Aug 15, 2021 4:00:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-15T16:00:52.401Z: Stopping **** pool...
Aug 15, 2021 4:03:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-15T16:03:10.027Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Aug 15, 2021 4:03:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-15T16:03:10.056Z: Worker pool stopped.
Aug 15, 2021 4:03:16 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-08-15_05_48_27-11108425642055097136 finished with status CANCELLED.
Aug 15, 2021 4:03:16 PM org.apache.beam.sdk.testutils.metrics.MetricsReader getCounterMetric
SEVERE: Failed to get metric totalBytes.count, from namespace co_gbk
Load test results for test (ID): f67b6992-8042-415a-824e-66b9432213ab and timestamp: 2021-08-15T12:48:21.016000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                       0.0
dataflow_v2_java11_total_bytes_count                      -1.0
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210815124333
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:24a962e431cee4be935c2e1d6ffb43301d7606b14065354818d8814c45c948a2
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210815124333]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:24a962e431cee4be935c2e1d6ffb43301d7606b14065354818d8814c45c948a2]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210815124333] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:24a962e431cee4be935c2e1d6ffb43301d7606b14065354818d8814c45c948a2])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:24a962e431cee4be935c2e1d6ffb43301d7606b14065354818d8814c45c948a2
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:24a962e431cee4be935c2e1d6ffb43301d7606b14065354818d8814c45c948a2
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:24a962e431cee4be935c2e1d6ffb43301d7606b14065354818d8814c45c948a2].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 20m 10s
107 actionable tasks: 76 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/utgt2kjbnm2oy

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #58

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/58/display/redirect?page=changes>

Changes:

[noreply] [BEAM-12757]: Remove call for Json Dag in Portable PipelineRunner as

[noreply] fix confusing function def (#15120)


------------------------------------------
[...truncated 65.71 KB...]
c3e6171450e5: Preparing
194a92b92252: Preparing
b67dbe7c10a6: Preparing
747666797212: Preparing
776a812648d3: Preparing
4eb17eeb05ce: Preparing
f0dbf6ddbdf5: Preparing
d026e7f66f94: Preparing
5d5723d02e5f: Preparing
f3893a5b664b: Preparing
fef215affba3: Preparing
f42aed5f7feb: Preparing
89819bafde36: Preparing
f3d5b8f65132: Preparing
ad83f0aa5c0a: Preparing
5a9a65095453: Preparing
4b0edb23340c: Preparing
afa3e488a0ee: Preparing
f0dbf6ddbdf5: Waiting
f42aed5f7feb: Waiting
89819bafde36: Waiting
f3893a5b664b: Waiting
f3d5b8f65132: Waiting
4b0edb23340c: Waiting
fef215affba3: Waiting
5d5723d02e5f: Waiting
ad83f0aa5c0a: Waiting
d026e7f66f94: Waiting
5a9a65095453: Waiting
776a812648d3: Waiting
4eb17eeb05ce: Waiting
194a92b92252: Pushed
747666797212: Pushed
c3e6171450e5: Pushed
776a812648d3: Pushed
8c1e6472fdc7: Pushed
b67dbe7c10a6: Pushed
f0dbf6ddbdf5: Pushed
d026e7f66f94: Pushed
f42aed5f7feb: Layer already exists
89819bafde36: Layer already exists
f3d5b8f65132: Layer already exists
ad83f0aa5c0a: Layer already exists
5a9a65095453: Layer already exists
4b0edb23340c: Layer already exists
f3893a5b664b: Pushed
afa3e488a0ee: Layer already exists
fef215affba3: Pushed
4eb17eeb05ce: Pushed
5d5723d02e5f: Pushed
20210814124336: digest: sha256:6acdef6c373329ec60f3633891d68cbb77ce24b7029c0ce4eb6b6f01a0c3a4a8 size: 4310

> Task :sdks:java:testing:load-tests:run
Aug 14, 2021 12:48:45 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Aug 14, 2021 12:48:45 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 191 files. Enable logging at DEBUG level to see which files will be staged.
Aug 14, 2021 12:48:46 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Aug 14, 2021 12:48:46 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Aug 14, 2021 12:48:49 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Aug 14, 2021 12:48:49 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <112622 bytes, hash aba6616ba28720f953c8f47cb5e729558ab824acfaa4c71541a2c44e70826db5> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-q6Zha6KHIPlTyPR8tecpVYq4JKz6pMcVQaLETnCCbbU.pb
Aug 14, 2021 12:48:51 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 191 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Aug 14, 2021 12:48:51 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 191 files cached, 0 files newly uploaded in 0 seconds
Aug 14, 2021 12:48:51 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Aug 14, 2021 12:48:52 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12d5c30e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@b887730, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26586b74, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@52f57666, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6e041285, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@267dc982, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@439b15f2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3aa41da1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@74fab04a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c83468e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@272ce069, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3d7fb838, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1c26273d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3a37a501, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7a583586, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3a26ec8d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@17143b3b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@100c8b75, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2bc378f7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@268cbb86]
Aug 14, 2021 12:48:52 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Aug 14, 2021 12:48:52 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Aug 14, 2021 12:48:52 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Aug 14, 2021 12:48:52 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Aug 14, 2021 12:48:52 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@20d87335, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2a8a4e0c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26c89563, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3bd6ba24, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58f437b0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@20f6f88c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4277127c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4c7e978c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@354e7004, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@22ebccb9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5875de6a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@608fe01f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2687725a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5a75ec37, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2c05ff9d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@33956d1a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2e1ddc90, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5b7c8930, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58867cd5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@40e32762]
Aug 14, 2021 12:48:52 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Aug 14, 2021 12:48:52 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Aug 14, 2021 12:48:52 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Aug 14, 2021 12:48:52 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Aug 14, 2021 12:48:52 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Aug 14, 2021 12:48:52 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Aug 14, 2021 12:48:52 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Aug 14, 2021 12:48:52 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Aug 14, 2021 12:48:52 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Aug 14, 2021 12:48:52 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Aug 14, 2021 12:48:52 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Aug 14, 2021 12:48:52 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.33.0-SNAPSHOT
Aug 14, 2021 12:48:53 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-08-14_05_48_52-4755429739656070683?project=apache-beam-testing
Aug 14, 2021 12:48:53 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-08-14_05_48_52-4755429739656070683
Aug 14, 2021 12:48:53 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-08-14_05_48_52-4755429739656070683
Aug 14, 2021 12:48:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-08-14T12:48:59.324Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-08-oji0. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Aug 14, 2021 12:49:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-14T12:49:03.523Z: Worker configuration: e2-standard-2 in us-central1-a.
Aug 14, 2021 12:49:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-14T12:49:04.097Z: Expanding SplittableParDo operations into optimizable parts.
Aug 14, 2021 12:49:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-14T12:49:04.133Z: Expanding CollectionToSingleton operations into optimizable parts.
Aug 14, 2021 12:49:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-14T12:49:04.199Z: Expanding CoGroupByKey operations into optimizable parts.
Aug 14, 2021 12:49:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-14T12:49:04.280Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Aug 14, 2021 12:49:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-14T12:49:04.306Z: Expanding GroupByKey operations into streaming Read/Write steps
Aug 14, 2021 12:49:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-14T12:49:04.373Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Aug 14, 2021 12:49:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-14T12:49:04.486Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Aug 14, 2021 12:49:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-14T12:49:04.522Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Aug 14, 2021 12:49:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-14T12:49:04.553Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Aug 14, 2021 12:49:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-14T12:49:04.585Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Aug 14, 2021 12:49:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-14T12:49:04.619Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Aug 14, 2021 12:49:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-14T12:49:04.653Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Aug 14, 2021 12:49:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-14T12:49:04.697Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Aug 14, 2021 12:49:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-14T12:49:04.729Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Aug 14, 2021 12:49:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-14T12:49:04.762Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Aug 14, 2021 12:49:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-14T12:49:04.797Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Aug 14, 2021 12:49:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-14T12:49:04.831Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Aug 14, 2021 12:49:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-14T12:49:04.874Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Aug 14, 2021 12:49:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-14T12:49:04.907Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Aug 14, 2021 12:49:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-14T12:49:04.944Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Aug 14, 2021 12:49:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-14T12:49:04.968Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Aug 14, 2021 12:49:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-14T12:49:04.994Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Aug 14, 2021 12:49:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-14T12:49:05.027Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Aug 14, 2021 12:49:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-14T12:49:05.064Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Aug 14, 2021 12:49:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-14T12:49:05.086Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Aug 14, 2021 12:49:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-14T12:49:05.117Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Aug 14, 2021 12:49:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-14T12:49:05.150Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Aug 14, 2021 12:49:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-14T12:49:05.185Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Aug 14, 2021 12:49:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-14T12:49:05.207Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Aug 14, 2021 12:49:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-14T12:49:05.556Z: Starting 5 ****s in us-central1-a...
Aug 14, 2021 12:49:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-14T12:49:12.685Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Aug 14, 2021 12:49:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-14T12:49:41.326Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Aug 14, 2021 12:50:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-14T12:50:35.628Z: Workers have started successfully.
Aug 14, 2021 12:50:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-14T12:50:35.663Z: Workers have started successfully.
Aug 14, 2021 4:00:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-14T16:00:35.903Z: Cancel request is committed for workflow job: 2021-08-14_05_48_52-4755429739656070683.
Aug 14, 2021 4:00:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-14T16:00:35.998Z: Cleaning up.
Aug 14, 2021 4:00:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-14T16:00:36.070Z: Stopping **** pool...
Aug 14, 2021 4:00:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-14T16:00:36.143Z: Stopping **** pool...
Aug 14, 2021 4:02:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-14T16:02:47.432Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Aug 14, 2021 4:02:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-14T16:02:47.473Z: Worker pool stopped.
Aug 14, 2021 4:02:52 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-08-14_05_48_52-4755429739656070683 finished with status CANCELLED.
Aug 14, 2021 4:02:53 PM org.apache.beam.sdk.testutils.metrics.MetricsReader getCounterMetric
SEVERE: Failed to get metric totalBytes.count, from namespace co_gbk
Load test results for test (ID): f476481f-7faa-4c39-857e-3303bd30908a and timestamp: 2021-08-14T12:48:46.281000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                       0.0
dataflow_v2_java11_total_bytes_count                      -1.0
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210814124336
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:6acdef6c373329ec60f3633891d68cbb77ce24b7029c0ce4eb6b6f01a0c3a4a8
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210814124336]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:6acdef6c373329ec60f3633891d68cbb77ce24b7029c0ce4eb6b6f01a0c3a4a8]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210814124336] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:6acdef6c373329ec60f3633891d68cbb77ce24b7029c0ce4eb6b6f01a0c3a4a8])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:6acdef6c373329ec60f3633891d68cbb77ce24b7029c0ce4eb6b6f01a0c3a4a8
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:6acdef6c373329ec60f3633891d68cbb77ce24b7029c0ce4eb6b6f01a0c3a4a8
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:6acdef6c373329ec60f3633891d68cbb77ce24b7029c0ce4eb6b6f01a0c3a4a8].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 35s
107 actionable tasks: 76 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/wm4yjr2ugjlpc

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #57

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/57/display/redirect?page=changes>

Changes:

[bsindhwani] Python Kata - Event Time Triggers

[bsindhwani] Python Kata - Event Time Triggers - Added license to task-info.yaml

[bsindhwani] Python Kata - Event Time Triggers - refactored the accumulation mode

[bsindhwani] Python Kata - Event Time Triggers - task description corrected to

[bsindhwani] Python Kata - Event Time Triggers - added streaming options

[bsindhwani] Python Kata - Event Time Triggers - add license text in task-info.yaml

[bsindhwani] Python Kata - Early Triggers

[bsindhwani] Python Kata - Event Time Triggers & Early Triggers - task.md - corrected

[bsindhwani] Python Kata - Window Accumulation Mode

[bsindhwani] Python Kata - Event Time Triggers - removed the external transform

[noreply] Loosen typing extensions requirement

[bsindhwani] Python Kata - added license to course-info.yaml

[Kyle Weaver] [BEAM-12719] ZetaSQL docs: replace out of date table with link to

[Robert Burke] [BEAM-11779] Remove appliance restriction.

[ryanthompson591] Add retry_gcs_file_copy when 500 errros happen from server

[noreply] [BEAM-6374] Emit PCollection metrics from GoSDK (#15289)

[noreply] Merge pull request #15165 from [BEAM-12593] Verify DataFrame API on

[noreply] Merge pull request #15265 from [BEAM-12545] Extend FileSink by

[noreply] [BEAM-12453]: Add interface to access I/O topic information for a Samza

[noreply] [BEAM-10212] Guard state caching with experiment (#15319)


------------------------------------------
[...truncated 65.91 KB...]
INFO: Uploading <112684 bytes, hash 412fbf0bb248041027c7b73f69e5374050f6a72482642dff07de8f2c05318fc3> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-QS-_C7JIBBAnx7c_aeU3QFD2pySCZC3_B96PLAUxj8M.pb
Aug 13, 2021 12:48:17 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 191 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Aug 13, 2021 12:48:17 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 191 files cached, 0 files newly uploaded in 0 seconds
Aug 13, 2021 12:48:17 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Aug 13, 2021 12:48:17 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3d96fa9e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3b545206, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77bb48d5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@181d8899, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12d5c30e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@b887730, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26586b74, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@52f57666, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6e041285, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@267dc982, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@439b15f2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3aa41da1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@74fab04a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c83468e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@272ce069, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3d7fb838, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1c26273d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3a37a501, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7a583586, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3a26ec8d]
Aug 13, 2021 12:48:17 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Aug 13, 2021 12:48:17 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Aug 13, 2021 12:48:17 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Aug 13, 2021 12:48:17 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Aug 13, 2021 12:48:17 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43f9dd56, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1d12e953, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@57cb70be, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2d4608a6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@20d87335, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2a8a4e0c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26c89563, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3bd6ba24, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58f437b0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@20f6f88c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4277127c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4c7e978c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@354e7004, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@22ebccb9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5875de6a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@608fe01f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2687725a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5a75ec37, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2c05ff9d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@33956d1a]
Aug 13, 2021 12:48:17 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Aug 13, 2021 12:48:17 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Aug 13, 2021 12:48:17 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Aug 13, 2021 12:48:17 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Aug 13, 2021 12:48:17 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Aug 13, 2021 12:48:17 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Aug 13, 2021 12:48:17 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Aug 13, 2021 12:48:17 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Aug 13, 2021 12:48:17 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Aug 13, 2021 12:48:17 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Aug 13, 2021 12:48:17 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Aug 13, 2021 12:48:17 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.33.0-SNAPSHOT
Aug 13, 2021 12:48:18 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-08-13_05_48_17-10441454629149339605?project=apache-beam-testing
Aug 13, 2021 12:48:18 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-08-13_05_48_17-10441454629149339605
Aug 13, 2021 12:48:18 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-08-13_05_48_17-10441454629149339605
Aug 13, 2021 12:48:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-08-13T12:48:27.702Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-08-rwgb. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Aug 13, 2021 12:48:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-13T12:48:31.074Z: Worker configuration: e2-standard-2 in us-central1-a.
Aug 13, 2021 12:48:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-13T12:48:31.814Z: Expanding SplittableParDo operations into optimizable parts.
Aug 13, 2021 12:48:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-13T12:48:31.850Z: Expanding CollectionToSingleton operations into optimizable parts.
Aug 13, 2021 12:48:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-13T12:48:31.964Z: Expanding CoGroupByKey operations into optimizable parts.
Aug 13, 2021 12:48:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-13T12:48:32.043Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Aug 13, 2021 12:48:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-13T12:48:32.085Z: Expanding GroupByKey operations into streaming Read/Write steps
Aug 13, 2021 12:48:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-13T12:48:32.134Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Aug 13, 2021 12:48:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-13T12:48:32.285Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Aug 13, 2021 12:48:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-13T12:48:32.371Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Aug 13, 2021 12:48:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-13T12:48:32.421Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Aug 13, 2021 12:48:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-13T12:48:32.458Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Aug 13, 2021 12:48:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-13T12:48:32.493Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Aug 13, 2021 12:48:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-13T12:48:32.521Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Aug 13, 2021 12:48:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-13T12:48:32.556Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Aug 13, 2021 12:48:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-13T12:48:32.584Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Aug 13, 2021 12:48:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-13T12:48:32.621Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Aug 13, 2021 12:48:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-13T12:48:32.657Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Aug 13, 2021 12:48:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-13T12:48:32.691Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Aug 13, 2021 12:48:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-13T12:48:32.716Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Aug 13, 2021 12:48:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-13T12:48:32.750Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Aug 13, 2021 12:48:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-13T12:48:32.778Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Aug 13, 2021 12:48:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-13T12:48:32.804Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Aug 13, 2021 12:48:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-13T12:48:32.840Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Aug 13, 2021 12:48:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-13T12:48:32.878Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Aug 13, 2021 12:48:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-13T12:48:32.908Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Aug 13, 2021 12:48:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-13T12:48:32.933Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Aug 13, 2021 12:48:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-13T12:48:32.967Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Aug 13, 2021 12:48:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-13T12:48:33.000Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Aug 13, 2021 12:48:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-13T12:48:33.036Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Aug 13, 2021 12:48:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-13T12:48:33.074Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Aug 13, 2021 12:48:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-13T12:48:33.482Z: Starting 5 ****s in us-central1-a...
Aug 13, 2021 12:48:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-13T12:48:42.166Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Aug 13, 2021 12:49:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-13T12:49:17.470Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Aug 13, 2021 12:50:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-13T12:50:11.187Z: Workers have started successfully.
Aug 13, 2021 12:50:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-13T12:50:11.256Z: Workers have started successfully.
Aug 13, 2021 3:15:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-13T15:15:37.095Z: Staged package kafka-clients-2.4.1-or4Vrm01S3aXE64MuzGE8iWqFvyYIFseskr9n8KjYNk.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/kafka-clients-2.4.1-or4Vrm01S3aXE64MuzGE8iWqFvyYIFseskr9n8KjYNk.jar' is inaccessible.
Aug 13, 2021 3:15:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-13T15:15:37.231Z: Staged package lz4-java-1.6.0-0ilUWqKx1SA8h2YUvbz_yswwNpf0-PJvdk4dbB7S5BY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/lz4-java-1.6.0-0ilUWqKx1SA8h2YUvbz_yswwNpf0-PJvdk4dbB7S5BY.jar' is inaccessible.
Aug 13, 2021 3:15:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-08-13T15:15:38.266Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Aug 13, 2021 3:18:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-08-13T15:18:37.717Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Aug 13, 2021 3:21:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-13T15:21:37.796Z: Staged package kafka-clients-2.4.1-or4Vrm01S3aXE64MuzGE8iWqFvyYIFseskr9n8KjYNk.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/kafka-clients-2.4.1-or4Vrm01S3aXE64MuzGE8iWqFvyYIFseskr9n8KjYNk.jar' is inaccessible.
Aug 13, 2021 3:21:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-13T15:21:37.958Z: Staged package lz4-java-1.6.0-0ilUWqKx1SA8h2YUvbz_yswwNpf0-PJvdk4dbB7S5BY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/lz4-java-1.6.0-0ilUWqKx1SA8h2YUvbz_yswwNpf0-PJvdk4dbB7S5BY.jar' is inaccessible.
Aug 13, 2021 3:21:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-08-13T15:21:38.935Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Aug 13, 2021 3:24:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-08-13T15:24:36.781Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Aug 13, 2021 3:27:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-13T15:27:36.930Z: Staged package kafka-clients-2.4.1-or4Vrm01S3aXE64MuzGE8iWqFvyYIFseskr9n8KjYNk.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/kafka-clients-2.4.1-or4Vrm01S3aXE64MuzGE8iWqFvyYIFseskr9n8KjYNk.jar' is inaccessible.
Aug 13, 2021 3:27:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-13T15:27:37.035Z: Staged package lz4-java-1.6.0-0ilUWqKx1SA8h2YUvbz_yswwNpf0-PJvdk4dbB7S5BY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/lz4-java-1.6.0-0ilUWqKx1SA8h2YUvbz_yswwNpf0-PJvdk4dbB7S5BY.jar' is inaccessible.
Aug 13, 2021 3:27:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-08-13T15:27:37.900Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Aug 13, 2021 3:30:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-08-13T15:30:38.947Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Aug 13, 2021 3:33:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-13T15:33:38.527Z: Staged package kafka-clients-2.4.1-or4Vrm01S3aXE64MuzGE8iWqFvyYIFseskr9n8KjYNk.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/kafka-clients-2.4.1-or4Vrm01S3aXE64MuzGE8iWqFvyYIFseskr9n8KjYNk.jar' is inaccessible.
Aug 13, 2021 3:33:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-13T15:33:38.659Z: Staged package lz4-java-1.6.0-0ilUWqKx1SA8h2YUvbz_yswwNpf0-PJvdk4dbB7S5BY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/lz4-java-1.6.0-0ilUWqKx1SA8h2YUvbz_yswwNpf0-PJvdk4dbB7S5BY.jar' is inaccessible.
Aug 13, 2021 3:33:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-08-13T15:33:40.516Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Aug 13, 2021 3:36:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-08-13T15:36:37.668Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Aug 13, 2021 3:39:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-13T15:39:36.781Z: Staged package kafka-clients-2.4.1-or4Vrm01S3aXE64MuzGE8iWqFvyYIFseskr9n8KjYNk.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/kafka-clients-2.4.1-or4Vrm01S3aXE64MuzGE8iWqFvyYIFseskr9n8KjYNk.jar' is inaccessible.
Aug 13, 2021 3:39:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-13T15:39:36.925Z: Staged package lz4-java-1.6.0-0ilUWqKx1SA8h2YUvbz_yswwNpf0-PJvdk4dbB7S5BY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/lz4-java-1.6.0-0ilUWqKx1SA8h2YUvbz_yswwNpf0-PJvdk4dbB7S5BY.jar' is inaccessible.
Aug 13, 2021 3:39:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-08-13T15:39:37.855Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Aug 13, 2021 3:42:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-08-13T15:42:37.428Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Aug 13, 2021 3:45:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-13T15:45:35.898Z: Staged package kafka-clients-2.4.1-or4Vrm01S3aXE64MuzGE8iWqFvyYIFseskr9n8KjYNk.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/kafka-clients-2.4.1-or4Vrm01S3aXE64MuzGE8iWqFvyYIFseskr9n8KjYNk.jar' is inaccessible.
Aug 13, 2021 3:45:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-13T15:45:35.961Z: Staged package lz4-java-1.6.0-0ilUWqKx1SA8h2YUvbz_yswwNpf0-PJvdk4dbB7S5BY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/lz4-java-1.6.0-0ilUWqKx1SA8h2YUvbz_yswwNpf0-PJvdk4dbB7S5BY.jar' is inaccessible.
Aug 13, 2021 3:45:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-08-13T15:45:36.689Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Aug 13, 2021 3:48:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-08-13T15:48:38.336Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Aug 13, 2021 3:51:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-13T15:51:36.237Z: Staged package kafka-clients-2.4.1-or4Vrm01S3aXE64MuzGE8iWqFvyYIFseskr9n8KjYNk.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/kafka-clients-2.4.1-or4Vrm01S3aXE64MuzGE8iWqFvyYIFseskr9n8KjYNk.jar' is inaccessible.
Aug 13, 2021 3:51:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-13T15:51:36.312Z: Staged package lz4-java-1.6.0-0ilUWqKx1SA8h2YUvbz_yswwNpf0-PJvdk4dbB7S5BY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/lz4-java-1.6.0-0ilUWqKx1SA8h2YUvbz_yswwNpf0-PJvdk4dbB7S5BY.jar' is inaccessible.
Aug 13, 2021 3:51:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-08-13T15:51:37.107Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Aug 13, 2021 3:54:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-08-13T15:54:38.092Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Aug 13, 2021 3:57:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-13T15:57:36.673Z: Staged package kafka-clients-2.4.1-or4Vrm01S3aXE64MuzGE8iWqFvyYIFseskr9n8KjYNk.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/kafka-clients-2.4.1-or4Vrm01S3aXE64MuzGE8iWqFvyYIFseskr9n8KjYNk.jar' is inaccessible.
Aug 13, 2021 3:57:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-13T15:57:36.753Z: Staged package lz4-java-1.6.0-0ilUWqKx1SA8h2YUvbz_yswwNpf0-PJvdk4dbB7S5BY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/lz4-java-1.6.0-0ilUWqKx1SA8h2YUvbz_yswwNpf0-PJvdk4dbB7S5BY.jar' is inaccessible.
Aug 13, 2021 3:57:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-08-13T15:57:37.607Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Aug 13, 2021 4:00:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-13T16:00:37.320Z: Cancel request is committed for workflow job: 2021-08-13_05_48_17-10441454629149339605.
Aug 13, 2021 4:00:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-08-13T16:00:38.421Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Aug 13, 2021 4:00:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-13T16:00:38.449Z: Cleaning up.
Aug 13, 2021 4:00:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-13T16:00:38.528Z: Stopping **** pool...
Aug 13, 2021 4:00:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-13T16:00:38.571Z: Stopping **** pool...
Aug 13, 2021 4:02:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-13T16:02:57.003Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Aug 13, 2021 4:02:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-13T16:02:57.045Z: Worker pool stopped.
Aug 13, 2021 4:03:03 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-08-13_05_48_17-10441454629149339605 finished with status CANCELLED.
Aug 13, 2021 4:03:03 PM org.apache.beam.sdk.testutils.metrics.MetricsReader getCounterMetric
SEVERE: Failed to get metric totalBytes.count, from namespace co_gbk
Load test results for test (ID): d75f5618-6dee-48cd-97ed-e314caf884f2 and timestamp: 2021-08-13T12:48:13.180000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                       0.0
dataflow_v2_java11_total_bytes_count                      -1.0
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210813124331
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:5a5908d4a78eddc74752b9ada201cc2d891c7b117eebbb18e4c3e0954087a68f
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210813124331]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:5a5908d4a78eddc74752b9ada201cc2d891c7b117eebbb18e4c3e0954087a68f]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210813124331] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:5a5908d4a78eddc74752b9ada201cc2d891c7b117eebbb18e4c3e0954087a68f])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:5a5908d4a78eddc74752b9ada201cc2d891c7b117eebbb18e4c3e0954087a68f
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:5a5908d4a78eddc74752b9ada201cc2d891c7b117eebbb18e4c3e0954087a68f
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:5a5908d4a78eddc74752b9ada201cc2d891c7b117eebbb18e4c3e0954087a68f].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 49s
107 actionable tasks: 76 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/hlx22xifbuhwi

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #56

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/56/display/redirect?page=changes>

Changes:

[marwant] Move misplaced cloud healthcare update info line from template to

[Robert Burke] [BEAM-12738][GoSDK] Fix Dataflow Job URL

[eugene.nikolayev] Fix docs typos in contrib guide.

[noreply] Merge pull request #15126 from [BEAM-12665] Add option in ReadAll

[noreply] Update ptransform.py error message (#15286)

[noreply] [GoSDK Infra] Limit simultaneous tests binaries to 3. (#15321)


------------------------------------------
[...truncated 65.39 KB...]
99f4aee1b185: Preparing
9a2138e32ab3: Preparing
e3244eebb4f3: Preparing
91a6356f528b: Preparing
11d8f7f27ce5: Preparing
d182537977e9: Preparing
f42aed5f7feb: Preparing
89819bafde36: Preparing
f3d5b8f65132: Preparing
ad83f0aa5c0a: Preparing
5a9a65095453: Preparing
4b0edb23340c: Preparing
afa3e488a0ee: Preparing
89819bafde36: Waiting
f3d5b8f65132: Waiting
4b0edb23340c: Waiting
e8e9a224b028: Waiting
afa3e488a0ee: Waiting
91a6356f528b: Waiting
ad83f0aa5c0a: Waiting
e3244eebb4f3: Waiting
99f4aee1b185: Waiting
5a9a65095453: Waiting
f42aed5f7feb: Waiting
d182537977e9: Waiting
9a2138e32ab3: Waiting
11d8f7f27ce5: Waiting
b2c1bfe2192f: Pushed
64f7ae5bbc63: Pushed
c0b548fe1a3b: Pushed
510df1f4fa92: Pushed
46fa13deb8e7: Pushed
e8e9a224b028: Pushed
9a2138e32ab3: Pushed
e3244eebb4f3: Pushed
f42aed5f7feb: Layer already exists
89819bafde36: Layer already exists
f3d5b8f65132: Layer already exists
ad83f0aa5c0a: Layer already exists
5a9a65095453: Layer already exists
4b0edb23340c: Layer already exists
afa3e488a0ee: Layer already exists
11d8f7f27ce5: Pushed
d182537977e9: Pushed
99f4aee1b185: Pushed
91a6356f528b: Pushed
20210812124337: digest: sha256:a513894e94f425154bd6398dd3c6eeefb389ecc105f19c2034e2cf127ceadf95 size: 4310

> Task :sdks:java:testing:load-tests:run
Aug 12, 2021 12:50:54 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Aug 12, 2021 12:50:54 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 191 files. Enable logging at DEBUG level to see which files will be staged.
Aug 12, 2021 12:50:55 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Aug 12, 2021 12:50:55 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Aug 12, 2021 12:50:57 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Aug 12, 2021 12:50:57 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <112684 bytes, hash b0fc3c7f12e5007c583360f8edf754509efabaa38d1a303925f516ee1e126918> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-sPw8fxLlAHxYM2D47fdUUJ76uqONGjA5JfUW7h4SaRg.pb
Aug 12, 2021 12:50:59 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 191 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Aug 12, 2021 12:51:00 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 191 files cached, 0 files newly uploaded in 0 seconds
Aug 12, 2021 12:51:00 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Aug 12, 2021 12:51:00 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3d96fa9e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3b545206, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77bb48d5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@181d8899, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12d5c30e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@b887730, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26586b74, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@52f57666, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6e041285, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@267dc982, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@439b15f2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3aa41da1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@74fab04a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c83468e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@272ce069, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3d7fb838, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1c26273d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3a37a501, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7a583586, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3a26ec8d]
Aug 12, 2021 12:51:00 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Aug 12, 2021 12:51:00 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Aug 12, 2021 12:51:00 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Aug 12, 2021 12:51:00 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Aug 12, 2021 12:51:00 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58f437b0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@20f6f88c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4277127c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4c7e978c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@354e7004, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@22ebccb9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5875de6a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@608fe01f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2687725a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5a75ec37, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2c05ff9d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@33956d1a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2e1ddc90, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5b7c8930, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58867cd5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@40e32762, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5a8ab2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3f50b680, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@54db056b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@d5af0a5]
Aug 12, 2021 12:51:00 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Aug 12, 2021 12:51:00 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Aug 12, 2021 12:51:00 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Aug 12, 2021 12:51:00 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Aug 12, 2021 12:51:00 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Aug 12, 2021 12:51:00 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Aug 12, 2021 12:51:00 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Aug 12, 2021 12:51:00 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Aug 12, 2021 12:51:00 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Aug 12, 2021 12:51:00 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Aug 12, 2021 12:51:00 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Aug 12, 2021 12:51:00 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.33.0-SNAPSHOT
Aug 12, 2021 12:51:01 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-08-12_05_51_00-5506486706734605957?project=apache-beam-testing
Aug 12, 2021 12:51:01 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-08-12_05_51_00-5506486706734605957
Aug 12, 2021 12:51:01 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-08-12_05_51_00-5506486706734605957
Aug 12, 2021 12:51:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-08-12T12:51:09.060Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-08-lupg. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Aug 12, 2021 12:51:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-12T12:51:13.380Z: Worker configuration: e2-standard-2 in us-central1-a.
Aug 12, 2021 12:51:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-12T12:51:14.097Z: Expanding SplittableParDo operations into optimizable parts.
Aug 12, 2021 12:51:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-12T12:51:14.138Z: Expanding CollectionToSingleton operations into optimizable parts.
Aug 12, 2021 12:51:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-12T12:51:14.243Z: Expanding CoGroupByKey operations into optimizable parts.
Aug 12, 2021 12:51:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-12T12:51:14.321Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Aug 12, 2021 12:51:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-12T12:51:14.353Z: Expanding GroupByKey operations into streaming Read/Write steps
Aug 12, 2021 12:51:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-12T12:51:14.421Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Aug 12, 2021 12:51:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-12T12:51:14.522Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Aug 12, 2021 12:51:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-12T12:51:14.557Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Aug 12, 2021 12:51:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-12T12:51:14.605Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Aug 12, 2021 12:51:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-12T12:51:14.641Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Aug 12, 2021 12:51:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-12T12:51:14.675Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Aug 12, 2021 12:51:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-12T12:51:14.709Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Aug 12, 2021 12:51:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-12T12:51:14.738Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Aug 12, 2021 12:51:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-12T12:51:14.781Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Aug 12, 2021 12:51:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-12T12:51:14.812Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Aug 12, 2021 12:51:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-12T12:51:14.848Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Aug 12, 2021 12:51:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-12T12:51:14.873Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Aug 12, 2021 12:51:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-12T12:51:14.904Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Aug 12, 2021 12:51:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-12T12:51:14.937Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Aug 12, 2021 12:51:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-12T12:51:14.971Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Aug 12, 2021 12:51:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-12T12:51:15.005Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Aug 12, 2021 12:51:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-12T12:51:15.041Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Aug 12, 2021 12:51:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-12T12:51:15.077Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Aug 12, 2021 12:51:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-12T12:51:15.110Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Aug 12, 2021 12:51:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-12T12:51:15.154Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Aug 12, 2021 12:51:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-12T12:51:15.193Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Aug 12, 2021 12:51:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-12T12:51:15.222Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Aug 12, 2021 12:51:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-12T12:51:15.271Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Aug 12, 2021 12:51:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-12T12:51:15.304Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Aug 12, 2021 12:51:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-12T12:51:15.753Z: Starting 5 ****s in us-central1-a...
Aug 12, 2021 12:51:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-12T12:51:26.940Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Aug 12, 2021 12:52:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-12T12:52:00.432Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Aug 12, 2021 12:52:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-12T12:52:58.419Z: Workers have started successfully.
Aug 12, 2021 12:52:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-12T12:52:58.449Z: Workers have started successfully.
Aug 12, 2021 2:45:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-12T14:45:16.953Z: Staged package error_prone_annotations-2.3.4-uvfW6pfOYGxT4RtoVLpfLOfvXCTd3wr6GNEmC9JbACw.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/error_prone_annotations-2.3.4-uvfW6pfOYGxT4RtoVLpfLOfvXCTd3wr6GNEmC9JbACw.jar' is inaccessible.
Aug 12, 2021 2:45:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-08-12T14:45:19.685Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Aug 12, 2021 4:00:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-12T16:00:44.924Z: Cancel request is committed for workflow job: 2021-08-12_05_51_00-5506486706734605957.
Aug 12, 2021 4:00:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-12T16:00:44.988Z: Cleaning up.
Aug 12, 2021 4:00:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-12T16:00:45.073Z: Stopping **** pool...
Aug 12, 2021 4:00:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-12T16:00:45.125Z: Stopping **** pool...
Aug 12, 2021 4:03:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-12T16:03:05.598Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Aug 12, 2021 4:03:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-12T16:03:05.668Z: Worker pool stopped.
Aug 12, 2021 4:03:10 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-08-12_05_51_00-5506486706734605957 finished with status CANCELLED.
Aug 12, 2021 4:03:11 PM org.apache.beam.sdk.testutils.metrics.MetricsReader getCounterMetric
SEVERE: Failed to get metric totalBytes.count, from namespace co_gbk
Load test results for test (ID): e245408b-8799-45ff-96c6-a8180a9b2e89 and timestamp: 2021-08-12T12:50:55.332000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                       0.0
dataflow_v2_java11_total_bytes_count                      -1.0
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210812124337
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:a513894e94f425154bd6398dd3c6eeefb389ecc105f19c2034e2cf127ceadf95
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210812124337]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:a513894e94f425154bd6398dd3c6eeefb389ecc105f19c2034e2cf127ceadf95]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210812124337] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:a513894e94f425154bd6398dd3c6eeefb389ecc105f19c2034e2cf127ceadf95])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:a513894e94f425154bd6398dd3c6eeefb389ecc105f19c2034e2cf127ceadf95
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:a513894e94f425154bd6398dd3c6eeefb389ecc105f19c2034e2cf127ceadf95
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:a513894e94f425154bd6398dd3c6eeefb389ecc105f19c2034e2cf127ceadf95].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 53s
107 actionable tasks: 76 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/w6kzlb4m5o37u

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #55

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/55/display/redirect?page=changes>

Changes:

[andyxu] Add support for python sdk container to turn on google cloud profiler

[andyxu] format file

[marwant] Update Google Cloud Healthcare API version from v1beta1 to GA

[chamikaramj] Fix typo

[zyichi] Update beam dataflow python container versions.

[noreply] [BEAM-12403][BEAM-12348] Add User State Support for Portable SamzaRunner


------------------------------------------
[...truncated 67.52 KB...]
> Task :sdks:java:testing:load-tests:run
Aug 11, 2021 12:48:41 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Aug 11, 2021 12:48:42 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 191 files. Enable logging at DEBUG level to see which files will be staged.
Aug 11, 2021 12:48:42 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Aug 11, 2021 12:48:42 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Aug 11, 2021 12:48:44 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Aug 11, 2021 12:48:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <112622 bytes, hash df0280aa2ed3190a1fd55e36dc68b25cc16d188becaf552a33fbba41a331a514> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-3wKAqi7TGQof1V423GiyXMFtGIvsr1UqM_u6QaMxpRQ.pb
Aug 11, 2021 12:48:46 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 191 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Aug 11, 2021 12:48:48 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 191 files cached, 0 files newly uploaded in 2 seconds
Aug 11, 2021 12:48:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Aug 11, 2021 12:48:48 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12d5c30e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@b887730, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26586b74, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@52f57666, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6e041285, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@267dc982, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@439b15f2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3aa41da1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@74fab04a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c83468e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@272ce069, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3d7fb838, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1c26273d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3a37a501, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7a583586, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3a26ec8d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@17143b3b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@100c8b75, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2bc378f7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@268cbb86]
Aug 11, 2021 12:48:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Aug 11, 2021 12:48:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Aug 11, 2021 12:48:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Aug 11, 2021 12:48:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Aug 11, 2021 12:48:48 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@20d87335, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2a8a4e0c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26c89563, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3bd6ba24, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58f437b0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@20f6f88c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4277127c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4c7e978c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@354e7004, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@22ebccb9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5875de6a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@608fe01f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2687725a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5a75ec37, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2c05ff9d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@33956d1a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2e1ddc90, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5b7c8930, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58867cd5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@40e32762]
Aug 11, 2021 12:48:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Aug 11, 2021 12:48:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Aug 11, 2021 12:48:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Aug 11, 2021 12:48:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Aug 11, 2021 12:48:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Aug 11, 2021 12:48:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Aug 11, 2021 12:48:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Aug 11, 2021 12:48:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Aug 11, 2021 12:48:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Aug 11, 2021 12:48:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Aug 11, 2021 12:48:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Aug 11, 2021 12:48:48 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.33.0-SNAPSHOT
Aug 11, 2021 12:48:49 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-08-11_05_48_49-15281179841542251974?project=apache-beam-testing
Aug 11, 2021 12:48:49 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-08-11_05_48_49-15281179841542251974
Aug 11, 2021 12:48:49 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-08-11_05_48_49-15281179841542251974
Aug 11, 2021 12:49:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-08-11T12:49:15.623Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-08-hes8. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Aug 11, 2021 12:49:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-11T12:49:19.150Z: Worker configuration: e2-standard-2 in us-central1-a.
Aug 11, 2021 12:49:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-11T12:49:19.751Z: Expanding SplittableParDo operations into optimizable parts.
Aug 11, 2021 12:49:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-11T12:49:19.780Z: Expanding CollectionToSingleton operations into optimizable parts.
Aug 11, 2021 12:49:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-11T12:49:19.851Z: Expanding CoGroupByKey operations into optimizable parts.
Aug 11, 2021 12:49:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-11T12:49:19.935Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Aug 11, 2021 12:49:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-11T12:49:19.962Z: Expanding GroupByKey operations into streaming Read/Write steps
Aug 11, 2021 12:49:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-11T12:49:20.006Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Aug 11, 2021 12:49:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-11T12:49:20.105Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Aug 11, 2021 12:49:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-11T12:49:20.133Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Aug 11, 2021 12:49:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-11T12:49:20.162Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Aug 11, 2021 12:49:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-11T12:49:20.194Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Aug 11, 2021 12:49:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-11T12:49:20.214Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Aug 11, 2021 12:49:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-11T12:49:20.243Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Aug 11, 2021 12:49:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-11T12:49:20.267Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Aug 11, 2021 12:49:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-11T12:49:20.296Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Aug 11, 2021 12:49:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-11T12:49:20.323Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Aug 11, 2021 12:49:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-11T12:49:20.348Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Aug 11, 2021 12:49:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-11T12:49:20.391Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Aug 11, 2021 12:49:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-11T12:49:20.420Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Aug 11, 2021 12:49:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-11T12:49:20.452Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Aug 11, 2021 12:49:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-11T12:49:20.483Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Aug 11, 2021 12:49:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-11T12:49:20.503Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Aug 11, 2021 12:49:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-11T12:49:20.533Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Aug 11, 2021 12:49:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-11T12:49:20.562Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Aug 11, 2021 12:49:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-11T12:49:20.595Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Aug 11, 2021 12:49:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-11T12:49:20.629Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Aug 11, 2021 12:49:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-11T12:49:20.659Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Aug 11, 2021 12:49:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-11T12:49:20.688Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Aug 11, 2021 12:49:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-11T12:49:20.725Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Aug 11, 2021 12:49:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-11T12:49:20.754Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Aug 11, 2021 12:49:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-11T12:49:21.090Z: Starting 5 ****s in us-central1-a...
Aug 11, 2021 12:49:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-11T12:49:26.040Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Aug 11, 2021 12:50:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-11T12:50:16.622Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Aug 11, 2021 12:51:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-11T12:51:11.590Z: Workers have started successfully.
Aug 11, 2021 12:51:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-11T12:51:11.621Z: Workers have started successfully.
Aug 11, 2021 2:25:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-11T14:25:21.480Z: Staged package aws-java-sdk-cloudwatch-1.11.974-Ub2yicrz1p27ushPndFcVBigc3SuIZ4G-Yaq45B1iUo.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/aws-java-sdk-cloudwatch-1.11.974-Ub2yicrz1p27ushPndFcVBigc3SuIZ4G-Yaq45B1iUo.jar' is inaccessible.
Aug 11, 2021 2:25:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-11T14:25:21.565Z: Staged package aws-java-sdk-core-1.11.974-s3lPhICKsq1xExGOyj03VQHRqOsuX5YUc-UUlnhiA64.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/aws-java-sdk-core-1.11.974-s3lPhICKsq1xExGOyj03VQHRqOsuX5YUc-UUlnhiA64.jar' is inaccessible.
Aug 11, 2021 2:25:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-11T14:25:21.613Z: Staged package aws-java-sdk-dynamodb-1.11.974-RLvIDrX-TDIDN06gNnaHm9dQbuHCD_XSBesOuniSTk4.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/aws-java-sdk-dynamodb-1.11.974-RLvIDrX-TDIDN06gNnaHm9dQbuHCD_XSBesOuniSTk4.jar' is inaccessible.
Aug 11, 2021 2:25:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-11T14:25:21.662Z: Staged package aws-java-sdk-kinesis-1.11.974-kbU_CjbRpkrhhFeoyIMQA81nTYmniJY77w8Smd7PLpY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/aws-java-sdk-kinesis-1.11.974-kbU_CjbRpkrhhFeoyIMQA81nTYmniJY77w8Smd7PLpY.jar' is inaccessible.
Aug 11, 2021 2:25:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-11T14:25:21.713Z: Staged package aws-java-sdk-kms-1.11.974-HWH_mHclw3Wv4_GJ56x62xFDHSPSkZWjeKAlx40XrTk.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/aws-java-sdk-kms-1.11.974-HWH_mHclw3Wv4_GJ56x62xFDHSPSkZWjeKAlx40XrTk.jar' is inaccessible.
Aug 11, 2021 2:25:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-11T14:25:21.767Z: Staged package aws-java-sdk-s3-1.11.974-Qgd0wgaOuI_uyxxQI4Ulljk9bZEq-3JZArQF0zU5yzA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/aws-java-sdk-s3-1.11.974-Qgd0wgaOuI_uyxxQI4Ulljk9bZEq-3JZArQF0zU5yzA.jar' is inaccessible.
Aug 11, 2021 2:25:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-11T14:25:22.540Z: Staged package commons-io-2.6--HfTBGYKwqFC84ZbrfyXHex-1zx0fH-NXS9ROcpzZRM.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/commons-io-2.6--HfTBGYKwqFC84ZbrfyXHex-1zx0fH-NXS9ROcpzZRM.jar' is inaccessible.
Aug 11, 2021 2:25:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-11T14:25:22.591Z: Staged package commons-lang-2.6-UPEbCfh3wpTVbyRGP0fSj5Kc9QRPZIZhwPDPuumi9Jw.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/commons-lang-2.6-UPEbCfh3wpTVbyRGP0fSj5Kc9QRPZIZhwPDPuumi9Jw.jar' is inaccessible.
Aug 11, 2021 2:25:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-11T14:25:24.101Z: Staged package ion-java-1.0.2-DRJ7IFofzgq8KjdXoEF0hlG8ZsFc9MBZusWDOyfUcaU.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/ion-java-1.0.2-DRJ7IFofzgq8KjdXoEF0hlG8ZsFc9MBZusWDOyfUcaU.jar' is inaccessible.
Aug 11, 2021 2:25:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-11T14:25:24.342Z: Staged package jmespath-java-1.11.974-QaUt5Hnw6wd7ULTQ5Oj4j_zpRr-rnMCdfGwiwkSo8-0.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/jmespath-java-1.11.974-QaUt5Hnw6wd7ULTQ5Oj4j_zpRr-rnMCdfGwiwkSo8-0.jar' is inaccessible.
Aug 11, 2021 2:25:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-08-11T14:25:25.336Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Aug 11, 2021 2:28:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-08-11T14:28:24.973Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Aug 11, 2021 2:31:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-11T14:31:21.508Z: Staged package aws-java-sdk-cloudwatch-1.11.974-Ub2yicrz1p27ushPndFcVBigc3SuIZ4G-Yaq45B1iUo.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/aws-java-sdk-cloudwatch-1.11.974-Ub2yicrz1p27ushPndFcVBigc3SuIZ4G-Yaq45B1iUo.jar' is inaccessible.
Aug 11, 2021 2:31:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-11T14:31:21.549Z: Staged package aws-java-sdk-core-1.11.974-s3lPhICKsq1xExGOyj03VQHRqOsuX5YUc-UUlnhiA64.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/aws-java-sdk-core-1.11.974-s3lPhICKsq1xExGOyj03VQHRqOsuX5YUc-UUlnhiA64.jar' is inaccessible.
Aug 11, 2021 2:31:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-11T14:31:21.597Z: Staged package aws-java-sdk-dynamodb-1.11.974-RLvIDrX-TDIDN06gNnaHm9dQbuHCD_XSBesOuniSTk4.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/aws-java-sdk-dynamodb-1.11.974-RLvIDrX-TDIDN06gNnaHm9dQbuHCD_XSBesOuniSTk4.jar' is inaccessible.
Aug 11, 2021 2:31:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-11T14:31:21.641Z: Staged package aws-java-sdk-kinesis-1.11.974-kbU_CjbRpkrhhFeoyIMQA81nTYmniJY77w8Smd7PLpY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/aws-java-sdk-kinesis-1.11.974-kbU_CjbRpkrhhFeoyIMQA81nTYmniJY77w8Smd7PLpY.jar' is inaccessible.
Aug 11, 2021 2:31:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-11T14:31:21.692Z: Staged package aws-java-sdk-kms-1.11.974-HWH_mHclw3Wv4_GJ56x62xFDHSPSkZWjeKAlx40XrTk.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/aws-java-sdk-kms-1.11.974-HWH_mHclw3Wv4_GJ56x62xFDHSPSkZWjeKAlx40XrTk.jar' is inaccessible.
Aug 11, 2021 2:31:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-11T14:31:21.735Z: Staged package aws-java-sdk-s3-1.11.974-Qgd0wgaOuI_uyxxQI4Ulljk9bZEq-3JZArQF0zU5yzA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/aws-java-sdk-s3-1.11.974-Qgd0wgaOuI_uyxxQI4Ulljk9bZEq-3JZArQF0zU5yzA.jar' is inaccessible.
Aug 11, 2021 2:31:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-11T14:31:22.464Z: Staged package commons-io-2.6--HfTBGYKwqFC84ZbrfyXHex-1zx0fH-NXS9ROcpzZRM.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/commons-io-2.6--HfTBGYKwqFC84ZbrfyXHex-1zx0fH-NXS9ROcpzZRM.jar' is inaccessible.
Aug 11, 2021 2:31:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-11T14:31:22.505Z: Staged package commons-lang-2.6-UPEbCfh3wpTVbyRGP0fSj5Kc9QRPZIZhwPDPuumi9Jw.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/commons-lang-2.6-UPEbCfh3wpTVbyRGP0fSj5Kc9QRPZIZhwPDPuumi9Jw.jar' is inaccessible.
Aug 11, 2021 2:31:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-11T14:31:24.066Z: Staged package ion-java-1.0.2-DRJ7IFofzgq8KjdXoEF0hlG8ZsFc9MBZusWDOyfUcaU.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/ion-java-1.0.2-DRJ7IFofzgq8KjdXoEF0hlG8ZsFc9MBZusWDOyfUcaU.jar' is inaccessible.
Aug 11, 2021 2:31:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-11T14:31:24.306Z: Staged package jmespath-java-1.11.974-QaUt5Hnw6wd7ULTQ5Oj4j_zpRr-rnMCdfGwiwkSo8-0.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/jmespath-java-1.11.974-QaUt5Hnw6wd7ULTQ5Oj4j_zpRr-rnMCdfGwiwkSo8-0.jar' is inaccessible.
Aug 11, 2021 2:31:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-08-11T14:31:25.295Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Aug 11, 2021 3:29:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-11T15:29:35.297Z: Workers have started successfully.
Aug 11, 2021 3:29:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-11T15:29:39.326Z: Worker configuration: e2-standard-2 in us-central1-a.
Aug 11, 2021 3:30:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-11T15:30:09.110Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Aug 11, 2021 4:00:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-11T16:00:56.068Z: Cancel request is committed for workflow job: 2021-08-11_05_48_49-15281179841542251974.
Aug 11, 2021 4:00:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-11T16:00:56.126Z: Cleaning up.
Aug 11, 2021 4:00:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-11T16:00:56.205Z: Stopping **** pool...
Aug 11, 2021 4:00:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-11T16:00:56.259Z: Stopping **** pool...
Aug 11, 2021 4:03:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-11T16:03:19.418Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Aug 11, 2021 4:03:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-11T16:03:19.456Z: Worker pool stopped.
Aug 11, 2021 4:03:24 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-08-11_05_48_49-15281179841542251974 finished with status CANCELLED.
Aug 11, 2021 4:03:24 PM org.apache.beam.sdk.testutils.metrics.MetricsReader getCounterMetric
SEVERE: Failed to get metric totalBytes.count, from namespace co_gbk
Load test results for test (ID): 21a341e3-29e0-489f-9ef6-0a822cc9a18d and timestamp: 2021-08-11T12:48:42.337000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                       0.0
dataflow_v2_java11_total_bytes_count                      -1.0
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210811124333
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:e9cea82fea25261d6dc440cc7d21331299f2471683375e9b53f9b48aa576444e
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210811124333]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:e9cea82fea25261d6dc440cc7d21331299f2471683375e9b53f9b48aa576444e]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210811124333] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:e9cea82fea25261d6dc440cc7d21331299f2471683375e9b53f9b48aa576444e])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:e9cea82fea25261d6dc440cc7d21331299f2471683375e9b53f9b48aa576444e
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:e9cea82fea25261d6dc440cc7d21331299f2471683375e9b53f9b48aa576444e
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:e9cea82fea25261d6dc440cc7d21331299f2471683375e9b53f9b48aa576444e].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 20m 9s
107 actionable tasks: 78 executed, 27 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/csekgwbvm6nqm

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #54

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/54/display/redirect?page=changes>

Changes:

[heejong] [BEAM-12716] Update XLang Kafka taxi example to use BigQuery sink

[noreply] Merge pull request #15298 from [BEAM-12729] Suppress Avro Runtime

[heejong] add comment

[noreply] Refactor _RemoveDuplicates to use counter state (#15242)


------------------------------------------
[...truncated 65.60 KB...]
e10bab3db295: Preparing
d7d6071bf603: Preparing
b2a0baf9bb94: Preparing
2e7c8fd61b4f: Preparing
0b3c0ae326ec: Preparing
79eb09620b8c: Preparing
f42aed5f7feb: Preparing
89819bafde36: Preparing
f3d5b8f65132: Preparing
ad83f0aa5c0a: Preparing
5a9a65095453: Preparing
4b0edb23340c: Preparing
afa3e488a0ee: Preparing
e10bab3db295: Waiting
d7d6071bf603: Waiting
b2a0baf9bb94: Waiting
2e7c8fd61b4f: Waiting
5a9a65095453: Waiting
ad83f0aa5c0a: Waiting
0b3c0ae326ec: Waiting
4b0edb23340c: Waiting
afa3e488a0ee: Waiting
89819bafde36: Waiting
f3d5b8f65132: Waiting
1cd0e689378f: Waiting
79eb09620b8c: Waiting
f42aed5f7feb: Waiting
2d99c6116e44: Pushed
4b4e6ca539ce: Pushed
a8894453503f: Pushed
4b706062fddb: Pushed
fc4cdbaa0b1a: Pushed
1cd0e689378f: Pushed
d7d6071bf603: Pushed
b2a0baf9bb94: Pushed
f42aed5f7feb: Layer already exists
89819bafde36: Layer already exists
f3d5b8f65132: Layer already exists
ad83f0aa5c0a: Layer already exists
0b3c0ae326ec: Pushed
5a9a65095453: Layer already exists
4b0edb23340c: Layer already exists
afa3e488a0ee: Layer already exists
e10bab3db295: Pushed
79eb09620b8c: Pushed
2e7c8fd61b4f: Pushed
20210810124332: digest: sha256:b36746bfd8bbfb5fcdc34374940d8d2c162ab4cf72320ff19e12360017a71891 size: 4310

> Task :sdks:java:testing:load-tests:run
Aug 10, 2021 12:48:15 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Aug 10, 2021 12:48:15 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 191 files. Enable logging at DEBUG level to see which files will be staged.
Aug 10, 2021 12:48:16 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Aug 10, 2021 12:48:16 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Aug 10, 2021 12:48:18 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Aug 10, 2021 12:48:18 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <112637 bytes, hash 6906868c816dc6922b0b53fdac39bb1d3246b1eeb51b61b5578bda52b97deb3a> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-aQaGjIFtxpIrC1P9rDm7HTJGse61G2G1V4vaUrl96zo.pb
Aug 10, 2021 12:48:19 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 191 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Aug 10, 2021 12:48:20 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 191 files cached, 0 files newly uploaded in 0 seconds
Aug 10, 2021 12:48:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Aug 10, 2021 12:48:20 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3d96fa9e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3b545206, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77bb48d5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@181d8899, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12d5c30e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@b887730, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26586b74, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@52f57666, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6e041285, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@267dc982, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@439b15f2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3aa41da1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@74fab04a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c83468e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@272ce069, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3d7fb838, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1c26273d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3a37a501, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7a583586, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3a26ec8d]
Aug 10, 2021 12:48:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Aug 10, 2021 12:48:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Aug 10, 2021 12:48:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Aug 10, 2021 12:48:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Aug 10, 2021 12:48:20 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43f9dd56, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1d12e953, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@57cb70be, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2d4608a6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@20d87335, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2a8a4e0c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26c89563, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3bd6ba24, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58f437b0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@20f6f88c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4277127c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4c7e978c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@354e7004, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@22ebccb9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5875de6a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@608fe01f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2687725a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5a75ec37, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2c05ff9d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@33956d1a]
Aug 10, 2021 12:48:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Aug 10, 2021 12:48:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Aug 10, 2021 12:48:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Aug 10, 2021 12:48:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Aug 10, 2021 12:48:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Aug 10, 2021 12:48:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Aug 10, 2021 12:48:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Aug 10, 2021 12:48:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Aug 10, 2021 12:48:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Aug 10, 2021 12:48:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Aug 10, 2021 12:48:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Aug 10, 2021 12:48:20 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.33.0-SNAPSHOT
Aug 10, 2021 12:48:21 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-08-10_05_48_20-18203970189854605837?project=apache-beam-testing
Aug 10, 2021 12:48:21 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-08-10_05_48_20-18203970189854605837
Aug 10, 2021 12:48:21 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-08-10_05_48_20-18203970189854605837
Aug 10, 2021 12:48:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-08-10T12:48:28.401Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-08-cbod. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Aug 10, 2021 12:48:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-10T12:48:33.097Z: Worker configuration: e2-standard-2 in us-central1-a.
Aug 10, 2021 12:48:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-10T12:48:33.761Z: Expanding SplittableParDo operations into optimizable parts.
Aug 10, 2021 12:48:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-10T12:48:33.804Z: Expanding CollectionToSingleton operations into optimizable parts.
Aug 10, 2021 12:48:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-10T12:48:33.873Z: Expanding CoGroupByKey operations into optimizable parts.
Aug 10, 2021 12:48:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-10T12:48:33.943Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Aug 10, 2021 12:48:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-10T12:48:33.971Z: Expanding GroupByKey operations into streaming Read/Write steps
Aug 10, 2021 12:48:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-10T12:48:34.028Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Aug 10, 2021 12:48:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-10T12:48:34.130Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Aug 10, 2021 12:48:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-10T12:48:34.164Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Aug 10, 2021 12:48:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-10T12:48:34.196Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Aug 10, 2021 12:48:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-10T12:48:34.230Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Aug 10, 2021 12:48:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-10T12:48:34.269Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Aug 10, 2021 12:48:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-10T12:48:34.305Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Aug 10, 2021 12:48:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-10T12:48:34.339Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Aug 10, 2021 12:48:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-10T12:48:34.366Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Aug 10, 2021 12:48:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-10T12:48:34.398Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Aug 10, 2021 12:48:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-10T12:48:34.431Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Aug 10, 2021 12:48:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-10T12:48:34.457Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Aug 10, 2021 12:48:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-10T12:48:34.480Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Aug 10, 2021 12:48:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-10T12:48:34.503Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Aug 10, 2021 12:48:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-10T12:48:34.533Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Aug 10, 2021 12:48:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-10T12:48:34.571Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Aug 10, 2021 12:48:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-10T12:48:34.596Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Aug 10, 2021 12:48:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-10T12:48:34.622Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Aug 10, 2021 12:48:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-10T12:48:34.658Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Aug 10, 2021 12:48:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-10T12:48:34.695Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Aug 10, 2021 12:48:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-10T12:48:34.730Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Aug 10, 2021 12:48:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-10T12:48:34.766Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Aug 10, 2021 12:48:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-10T12:48:34.826Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Aug 10, 2021 12:48:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-10T12:48:34.858Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Aug 10, 2021 12:48:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-10T12:48:35.199Z: Starting 5 ****s in us-central1-a...
Aug 10, 2021 12:49:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-10T12:49:07.803Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Aug 10, 2021 12:49:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-10T12:49:10.547Z: Autoscaling: Raised the number of ****s to 3 so that the pipeline can catch up with its backlog and keep up with its input rate.
Aug 10, 2021 12:49:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-10T12:49:10.582Z: Resized **** pool to 3, though goal was 5.  This could be a quota issue.
Aug 10, 2021 12:49:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-10T12:49:20.867Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Aug 10, 2021 12:50:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-10T12:50:16.901Z: Workers have started successfully.
Aug 10, 2021 12:50:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-10T12:50:16.942Z: Workers have started successfully.
Aug 10, 2021 4:01:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-10T16:01:03.329Z: Cancel request is committed for workflow job: 2021-08-10_05_48_20-18203970189854605837.
Aug 10, 2021 4:01:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-10T16:01:03.491Z: Cleaning up.
Aug 10, 2021 4:01:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-10T16:01:03.604Z: Stopping **** pool...
Aug 10, 2021 4:01:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-10T16:01:03.666Z: Stopping **** pool...
Aug 10, 2021 4:03:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-10T16:03:22.033Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Aug 10, 2021 4:03:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-10T16:03:22.080Z: Worker pool stopped.
Aug 10, 2021 4:03:28 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-08-10_05_48_20-18203970189854605837 finished with status CANCELLED.
Aug 10, 2021 4:03:28 PM org.apache.beam.sdk.testutils.metrics.MetricsReader getCounterMetric
SEVERE: Failed to get metric totalBytes.count, from namespace co_gbk
Load test results for test (ID): 4607c91f-ed4c-4980-8009-d8ff90b43c0d and timestamp: 2021-08-10T12:48:16.035000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                       0.0
dataflow_v2_java11_total_bytes_count                      -1.0
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210810124332
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:b36746bfd8bbfb5fcdc34374940d8d2c162ab4cf72320ff19e12360017a71891
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210810124332]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:b36746bfd8bbfb5fcdc34374940d8d2c162ab4cf72320ff19e12360017a71891]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210810124332] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:b36746bfd8bbfb5fcdc34374940d8d2c162ab4cf72320ff19e12360017a71891])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:b36746bfd8bbfb5fcdc34374940d8d2c162ab4cf72320ff19e12360017a71891
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:b36746bfd8bbfb5fcdc34374940d8d2c162ab4cf72320ff19e12360017a71891
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:b36746bfd8bbfb5fcdc34374940d8d2c162ab4cf72320ff19e12360017a71891].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 20m 14s
107 actionable tasks: 76 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/l7rowiwu7ywac

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #53

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/53/display/redirect>

Changes:


------------------------------------------
[...truncated 66.75 KB...]
594ad198bb63: Pushed
18f2229c1b3e: Pushed
42de1cda4c9d: Pushed
10e021a4d3da: Pushed
5e5332ba9d1d: Pushed
f42aed5f7feb: Layer already exists
6f6642ffaf5e: Pushed
89819bafde36: Layer already exists
f3d5b8f65132: Layer already exists
ad83f0aa5c0a: Layer already exists
5a9a65095453: Layer already exists
4b0edb23340c: Layer already exists
afa3e488a0ee: Layer already exists
a5ead0e2f4ce: Pushed
25fda4fd0da0: Pushed
3e7adf8e1a13: Pushed
20210809124334: digest: sha256:b05a4a878398de7f2465b1442bdd270ad368d818f2e5bd0908d7f49fd5ef6ffb size: 4310

> Task :sdks:java:testing:load-tests:run
Aug 09, 2021 12:48:28 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Aug 09, 2021 12:48:28 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 191 files. Enable logging at DEBUG level to see which files will be staged.
Aug 09, 2021 12:48:29 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Aug 09, 2021 12:48:29 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Aug 09, 2021 12:48:32 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Aug 09, 2021 12:48:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <112637 bytes, hash 02454796fd8778eccfba1e5874589535caa01030141bd4040e706e3f7bcee6e0> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-AkVHlv2HeOzPuh5YdFiVNcqgEDAUG9QEDnBuP3vO5uA.pb
Aug 09, 2021 12:48:33 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 191 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Aug 09, 2021 12:48:34 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 191 files cached, 0 files newly uploaded in 0 seconds
Aug 09, 2021 12:48:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Aug 09, 2021 12:48:34 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12d5c30e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@b887730, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26586b74, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@52f57666, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6e041285, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@267dc982, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@439b15f2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3aa41da1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@74fab04a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c83468e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@272ce069, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3d7fb838, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1c26273d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3a37a501, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7a583586, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3a26ec8d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@17143b3b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@100c8b75, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2bc378f7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@268cbb86]
Aug 09, 2021 12:48:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Aug 09, 2021 12:48:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Aug 09, 2021 12:48:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Aug 09, 2021 12:48:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Aug 09, 2021 12:48:34 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@20d87335, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2a8a4e0c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26c89563, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3bd6ba24, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58f437b0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@20f6f88c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4277127c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4c7e978c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@354e7004, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@22ebccb9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5875de6a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@608fe01f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2687725a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5a75ec37, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2c05ff9d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@33956d1a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2e1ddc90, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5b7c8930, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58867cd5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@40e32762]
Aug 09, 2021 12:48:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Aug 09, 2021 12:48:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Aug 09, 2021 12:48:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Aug 09, 2021 12:48:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Aug 09, 2021 12:48:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Aug 09, 2021 12:48:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Aug 09, 2021 12:48:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Aug 09, 2021 12:48:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Aug 09, 2021 12:48:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Aug 09, 2021 12:48:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Aug 09, 2021 12:48:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Aug 09, 2021 12:48:34 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.33.0-SNAPSHOT
Aug 09, 2021 12:48:35 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-08-09_05_48_34-12515097779174733083?project=apache-beam-testing
Aug 09, 2021 12:48:35 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-08-09_05_48_34-12515097779174733083
Aug 09, 2021 12:48:35 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-08-09_05_48_34-12515097779174733083
Aug 09, 2021 12:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-08-09T12:48:44.043Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-08-1f6d. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Aug 09, 2021 12:48:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-09T12:48:47.703Z: Worker configuration: e2-standard-2 in us-central1-a.
Aug 09, 2021 12:48:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-09T12:48:48.417Z: Expanding SplittableParDo operations into optimizable parts.
Aug 09, 2021 12:48:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-09T12:48:48.454Z: Expanding CollectionToSingleton operations into optimizable parts.
Aug 09, 2021 12:48:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-09T12:48:48.541Z: Expanding CoGroupByKey operations into optimizable parts.
Aug 09, 2021 12:48:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-09T12:48:48.612Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Aug 09, 2021 12:48:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-09T12:48:48.654Z: Expanding GroupByKey operations into streaming Read/Write steps
Aug 09, 2021 12:48:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-09T12:48:48.716Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Aug 09, 2021 12:48:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-09T12:48:48.832Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Aug 09, 2021 12:48:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-09T12:48:48.865Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Aug 09, 2021 12:48:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-09T12:48:48.889Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Aug 09, 2021 12:48:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-09T12:48:48.923Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Aug 09, 2021 12:48:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-09T12:48:48.960Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Aug 09, 2021 12:48:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-09T12:48:48.994Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Aug 09, 2021 12:48:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-09T12:48:49.026Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Aug 09, 2021 12:48:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-09T12:48:49.057Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Aug 09, 2021 12:48:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-09T12:48:49.113Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Aug 09, 2021 12:48:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-09T12:48:49.160Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Aug 09, 2021 12:48:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-09T12:48:49.195Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Aug 09, 2021 12:48:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-09T12:48:49.229Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Aug 09, 2021 12:48:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-09T12:48:49.264Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Aug 09, 2021 12:48:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-09T12:48:49.297Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Aug 09, 2021 12:48:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-09T12:48:49.335Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Aug 09, 2021 12:48:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-09T12:48:49.367Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Aug 09, 2021 12:48:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-09T12:48:49.391Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Aug 09, 2021 12:48:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-09T12:48:49.416Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Aug 09, 2021 12:48:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-09T12:48:49.454Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Aug 09, 2021 12:48:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-09T12:48:49.487Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Aug 09, 2021 12:48:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-09T12:48:49.521Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Aug 09, 2021 12:48:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-09T12:48:49.555Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Aug 09, 2021 12:48:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-09T12:48:49.588Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Aug 09, 2021 12:48:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-09T12:48:49.969Z: Starting 5 ****s in us-central1-a...
Aug 09, 2021 12:49:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-09T12:49:18.758Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Aug 09, 2021 12:49:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-09T12:49:29.468Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Aug 09, 2021 12:50:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-09T12:50:23.141Z: Workers have started successfully.
Aug 09, 2021 12:50:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-09T12:50:23.177Z: Workers have started successfully.
Aug 09, 2021 3:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-09T15:45:50.104Z: Staged package amazon-kinesis-producer-0.14.1-5YYT_oyDFV17VWs8Of43O7N08sRfHu0SuJeqVSFCuLg.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/amazon-kinesis-producer-0.14.1-5YYT_oyDFV17VWs8Of43O7N08sRfHu0SuJeqVSFCuLg.jar' is inaccessible.
Aug 09, 2021 3:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-09T15:45:51.832Z: Staged package google-api-client-1.31.1-zAzs7d51UaRiAIkdNgZIN8hbV3QoubkFa6XKkxG5ZXQ.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-api-client-1.31.1-zAzs7d51UaRiAIkdNgZIN8hbV3QoubkFa6XKkxG5ZXQ.jar' is inaccessible.
Aug 09, 2021 3:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-09T15:45:51.864Z: Staged package google-api-client-jackson2-1.31.0-4P1Zf2ZwEY361HJ9G7BAkJFYtfoxvxwky7pQt8Nlgi8.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-api-client-jackson2-1.31.0-4P1Zf2ZwEY361HJ9G7BAkJFYtfoxvxwky7pQt8Nlgi8.jar' is inaccessible.
Aug 09, 2021 3:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-09T15:45:52.051Z: Staged package google-api-services-storage-v1-rev20210127-1.31.0-qwG3Ff_jgjMd1lU6mQjHSXDOU7A27GbAbi5U6zK8i7k.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-api-services-storage-v1-rev20210127-1.31.0-qwG3Ff_jgjMd1lU6mQjHSXDOU7A27GbAbi5U6zK8i7k.jar' is inaccessible.
Aug 09, 2021 3:45:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-08-09T15:45:54.561Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Aug 09, 2021 3:48:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-08-09T15:48:55.242Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Aug 09, 2021 3:51:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-09T15:51:50.086Z: Staged package amazon-kinesis-producer-0.14.1-5YYT_oyDFV17VWs8Of43O7N08sRfHu0SuJeqVSFCuLg.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/amazon-kinesis-producer-0.14.1-5YYT_oyDFV17VWs8Of43O7N08sRfHu0SuJeqVSFCuLg.jar' is inaccessible.
Aug 09, 2021 3:51:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-09T15:51:51.526Z: Staged package google-api-client-1.31.1-zAzs7d51UaRiAIkdNgZIN8hbV3QoubkFa6XKkxG5ZXQ.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-api-client-1.31.1-zAzs7d51UaRiAIkdNgZIN8hbV3QoubkFa6XKkxG5ZXQ.jar' is inaccessible.
Aug 09, 2021 3:51:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-09T15:51:51.564Z: Staged package google-api-client-jackson2-1.31.0-4P1Zf2ZwEY361HJ9G7BAkJFYtfoxvxwky7pQt8Nlgi8.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-api-client-jackson2-1.31.0-4P1Zf2ZwEY361HJ9G7BAkJFYtfoxvxwky7pQt8Nlgi8.jar' is inaccessible.
Aug 09, 2021 3:51:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-09T15:51:51.793Z: Staged package google-api-services-storage-v1-rev20210127-1.31.0-qwG3Ff_jgjMd1lU6mQjHSXDOU7A27GbAbi5U6zK8i7k.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-api-services-storage-v1-rev20210127-1.31.0-qwG3Ff_jgjMd1lU6mQjHSXDOU7A27GbAbi5U6zK8i7k.jar' is inaccessible.
Aug 09, 2021 3:51:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-08-09T15:51:54.958Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Aug 09, 2021 3:54:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-08-09T15:54:54.437Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Aug 09, 2021 3:57:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-09T15:57:50.090Z: Staged package amazon-kinesis-producer-0.14.1-5YYT_oyDFV17VWs8Of43O7N08sRfHu0SuJeqVSFCuLg.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/amazon-kinesis-producer-0.14.1-5YYT_oyDFV17VWs8Of43O7N08sRfHu0SuJeqVSFCuLg.jar' is inaccessible.
Aug 09, 2021 3:57:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-09T15:57:51.805Z: Staged package google-api-client-1.31.1-zAzs7d51UaRiAIkdNgZIN8hbV3QoubkFa6XKkxG5ZXQ.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-api-client-1.31.1-zAzs7d51UaRiAIkdNgZIN8hbV3QoubkFa6XKkxG5ZXQ.jar' is inaccessible.
Aug 09, 2021 3:57:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-09T15:57:51.836Z: Staged package google-api-client-jackson2-1.31.0-4P1Zf2ZwEY361HJ9G7BAkJFYtfoxvxwky7pQt8Nlgi8.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-api-client-jackson2-1.31.0-4P1Zf2ZwEY361HJ9G7BAkJFYtfoxvxwky7pQt8Nlgi8.jar' is inaccessible.
Aug 09, 2021 3:57:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-09T15:57:52.046Z: Staged package google-api-services-storage-v1-rev20210127-1.31.0-qwG3Ff_jgjMd1lU6mQjHSXDOU7A27GbAbi5U6zK8i7k.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-api-services-storage-v1-rev20210127-1.31.0-qwG3Ff_jgjMd1lU6mQjHSXDOU7A27GbAbi5U6zK8i7k.jar' is inaccessible.
Aug 09, 2021 3:57:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-08-09T15:57:55.104Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Aug 09, 2021 4:00:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-09T16:00:51.994Z: Cancel request is committed for workflow job: 2021-08-09_05_48_34-12515097779174733083.
Aug 09, 2021 4:00:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-09T16:00:52.025Z: Cleaning up.
Aug 09, 2021 4:00:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-09T16:00:52.087Z: Stopping **** pool...
Aug 09, 2021 4:00:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-09T16:00:52.146Z: Stopping **** pool...
Aug 09, 2021 4:03:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-09T16:03:10.745Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Aug 09, 2021 4:03:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-09T16:03:10.781Z: Worker pool stopped.
Aug 09, 2021 4:03:18 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-08-09_05_48_34-12515097779174733083 finished with status CANCELLED.
Aug 09, 2021 4:03:18 PM org.apache.beam.sdk.testutils.metrics.MetricsReader getCounterMetric
SEVERE: Failed to get metric totalBytes.count, from namespace co_gbk
Load test results for test (ID): e97756c4-15e5-4a9f-ae31-e540505fcb2f and timestamp: 2021-08-09T12:48:29.348000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                       0.0
dataflow_v2_java11_total_bytes_count                      -1.0
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210809124334
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:b05a4a878398de7f2465b1442bdd270ad368d818f2e5bd0908d7f49fd5ef6ffb
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210809124334]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:b05a4a878398de7f2465b1442bdd270ad368d818f2e5bd0908d7f49fd5ef6ffb]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210809124334] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:b05a4a878398de7f2465b1442bdd270ad368d818f2e5bd0908d7f49fd5ef6ffb])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:b05a4a878398de7f2465b1442bdd270ad368d818f2e5bd0908d7f49fd5ef6ffb
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:b05a4a878398de7f2465b1442bdd270ad368d818f2e5bd0908d7f49fd5ef6ffb
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:b05a4a878398de7f2465b1442bdd270ad368d818f2e5bd0908d7f49fd5ef6ffb].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 20m 1s
107 actionable tasks: 76 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/itiwcxjzx6nck

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #52

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/52/display/redirect>

Changes:


------------------------------------------
[...truncated 193.40 KB...]
SEVERE: 2021-08-08T15:54:09.280Z: Staged package proto-google-cloud-spanner-admin-database-v1-6.2.0-he117VVIvCnVX4p-eO1rG76etOGpA8L4fOe05Te65Go.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/proto-google-cloud-spanner-admin-database-v1-6.2.0-he117VVIvCnVX4p-eO1rG76etOGpA8L4fOe05Te65Go.jar' is inaccessible.
Aug 08, 2021 3:54:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-08T15:54:09.312Z: Staged package proto-google-cloud-spanner-admin-instance-v1-6.2.0-RRwYnG2iuMNY7JFZjHiAi508xBCHBI7EytAorrVOcg0.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/proto-google-cloud-spanner-admin-instance-v1-6.2.0-RRwYnG2iuMNY7JFZjHiAi508xBCHBI7EytAorrVOcg0.jar' is inaccessible.
Aug 08, 2021 3:54:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-08T15:54:09.355Z: Staged package proto-google-cloud-spanner-v1-6.2.0-Om3nWCqO9bQz9PEbtHTtFGRgNhKs1W0z0FQRF1md-7c.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/proto-google-cloud-spanner-v1-6.2.0-Om3nWCqO9bQz9PEbtHTtFGRgNhKs1W0z0FQRF1md-7c.jar' is inaccessible.
Aug 08, 2021 3:54:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-08T15:54:09.400Z: Staged package proto-google-common-protos-2.1.0-IEtnIIZzW2vwQX0hGOmeXeZDXWKbBJQAu9cXzWHPsFY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/proto-google-common-protos-2.1.0-IEtnIIZzW2vwQX0hGOmeXeZDXWKbBJQAu9cXzWHPsFY.jar' is inaccessible.
Aug 08, 2021 3:54:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-08T15:54:09.444Z: Staged package proto-google-iam-v1-1.0.12-3LMBbLCJKGCoBUkJYvX3mSwYkfgRW8MRFd89jtgSUWw.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/proto-google-iam-v1-1.0.12-3LMBbLCJKGCoBUkJYvX3mSwYkfgRW8MRFd89jtgSUWw.jar' is inaccessible.
Aug 08, 2021 3:54:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-08T15:54:09.477Z: Staged package protobuf-java-3.15.8-jQ4vmDT0_BoIOmUjmtxQfKg2gsdUon9MgOTyGZDv9oY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/protobuf-java-3.15.8-jQ4vmDT0_BoIOmUjmtxQfKg2gsdUon9MgOTyGZDv9oY.jar' is inaccessible.
Aug 08, 2021 3:54:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-08T15:54:09.518Z: Staged package protobuf-java-util-3.15.8-fw4vEAG-VPmjJB-dnuwng5h3x2SzCFLgZNuRKM8oGQE.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/protobuf-java-util-3.15.8-fw4vEAG-VPmjJB-dnuwng5h3x2SzCFLgZNuRKM8oGQE.jar' is inaccessible.
Aug 08, 2021 3:54:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-08T15:54:09.687Z: Staged package zstd-jni-1.4.5-2-lzQxwUtNCahuI7cYQRb8rC2FUB60p0MPfRhczhr0YFA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/zstd-jni-1.4.5-2-lzQxwUtNCahuI7cYQRb8rC2FUB60p0MPfRhczhr0YFA.jar' is inaccessible.
Aug 08, 2021 3:54:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-08-08T15:54:09.716Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Aug 08, 2021 3:57:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-08-08T15:57:08.622Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Aug 08, 2021 4:00:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-08T16:00:06.127Z: Staged package checker-qual-3.10.0-pNyILKaqxJbTM4Hl5esGBMRUg7AEvD6sk2jxuxfLJRI.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/checker-qual-3.10.0-pNyILKaqxJbTM4Hl5esGBMRUg7AEvD6sk2jxuxfLJRI.jar' is inaccessible.
Aug 08, 2021 4:00:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-08T16:00:06.402Z: Staged package gax-1.63.0-Cx6RlsmOjRvQWajIVLU1oC1w6hTF4cGHLxo4KPXiWiY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/gax-1.63.0-Cx6RlsmOjRvQWajIVLU1oC1w6hTF4cGHLxo4KPXiWiY.jar' is inaccessible.
Aug 08, 2021 4:00:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-08T16:00:06.484Z: Staged package gax-grpc-1.63.0-hpi4LFdyjgrjmY_cjkhXaM-bpwd533b0CI4CBy1Qr2Q.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/gax-grpc-1.63.0-hpi4LFdyjgrjmY_cjkhXaM-bpwd533b0CI4CBy1Qr2Q.jar' is inaccessible.
Aug 08, 2021 4:00:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-08T16:00:06.517Z: Staged package gax-httpjson-0.80.0-mI65R2zr4_eFb9xmUsMDJ91-yYKsZyYjNvOVhVVCdgI.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/gax-httpjson-0.80.0-mI65R2zr4_eFb9xmUsMDJ91-yYKsZyYjNvOVhVVCdgI.jar' is inaccessible.
Aug 08, 2021 4:00:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-08T16:00:06.670Z: Staged package google-auth-library-credentials-0.25.2-MDcwcOTwXH22YrOH5hY87DYgG6pmBEUSA49EcX-9ZyU.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-auth-library-credentials-0.25.2-MDcwcOTwXH22YrOH5hY87DYgG6pmBEUSA49EcX-9ZyU.jar' is inaccessible.
Aug 08, 2021 4:00:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-08T16:00:06.703Z: Staged package google-auth-library-oauth2-http-0.25.2-lfJRTZaX0Q33YYsVOPWmcqjVVBoB0kcxG1cJdCR5EqQ.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-auth-library-oauth2-http-0.25.2-lfJRTZaX0Q33YYsVOPWmcqjVVBoB0kcxG1cJdCR5EqQ.jar' is inaccessible.
Aug 08, 2021 4:00:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-08T16:00:06.740Z: Staged package google-cloud-bigquery-1.127.12-zHVMhL11Zvn0CjPbB9GVrMtazggV0DFCHTOHeRjbf2U.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-cloud-bigquery-1.127.12-zHVMhL11Zvn0CjPbB9GVrMtazggV0DFCHTOHeRjbf2U.jar' is inaccessible.
Aug 08, 2021 4:00:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-08T16:00:06.785Z: Staged package google-cloud-bigtable-1.22.0--YGsRYoPCMZYXYA7jIAaClU6JarpUaM7eKZM7XpWpzo.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-cloud-bigtable-1.22.0--YGsRYoPCMZYXYA7jIAaClU6JarpUaM7eKZM7XpWpzo.jar' is inaccessible.
Aug 08, 2021 4:00:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-08T16:00:06.826Z: Staged package google-cloud-core-1.94.6-WO6HrayK0srg4GyQb_T3LWWuQFGiDH4f_spgB6D754Y.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-cloud-core-1.94.6-WO6HrayK0srg4GyQb_T3LWWuQFGiDH4f_spgB6D754Y.jar' is inaccessible.
Aug 08, 2021 4:00:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-08T16:00:06.863Z: Staged package google-cloud-core-grpc-1.94.6-1cDxCddwQniURuQPX-LiAV1uxmpw-oEXNvzJ7n3-bCo.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-cloud-core-grpc-1.94.6-1cDxCddwQniURuQPX-LiAV1uxmpw-oEXNvzJ7n3-bCo.jar' is inaccessible.
Aug 08, 2021 4:00:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-08T16:00:06.903Z: Staged package google-cloud-core-http-1.94.6-BrjPhre9CcmKC1AU1bFPQAk4zLeSRVy3Tr5l25xGHQw.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-cloud-core-http-1.94.6-BrjPhre9CcmKC1AU1bFPQAk4zLeSRVy3Tr5l25xGHQw.jar' is inaccessible.
Aug 08, 2021 4:00:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-08T16:00:06.949Z: Staged package google-cloud-pubsub-1.112.0-MMTQF5nNlW-TaBBe1xNbmTQghyjc-eYqdGNbjop0dc0.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-cloud-pubsub-1.112.0-MMTQF5nNlW-TaBBe1xNbmTQghyjc-eYqdGNbjop0dc0.jar' is inaccessible.
Aug 08, 2021 4:00:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-08T16:00:06.993Z: Staged package google-cloud-spanner-6.2.0-Tl9XPX3koTNmg42Dz5ebZpG_xpeINV6OfbVmLOQuH2g.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-cloud-spanner-6.2.0-Tl9XPX3koTNmg42Dz5ebZpG_xpeINV6OfbVmLOQuH2g.jar' is inaccessible.
Aug 08, 2021 4:00:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-08T16:00:07.040Z: Staged package google-http-client-1.39.2-arSWeGCSfwUd1_K1deS3qO8182fcLjQIBzqk3uMo4ic.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-http-client-1.39.2-arSWeGCSfwUd1_K1deS3qO8182fcLjQIBzqk3uMo4ic.jar' is inaccessible.
Aug 08, 2021 4:00:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-08T16:00:07.076Z: Staged package google-http-client-apache-v2-1.39.2-UtOSCfZav29XfhyrRmrbmfEtHNK6TpjYblAfddf4UM8.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-http-client-apache-v2-1.39.2-UtOSCfZav29XfhyrRmrbmfEtHNK6TpjYblAfddf4UM8.jar' is inaccessible.
Aug 08, 2021 4:00:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-08T16:00:07.110Z: Staged package google-http-client-appengine-1.39.2-9jINwNsDWv9uqy4odIVg2nGKtGX7w0gbZT6j8nqlysk.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-http-client-appengine-1.39.2-9jINwNsDWv9uqy4odIVg2nGKtGX7w0gbZT6j8nqlysk.jar' is inaccessible.
Aug 08, 2021 4:00:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-08T16:00:07.144Z: Staged package google-http-client-gson-1.39.2-z4GUT9Qf4NqoabkTvQqq7w5Nj-Z3ShlVQPRU7NPPULY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-http-client-gson-1.39.2-z4GUT9Qf4NqoabkTvQqq7w5Nj-Z3ShlVQPRU7NPPULY.jar' is inaccessible.
Aug 08, 2021 4:00:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-08T16:00:07.173Z: Staged package google-http-client-jackson2-1.39.2-dKSTp7ol4d-z06Td9xMwbVo3htsAo0idWQYdyM_zIbE.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-http-client-jackson2-1.39.2-dKSTp7ol4d-z06Td9xMwbVo3htsAo0idWQYdyM_zIbE.jar' is inaccessible.
Aug 08, 2021 4:00:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-08T16:00:07.206Z: Staged package google-http-client-protobuf-1.39.2-48CRMH_IJ9gY8fZ3-5ANC0qF7rqbijaXD90xuQTMr5s.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-http-client-protobuf-1.39.2-48CRMH_IJ9gY8fZ3-5ANC0qF7rqbijaXD90xuQTMr5s.jar' is inaccessible.
Aug 08, 2021 4:00:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-08T16:00:07.251Z: Staged package grpc-alts-1.37.0-1pImhp1BwYeG32uYRxmeh0NZ9mjB6n37bBrYllOHP7s.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/grpc-alts-1.37.0-1pImhp1BwYeG32uYRxmeh0NZ9mjB6n37bBrYllOHP7s.jar' is inaccessible.
Aug 08, 2021 4:00:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-08T16:00:07.289Z: Staged package grpc-api-1.37.0-cwSURHkXII27hqdS5LctA5oWvU41MkWE_B68wNpz3v8.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/grpc-api-1.37.0-cwSURHkXII27hqdS5LctA5oWvU41MkWE_B68wNpz3v8.jar' is inaccessible.
Aug 08, 2021 4:00:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-08T16:00:07.336Z: Staged package grpc-auth-1.37.0-MnmZnsS7PY544IIITuiC1MDYLDnc6bGFYfHg2TYR_Ag.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/grpc-auth-1.37.0-MnmZnsS7PY544IIITuiC1MDYLDnc6bGFYfHg2TYR_Ag.jar' is inaccessible.
Aug 08, 2021 4:00:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-08T16:00:07.375Z: Staged package grpc-context-1.37.0-XOtiTb6T0PV5GY3NDBwbTA6AhtggIaU2y0YqhpBXjgM.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/grpc-context-1.37.0-XOtiTb6T0PV5GY3NDBwbTA6AhtggIaU2y0YqhpBXjgM.jar' is inaccessible.
Aug 08, 2021 4:00:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-08T16:00:07.414Z: Staged package grpc-core-1.37.0-FT_FqNdvB55GKsHljpOp3eJ99dajZRaFNyiXVb11t6w.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/grpc-core-1.37.0-FT_FqNdvB55GKsHljpOp3eJ99dajZRaFNyiXVb11t6w.jar' is inaccessible.
Aug 08, 2021 4:00:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-08T16:00:07.445Z: Staged package grpc-google-cloud-bigquerystorage-v1-1.17.0-02gx4b_vwwwBelFcFNwMP-XRHckCHFwMVTIlO8X4baA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/grpc-google-cloud-bigquerystorage-v1-1.17.0-02gx4b_vwwwBelFcFNwMP-XRHckCHFwMVTIlO8X4baA.jar' is inaccessible.
Aug 08, 2021 4:00:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-08T16:00:07.504Z: Staged package grpc-google-cloud-bigquerystorage-v1beta1-0.117.0-ZB91IVdshSTiwsYGytbKB1XsNxKB9l1mSHOUfw0yWFw.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/grpc-google-cloud-bigquerystorage-v1beta1-0.117.0-ZB91IVdshSTiwsYGytbKB1XsNxKB9l1mSHOUfw0yWFw.jar' is inaccessible.
Aug 08, 2021 4:00:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-08T16:00:07.542Z: Staged package grpc-google-cloud-bigquerystorage-v1beta2-0.117.0-yh1Ffb1FlGQv3nd38pYh4V_o6axvSbIkGqFIz4vdX3g.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/grpc-google-cloud-bigquerystorage-v1beta2-0.117.0-yh1Ffb1FlGQv3nd38pYh4V_o6axvSbIkGqFIz4vdX3g.jar' is inaccessible.
Aug 08, 2021 4:00:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-08T16:00:07.577Z: Staged package grpc-google-cloud-bigtable-admin-v2-1.22.0-Ym7z3BEqasHVclRl5AAwCln9ujk53f9LlVXndOQdYLA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/grpc-google-cloud-bigtable-admin-v2-1.22.0-Ym7z3BEqasHVclRl5AAwCln9ujk53f9LlVXndOQdYLA.jar' is inaccessible.
Aug 08, 2021 4:00:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-08T16:00:07.609Z: Staged package grpc-google-cloud-bigtable-v2-1.22.0-Y8qNsZoh7QdYSbNDZO11jjnU-kEFqMJokL-qHXXdYlE.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/grpc-google-cloud-bigtable-v2-1.22.0-Y8qNsZoh7QdYSbNDZO11jjnU-kEFqMJokL-qHXXdYlE.jar' is inaccessible.
Aug 08, 2021 4:00:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-08T16:00:07.645Z: Staged package grpc-google-cloud-pubsub-v1-1.94.0-YbRKfFIJ2b9i8zjI7rsveW9Nw7EbWMF8ftzN-NL0MJc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/grpc-google-cloud-pubsub-v1-1.94.0-YbRKfFIJ2b9i8zjI7rsveW9Nw7EbWMF8ftzN-NL0MJc.jar' is inaccessible.
Aug 08, 2021 4:00:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-08T16:00:07.690Z: Staged package grpc-google-cloud-spanner-admin-database-v1-6.2.0-T4fJ1eGg_mnIce0nZ_P78rbzGe9ZXBqmpX7djS2hqPM.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/grpc-google-cloud-spanner-admin-database-v1-6.2.0-T4fJ1eGg_mnIce0nZ_P78rbzGe9ZXBqmpX7djS2hqPM.jar' is inaccessible.
Aug 08, 2021 4:00:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-08T16:00:07.727Z: Staged package grpc-google-cloud-spanner-admin-instance-v1-6.2.0-76Yv9jtfyiuxo7EnaE0TWOD8M9G1_JvteF2QXmu1NAE.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/grpc-google-cloud-spanner-admin-instance-v1-6.2.0-76Yv9jtfyiuxo7EnaE0TWOD8M9G1_JvteF2QXmu1NAE.jar' is inaccessible.
Aug 08, 2021 4:00:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-08T16:00:07.766Z: Staged package grpc-google-cloud-spanner-v1-6.2.0-PktAWd9OinuI641yk2u1cXb-LtabSSB0EY_c25jgxM0.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/grpc-google-cloud-spanner-v1-6.2.0-PktAWd9OinuI641yk2u1cXb-LtabSSB0EY_c25jgxM0.jar' is inaccessible.
Aug 08, 2021 4:00:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-08T16:00:07.801Z: Staged package grpc-google-common-protos-2.1.0--MiDqADro9lpRF4jeKEnVMFg8FWvgpM0r5u0RnmbMJ8.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/grpc-google-common-protos-2.1.0--MiDqADro9lpRF4jeKEnVMFg8FWvgpM0r5u0RnmbMJ8.jar' is inaccessible.
Aug 08, 2021 4:00:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-08T16:00:07.839Z: Staged package grpc-grpclb-1.37.0-k0VTMXlza0kKHWVsOBkRHhN1q06moj658_SOtph53cg.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/grpc-grpclb-1.37.0-k0VTMXlza0kKHWVsOBkRHhN1q06moj658_SOtph53cg.jar' is inaccessible.
Aug 08, 2021 4:00:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-08T16:00:07.870Z: Staged package grpc-netty-1.37.0-iBdn1lqsvQpQlDPYkofabzaUWFsEOT8QMiUXB0P_1Tc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/grpc-netty-1.37.0-iBdn1lqsvQpQlDPYkofabzaUWFsEOT8QMiUXB0P_1Tc.jar' is inaccessible.
Aug 08, 2021 4:00:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-08T16:00:07.904Z: Staged package grpc-netty-shaded-1.37.0-R6JMgs-ax8p1eht0qdV9Hb5p6ayL1d9R93xz6mZ7bWg.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/grpc-netty-shaded-1.37.0-R6JMgs-ax8p1eht0qdV9Hb5p6ayL1d9R93xz6mZ7bWg.jar' is inaccessible.
Aug 08, 2021 4:00:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-08T16:00:07.946Z: Staged package grpc-protobuf-1.37.0-07c0poBIPfz_Fi63E6garGocKpEba5z-wjMikIcpIoc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/grpc-protobuf-1.37.0-07c0poBIPfz_Fi63E6garGocKpEba5z-wjMikIcpIoc.jar' is inaccessible.
Aug 08, 2021 4:00:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-08T16:00:08.020Z: Staged package grpc-protobuf-lite-1.37.0-jPcqS6mhCpRAS8cb8InNj4NFiafjYzRe6W-En2Uqt9M.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/grpc-protobuf-lite-1.37.0-jPcqS6mhCpRAS8cb8InNj4NFiafjYzRe6W-En2Uqt9M.jar' is inaccessible.
Aug 08, 2021 4:00:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-08T16:00:08.066Z: Staged package grpc-stub-1.37.0-S2e9fmu0Oryp12fLayrvYluHst0hbe2IeYM0Nw6n3Ew.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/grpc-stub-1.37.0-S2e9fmu0Oryp12fLayrvYluHst0hbe2IeYM0Nw6n3Ew.jar' is inaccessible.
Aug 08, 2021 4:00:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-08T16:00:08.116Z: Staged package guava-30.1-jre-5t0HL50_4CpGAGiDgL1CK9rBhMr2_iQYz90JNPCUMqo.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/guava-30.1-jre-5t0HL50_4CpGAGiDgL1CK9rBhMr2_iQYz90JNPCUMqo.jar' is inaccessible.
Aug 08, 2021 4:00:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-08T16:00:08.692Z: Staged package opencensus-contrib-grpc-util-0.28.0-uRaDRuavZZMwChvCfvdCVKofJAGYhZON2PuFK4d9VfA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/opencensus-contrib-grpc-util-0.28.0-uRaDRuavZZMwChvCfvdCVKofJAGYhZON2PuFK4d9VfA.jar' is inaccessible.
Aug 08, 2021 4:00:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-08T16:00:08.767Z: Staged package proto-google-cloud-bigquerystorage-v1-1.17.0-95Kd13lib16MutfGTvzMCJqbS65ZQNT9VRHgBwWtNQE.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/proto-google-cloud-bigquerystorage-v1-1.17.0-95Kd13lib16MutfGTvzMCJqbS65ZQNT9VRHgBwWtNQE.jar' is inaccessible.
Aug 08, 2021 4:00:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-08T16:00:08.807Z: Staged package proto-google-cloud-bigquerystorage-v1beta1-0.117.0-pK3SZjZI7_R19YB-f6BAjzEvPBP51B8g0W3RCLvzipc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/proto-google-cloud-bigquerystorage-v1beta1-0.117.0-pK3SZjZI7_R19YB-f6BAjzEvPBP51B8g0W3RCLvzipc.jar' is inaccessible.
Aug 08, 2021 4:00:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-08T16:00:08.854Z: Staged package proto-google-cloud-bigquerystorage-v1beta2-0.117.0-z1LoKma-K7JKLlprGXj2bIBy022pp9EfUhh25dTn0Ow.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/proto-google-cloud-bigquerystorage-v1beta2-0.117.0-z1LoKma-K7JKLlprGXj2bIBy022pp9EfUhh25dTn0Ow.jar' is inaccessible.
Aug 08, 2021 4:00:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-08T16:00:08.901Z: Staged package proto-google-cloud-bigtable-admin-v2-1.22.0-8bSazr-ySDSZJEkzlM5BFKebDGKoNY6i7jy3ITx48xg.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/proto-google-cloud-bigtable-admin-v2-1.22.0-8bSazr-ySDSZJEkzlM5BFKebDGKoNY6i7jy3ITx48xg.jar' is inaccessible.
Aug 08, 2021 4:00:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-08T16:00:08.944Z: Staged package proto-google-cloud-bigtable-v2-1.22.0-Ms5-Knu0K7tG_UgdRO_j6rJsVypzQswBWLLNwf5coic.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/proto-google-cloud-bigtable-v2-1.22.0-Ms5-Knu0K7tG_UgdRO_j6rJsVypzQswBWLLNwf5coic.jar' is inaccessible.
Aug 08, 2021 4:00:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-08T16:00:08.984Z: Staged package proto-google-cloud-datastore-v1-0.89.0-hkM_w8_zlqdrJoDOXo3TSJYS03GQI6BxuM9xOBX9rOw.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/proto-google-cloud-datastore-v1-0.89.0-hkM_w8_zlqdrJoDOXo3TSJYS03GQI6BxuM9xOBX9rOw.jar' is inaccessible.
Aug 08, 2021 4:00:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-08T16:00:09.082Z: Staged package proto-google-cloud-pubsub-v1-1.94.0-r1Ij-HGZWHgOZ9Qf0ZbMASV2fcefR39R02Xxw5hEEcI.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/proto-google-cloud-pubsub-v1-1.94.0-r1Ij-HGZWHgOZ9Qf0ZbMASV2fcefR39R02Xxw5hEEcI.jar' is inaccessible.
Aug 08, 2021 4:00:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-08T16:00:09.139Z: Staged package proto-google-cloud-spanner-admin-database-v1-6.2.0-he117VVIvCnVX4p-eO1rG76etOGpA8L4fOe05Te65Go.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/proto-google-cloud-spanner-admin-database-v1-6.2.0-he117VVIvCnVX4p-eO1rG76etOGpA8L4fOe05Te65Go.jar' is inaccessible.
Aug 08, 2021 4:00:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-08T16:00:09.185Z: Staged package proto-google-cloud-spanner-admin-instance-v1-6.2.0-RRwYnG2iuMNY7JFZjHiAi508xBCHBI7EytAorrVOcg0.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/proto-google-cloud-spanner-admin-instance-v1-6.2.0-RRwYnG2iuMNY7JFZjHiAi508xBCHBI7EytAorrVOcg0.jar' is inaccessible.
Aug 08, 2021 4:00:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-08T16:00:09.215Z: Staged package proto-google-cloud-spanner-v1-6.2.0-Om3nWCqO9bQz9PEbtHTtFGRgNhKs1W0z0FQRF1md-7c.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/proto-google-cloud-spanner-v1-6.2.0-Om3nWCqO9bQz9PEbtHTtFGRgNhKs1W0z0FQRF1md-7c.jar' is inaccessible.
Aug 08, 2021 4:00:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-08T16:00:09.260Z: Staged package proto-google-common-protos-2.1.0-IEtnIIZzW2vwQX0hGOmeXeZDXWKbBJQAu9cXzWHPsFY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/proto-google-common-protos-2.1.0-IEtnIIZzW2vwQX0hGOmeXeZDXWKbBJQAu9cXzWHPsFY.jar' is inaccessible.
Aug 08, 2021 4:00:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-08T16:00:09.310Z: Staged package proto-google-iam-v1-1.0.12-3LMBbLCJKGCoBUkJYvX3mSwYkfgRW8MRFd89jtgSUWw.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/proto-google-iam-v1-1.0.12-3LMBbLCJKGCoBUkJYvX3mSwYkfgRW8MRFd89jtgSUWw.jar' is inaccessible.
Aug 08, 2021 4:00:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-08T16:00:09.362Z: Staged package protobuf-java-3.15.8-jQ4vmDT0_BoIOmUjmtxQfKg2gsdUon9MgOTyGZDv9oY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/protobuf-java-3.15.8-jQ4vmDT0_BoIOmUjmtxQfKg2gsdUon9MgOTyGZDv9oY.jar' is inaccessible.
Aug 08, 2021 4:00:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-08T16:00:09.405Z: Staged package protobuf-java-util-3.15.8-fw4vEAG-VPmjJB-dnuwng5h3x2SzCFLgZNuRKM8oGQE.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/protobuf-java-util-3.15.8-fw4vEAG-VPmjJB-dnuwng5h3x2SzCFLgZNuRKM8oGQE.jar' is inaccessible.
Aug 08, 2021 4:00:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-08T16:00:09.570Z: Staged package zstd-jni-1.4.5-2-lzQxwUtNCahuI7cYQRb8rC2FUB60p0MPfRhczhr0YFA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/zstd-jni-1.4.5-2-lzQxwUtNCahuI7cYQRb8rC2FUB60p0MPfRhczhr0YFA.jar' is inaccessible.
Aug 08, 2021 4:00:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-08-08T16:00:09.596Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Aug 08, 2021 4:00:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-08T16:00:36.164Z: Cancel request is committed for workflow job: 2021-08-08_05_50_53-13406600479714313928.
Aug 08, 2021 4:00:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-08T16:00:36.189Z: Cleaning up.
Aug 08, 2021 4:00:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-08T16:00:36.253Z: Stopping **** pool...
Aug 08, 2021 4:00:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-08T16:00:36.304Z: Stopping **** pool...
Aug 08, 2021 4:02:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-08T16:02:57.785Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Aug 08, 2021 4:02:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-08T16:02:57.824Z: Worker pool stopped.
Aug 08, 2021 4:03:04 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-08-08_05_50_53-13406600479714313928 finished with status CANCELLED.
Aug 08, 2021 4:03:04 PM org.apache.beam.sdk.testutils.metrics.MetricsReader getCounterMetric
SEVERE: Failed to get metric totalBytes.count, from namespace co_gbk
Load test results for test (ID): 6cbee24f-34ec-4f9c-9e37-54d204a6556b and timestamp: 2021-08-08T12:50:47.553000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                       0.0
dataflow_v2_java11_total_bytes_count                      -1.0
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210808124341
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:a94d30b714c0828ec81b1e029573c8402755faf226062b39fdd5bcd53d0e4bcc
Deleted: sha256:270f22ba0074df34d4fee5d37103db6566cfa5100f5ebe6021effc1fe1ed9ac5
Deleted: sha256:f4a729d73f04d574bef32fe9ef6ff483f7ffe015c2fb578968ff42fc9c04dc71
Deleted: sha256:d1e787495bf440dea459f11d444474a621c0818c6c63e4ef1d8544a9048b64b9
Deleted: sha256:61742283ca524f96cb4670927c8236fc09a1af927c4689c109655083d961f1e9
Deleted: sha256:47ed60139635e8af8f92b678e6645f271ef9da92222ec974f95c24824f40a289
Deleted: sha256:35c23f4fa4d2ba0c4eee8154d3621e798f2802b40672aa9189a559723bf63b82
Deleted: sha256:6ab1dcc5520f53059449e73c1647d56ce19d011233881b853fc4dc368c003f99
Deleted: sha256:da146407dedb8acce7e4c537ad52a4255d7601dc42c6cef39b47a32ad0052b57
Deleted: sha256:05da4f5097ffcfbb8c1394e17740cf542d21b14408ef488593a209e6a79339d5
Deleted: sha256:bb190a81e5c43694d072e078b7e627ed80cb3beb0a5d753718a7ff0ef5853cf0
Deleted: sha256:b2bfa0714ebfb04fe007858967150d51de77ba7ee735bc19417cd7868c5fc8b0
Deleted: sha256:caa107e4f98e72ecdbbcddddfc5479974889721fa29754d58114b447ee58ea01
Deleted: sha256:870445c824d459d51e1a36f00fc9bac9189341e12adabd96de52189770f2d475
Deleted: sha256:b244256b9a1d27db28b9b128cf4755c3c4cced9f5a697da258ea202b94416398
Deleted: sha256:78bb74981ffabd972328a269a61b0b386b6b277481095e56a86d766f5bc95cfb
Deleted: sha256:d5ed09061b431e38e1a2d45612c14bb3ad5da2336c1a495f66b7a8fa5a667f5f
Deleted: sha256:e429b22e6aa7c61c0eb04c491670138fc868a040f9d336d8f7c572f98a93d621
Deleted: sha256:6fc9c0632fda29f64c3ae87fe28bfb6663082e9222a611d1b912283f89a42951
Deleted: sha256:a2a5ebc91089952841cfb2f6b21613f622670ed78d608ae2d04e32bb030d0aa6
Deleted: sha256:9c8b0af59ffa11efef8bb8e1b323f07e178f30621741c829dd23e538c51dbc68
Deleted: sha256:0abdbd70a50c0d5500ea99c67f6ea8dc0d3117c919e72bf630b07972bfc29a71
Deleted: sha256:585cf0e9f534eeb411628ba3b70cba6a56c117e23fa29b9e2c714f1df3a981b7
Deleted: sha256:cb84ef77992f9f607ad6f9408917e438bd8bbe13f3eb79d94c6969be58a487d1
Deleted: sha256:835832c37ac4fadf12b3a6f3f570289159da0950edf97052eeb635a498d3f34a
Deleted: sha256:f0602afbde97d35967e006a06a953c7ae654b16701efe3225838d6683a572982
Deleted: sha256:89362c7fc6f7d65e9c1b8b6f3f829c113a8b465771826c1209e6c09187ac7c93
Deleted: sha256:73e422cb1d5d54c6c430d1c84f25d06c32d892a41e758097df44a3baa4e85018
Deleted: sha256:b8f5874f5017a7b30fe1789164c083c1f992d8bf03fa90988eba31927fefc430
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210808124341]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:a94d30b714c0828ec81b1e029573c8402755faf226062b39fdd5bcd53d0e4bcc]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210808124341] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:a94d30b714c0828ec81b1e029573c8402755faf226062b39fdd5bcd53d0e4bcc])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:a94d30b714c0828ec81b1e029573c8402755faf226062b39fdd5bcd53d0e4bcc
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:a94d30b714c0828ec81b1e029573c8402755faf226062b39fdd5bcd53d0e4bcc
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:a94d30b714c0828ec81b1e029573c8402755faf226062b39fdd5bcd53d0e4bcc].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:8ec252428cf9a369c994cb1d7b3ed4d3346bb39f1e15bfea97303ce453c17724
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:8ec252428cf9a369c994cb1d7b3ed4d3346bb39f1e15bfea97303ce453c17724
ERROR: (gcloud.container.images.delete) Not found: response: {'status': '404', 'content-length': '168', 'x-xss-protection': '0', 'transfer-encoding': 'chunked', 'server': 'Docker Registry', '-content-encoding': 'gzip', 'docker-distribution-api-version': 'registry/2.0', 'cache-control': 'private', 'date': 'Sun, 08 Aug 2021 16:03:10 GMT', 'x-frame-options': 'SAMEORIGIN', 'content-type': 'application/json'}
Failed to compute blob liveness for manifest: 'sha256:8ec252428cf9a369c994cb1d7b3ed4d3346bb39f1e15bfea97303ce453c17724': None

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages FAILED

FAILURE: Build completed with 2 failures.

1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

2: Task failed with an exception.
-----------
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src/runners/google-cloud-dataflow-java/build.gradle'> line: 282

* What went wrong:
Execution failed for task ':runners:google-cloud-dataflow-java:cleanUpDockerImages'.
> Process 'command './scripts/cleanup_untagged_gcr_images.sh'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 45s
107 actionable tasks: 76 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/u5ks465zbbtaa

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #51

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/51/display/redirect?page=changes>

Changes:

[noreply] [BEAM-10212] Integrate caching client (#15214)

[noreply] Merge pull request #15249: [BEAM-12690] Fix GroupIntoBatches watermark

[noreply] Merge pull request #15246 from [BEAM-12685] Allow managed thread count


------------------------------------------
[...truncated 76.00 KB...]
Aug 07, 2021 12:48:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-08-07T12:48:24.039Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-08-87tg. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Aug 07, 2021 12:48:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-07T12:48:27.956Z: Worker configuration: e2-standard-2 in us-central1-a.
Aug 07, 2021 12:48:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-07T12:48:28.637Z: Expanding SplittableParDo operations into optimizable parts.
Aug 07, 2021 12:48:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-07T12:48:28.687Z: Expanding CollectionToSingleton operations into optimizable parts.
Aug 07, 2021 12:48:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-07T12:48:28.790Z: Expanding CoGroupByKey operations into optimizable parts.
Aug 07, 2021 12:48:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-07T12:48:28.877Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Aug 07, 2021 12:48:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-07T12:48:28.905Z: Expanding GroupByKey operations into streaming Read/Write steps
Aug 07, 2021 12:48:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-07T12:48:28.975Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Aug 07, 2021 12:48:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-07T12:48:29.180Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Aug 07, 2021 12:48:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-07T12:48:29.236Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Aug 07, 2021 12:48:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-07T12:48:29.270Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Aug 07, 2021 12:48:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-07T12:48:29.307Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Aug 07, 2021 12:48:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-07T12:48:29.343Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Aug 07, 2021 12:48:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-07T12:48:29.375Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Aug 07, 2021 12:48:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-07T12:48:29.412Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Aug 07, 2021 12:48:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-07T12:48:29.449Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Aug 07, 2021 12:48:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-07T12:48:29.485Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Aug 07, 2021 12:48:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-07T12:48:29.510Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Aug 07, 2021 12:48:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-07T12:48:29.544Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Aug 07, 2021 12:48:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-07T12:48:29.581Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Aug 07, 2021 12:48:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-07T12:48:29.604Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Aug 07, 2021 12:48:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-07T12:48:29.651Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Aug 07, 2021 12:48:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-07T12:48:29.682Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Aug 07, 2021 12:48:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-07T12:48:29.720Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Aug 07, 2021 12:48:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-07T12:48:29.742Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Aug 07, 2021 12:48:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-07T12:48:29.766Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Aug 07, 2021 12:48:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-07T12:48:29.798Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Aug 07, 2021 12:48:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-07T12:48:29.834Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Aug 07, 2021 12:48:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-07T12:48:29.867Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Aug 07, 2021 12:48:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-07T12:48:29.903Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Aug 07, 2021 12:48:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-07T12:48:29.939Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Aug 07, 2021 12:48:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-07T12:48:30.387Z: Starting 5 ****s in us-central1-a...
Aug 07, 2021 12:48:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-07T12:48:53.879Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Aug 07, 2021 12:49:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-07T12:49:14.104Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Aug 07, 2021 12:50:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-07T12:50:05.975Z: Workers have started successfully.
Aug 07, 2021 12:50:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-07T12:50:06.007Z: Workers have started successfully.
Aug 07, 2021 3:33:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-07T15:33:31.305Z: Staged package beam-sdks-java-io-kinesis-2.33.0-SNAPSHOT-l-wlPwuqL3JoShJc8MdSNmXKDFI5maGcuJPHhyiddpA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-kinesis-2.33.0-SNAPSHOT-l-wlPwuqL3JoShJc8MdSNmXKDFI5maGcuJPHhyiddpA.jar' is inaccessible.
Aug 07, 2021 3:33:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-07T15:33:31.400Z: Staged package beam-sdks-java-load-tests-2.33.0-SNAPSHOT-V9yHrJQiSt9kiJZ7EuXMHFTsASG3sfXHAWinjagDbFA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-load-tests-2.33.0-SNAPSHOT-V9yHrJQiSt9kiJZ7EuXMHFTsASG3sfXHAWinjagDbFA.jar' is inaccessible.
Aug 07, 2021 3:33:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-07T15:33:31.630Z: Staged package common-config-5.3.2-B_HQvj7POWJFHkM1OEwdiS2Y6zStZi7OXoiw-BJN7Nc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/common-config-5.3.2-B_HQvj7POWJFHkM1OEwdiS2Y6zStZi7OXoiw-BJN7Nc.jar' is inaccessible.
Aug 07, 2021 3:33:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-07T15:33:31.669Z: Staged package common-utils-5.3.2-4Rm-xdfQw0Yp9EH6ym0CZyPloNIGMAJ4b5DO_bByg9c.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/common-utils-5.3.2-4Rm-xdfQw0Yp9EH6ym0CZyPloNIGMAJ4b5DO_bByg9c.jar' is inaccessible.
Aug 07, 2021 3:33:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-07T15:33:33.484Z: Staged package kafka-avro-serializer-5.3.2-7FAoV2nngjZLbCXD3IPBOus_R_2KrM_zffAaNlVOls4.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/kafka-avro-serializer-5.3.2-7FAoV2nngjZLbCXD3IPBOus_R_2KrM_zffAaNlVOls4.jar' is inaccessible.
Aug 07, 2021 3:33:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-07T15:33:33.541Z: Staged package kafka-schema-registry-client-5.3.2-0mGQmXEF1E2hwRsR-O-mddgC_DwcNIU4Ba9SQcw2hKY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/kafka-schema-registry-client-5.3.2-0mGQmXEF1E2hwRsR-O-mddgC_DwcNIU4Ba9SQcw2hKY.jar' is inaccessible.
Aug 07, 2021 3:33:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-07T15:33:34.630Z: Staged package spring-core-4.3.18.RELEASE-qd-hKocv03STfOrJiihBIfzwVyBqE8q9UnPLPqutu2w.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/spring-core-4.3.18.RELEASE-qd-hKocv03STfOrJiihBIfzwVyBqE8q9UnPLPqutu2w.jar' is inaccessible.
Aug 07, 2021 3:33:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-07T15:33:34.674Z: Staged package spring-expression-4.3.18.RELEASE-aB3PS8pe5rtvJwR2BF3WrB6nO_rdEY2KpkFz7bel1_A.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/spring-expression-4.3.18.RELEASE-aB3PS8pe5rtvJwR2BF3WrB6nO_rdEY2KpkFz7bel1_A.jar' is inaccessible.
Aug 07, 2021 3:33:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-07T15:33:34.771Z: Staged package zkclient-0.10-JumIuLuoOMck_YNQszHui1_8WcOpwHTfEVxMOmyEOHg.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/zkclient-0.10-JumIuLuoOMck_YNQszHui1_8WcOpwHTfEVxMOmyEOHg.jar' is inaccessible.
Aug 07, 2021 3:33:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-08-07T15:33:34.819Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Aug 07, 2021 3:36:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-08-07T15:36:34.105Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Aug 07, 2021 3:39:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-07T15:39:31.228Z: Staged package beam-sdks-java-io-kinesis-2.33.0-SNAPSHOT-l-wlPwuqL3JoShJc8MdSNmXKDFI5maGcuJPHhyiddpA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-kinesis-2.33.0-SNAPSHOT-l-wlPwuqL3JoShJc8MdSNmXKDFI5maGcuJPHhyiddpA.jar' is inaccessible.
Aug 07, 2021 3:39:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-07T15:39:31.291Z: Staged package beam-sdks-java-load-tests-2.33.0-SNAPSHOT-V9yHrJQiSt9kiJZ7EuXMHFTsASG3sfXHAWinjagDbFA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-load-tests-2.33.0-SNAPSHOT-V9yHrJQiSt9kiJZ7EuXMHFTsASG3sfXHAWinjagDbFA.jar' is inaccessible.
Aug 07, 2021 3:39:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-07T15:39:31.513Z: Staged package common-config-5.3.2-B_HQvj7POWJFHkM1OEwdiS2Y6zStZi7OXoiw-BJN7Nc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/common-config-5.3.2-B_HQvj7POWJFHkM1OEwdiS2Y6zStZi7OXoiw-BJN7Nc.jar' is inaccessible.
Aug 07, 2021 3:39:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-07T15:39:31.603Z: Staged package common-utils-5.3.2-4Rm-xdfQw0Yp9EH6ym0CZyPloNIGMAJ4b5DO_bByg9c.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/common-utils-5.3.2-4Rm-xdfQw0Yp9EH6ym0CZyPloNIGMAJ4b5DO_bByg9c.jar' is inaccessible.
Aug 07, 2021 3:39:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-07T15:39:33.659Z: Staged package kafka-avro-serializer-5.3.2-7FAoV2nngjZLbCXD3IPBOus_R_2KrM_zffAaNlVOls4.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/kafka-avro-serializer-5.3.2-7FAoV2nngjZLbCXD3IPBOus_R_2KrM_zffAaNlVOls4.jar' is inaccessible.
Aug 07, 2021 3:39:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-07T15:39:33.756Z: Staged package kafka-schema-registry-client-5.3.2-0mGQmXEF1E2hwRsR-O-mddgC_DwcNIU4Ba9SQcw2hKY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/kafka-schema-registry-client-5.3.2-0mGQmXEF1E2hwRsR-O-mddgC_DwcNIU4Ba9SQcw2hKY.jar' is inaccessible.
Aug 07, 2021 3:39:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-07T15:39:34.699Z: Staged package spring-core-4.3.18.RELEASE-qd-hKocv03STfOrJiihBIfzwVyBqE8q9UnPLPqutu2w.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/spring-core-4.3.18.RELEASE-qd-hKocv03STfOrJiihBIfzwVyBqE8q9UnPLPqutu2w.jar' is inaccessible.
Aug 07, 2021 3:39:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-07T15:39:34.749Z: Staged package spring-expression-4.3.18.RELEASE-aB3PS8pe5rtvJwR2BF3WrB6nO_rdEY2KpkFz7bel1_A.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/spring-expression-4.3.18.RELEASE-aB3PS8pe5rtvJwR2BF3WrB6nO_rdEY2KpkFz7bel1_A.jar' is inaccessible.
Aug 07, 2021 3:39:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-07T15:39:34.848Z: Staged package zkclient-0.10-JumIuLuoOMck_YNQszHui1_8WcOpwHTfEVxMOmyEOHg.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/zkclient-0.10-JumIuLuoOMck_YNQszHui1_8WcOpwHTfEVxMOmyEOHg.jar' is inaccessible.
Aug 07, 2021 3:39:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-08-07T15:39:34.897Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Aug 07, 2021 3:42:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-08-07T15:42:34.200Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Aug 07, 2021 3:45:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-07T15:45:31.293Z: Staged package beam-sdks-java-io-kinesis-2.33.0-SNAPSHOT-l-wlPwuqL3JoShJc8MdSNmXKDFI5maGcuJPHhyiddpA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-kinesis-2.33.0-SNAPSHOT-l-wlPwuqL3JoShJc8MdSNmXKDFI5maGcuJPHhyiddpA.jar' is inaccessible.
Aug 07, 2021 3:45:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-07T15:45:31.370Z: Staged package beam-sdks-java-load-tests-2.33.0-SNAPSHOT-V9yHrJQiSt9kiJZ7EuXMHFTsASG3sfXHAWinjagDbFA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-load-tests-2.33.0-SNAPSHOT-V9yHrJQiSt9kiJZ7EuXMHFTsASG3sfXHAWinjagDbFA.jar' is inaccessible.
Aug 07, 2021 3:45:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-07T15:45:31.608Z: Staged package common-config-5.3.2-B_HQvj7POWJFHkM1OEwdiS2Y6zStZi7OXoiw-BJN7Nc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/common-config-5.3.2-B_HQvj7POWJFHkM1OEwdiS2Y6zStZi7OXoiw-BJN7Nc.jar' is inaccessible.
Aug 07, 2021 3:45:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-07T15:45:31.664Z: Staged package common-utils-5.3.2-4Rm-xdfQw0Yp9EH6ym0CZyPloNIGMAJ4b5DO_bByg9c.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/common-utils-5.3.2-4Rm-xdfQw0Yp9EH6ym0CZyPloNIGMAJ4b5DO_bByg9c.jar' is inaccessible.
Aug 07, 2021 3:45:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-07T15:45:33.925Z: Staged package kafka-avro-serializer-5.3.2-7FAoV2nngjZLbCXD3IPBOus_R_2KrM_zffAaNlVOls4.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/kafka-avro-serializer-5.3.2-7FAoV2nngjZLbCXD3IPBOus_R_2KrM_zffAaNlVOls4.jar' is inaccessible.
Aug 07, 2021 3:45:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-07T15:45:34.003Z: Staged package kafka-schema-registry-client-5.3.2-0mGQmXEF1E2hwRsR-O-mddgC_DwcNIU4Ba9SQcw2hKY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/kafka-schema-registry-client-5.3.2-0mGQmXEF1E2hwRsR-O-mddgC_DwcNIU4Ba9SQcw2hKY.jar' is inaccessible.
Aug 07, 2021 3:45:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-07T15:45:35.435Z: Staged package spring-core-4.3.18.RELEASE-qd-hKocv03STfOrJiihBIfzwVyBqE8q9UnPLPqutu2w.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/spring-core-4.3.18.RELEASE-qd-hKocv03STfOrJiihBIfzwVyBqE8q9UnPLPqutu2w.jar' is inaccessible.
Aug 07, 2021 3:45:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-07T15:45:35.479Z: Staged package spring-expression-4.3.18.RELEASE-aB3PS8pe5rtvJwR2BF3WrB6nO_rdEY2KpkFz7bel1_A.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/spring-expression-4.3.18.RELEASE-aB3PS8pe5rtvJwR2BF3WrB6nO_rdEY2KpkFz7bel1_A.jar' is inaccessible.
Aug 07, 2021 3:45:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-07T15:45:35.570Z: Staged package zkclient-0.10-JumIuLuoOMck_YNQszHui1_8WcOpwHTfEVxMOmyEOHg.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/zkclient-0.10-JumIuLuoOMck_YNQszHui1_8WcOpwHTfEVxMOmyEOHg.jar' is inaccessible.
Aug 07, 2021 3:45:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-08-07T15:45:35.601Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Aug 07, 2021 3:48:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-08-07T15:48:34.686Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Aug 07, 2021 3:51:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-07T15:51:31.290Z: Staged package beam-sdks-java-io-kinesis-2.33.0-SNAPSHOT-l-wlPwuqL3JoShJc8MdSNmXKDFI5maGcuJPHhyiddpA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-kinesis-2.33.0-SNAPSHOT-l-wlPwuqL3JoShJc8MdSNmXKDFI5maGcuJPHhyiddpA.jar' is inaccessible.
Aug 07, 2021 3:51:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-07T15:51:31.364Z: Staged package beam-sdks-java-load-tests-2.33.0-SNAPSHOT-V9yHrJQiSt9kiJZ7EuXMHFTsASG3sfXHAWinjagDbFA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-load-tests-2.33.0-SNAPSHOT-V9yHrJQiSt9kiJZ7EuXMHFTsASG3sfXHAWinjagDbFA.jar' is inaccessible.
Aug 07, 2021 3:51:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-07T15:51:31.593Z: Staged package common-config-5.3.2-B_HQvj7POWJFHkM1OEwdiS2Y6zStZi7OXoiw-BJN7Nc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/common-config-5.3.2-B_HQvj7POWJFHkM1OEwdiS2Y6zStZi7OXoiw-BJN7Nc.jar' is inaccessible.
Aug 07, 2021 3:51:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-07T15:51:31.633Z: Staged package common-utils-5.3.2-4Rm-xdfQw0Yp9EH6ym0CZyPloNIGMAJ4b5DO_bByg9c.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/common-utils-5.3.2-4Rm-xdfQw0Yp9EH6ym0CZyPloNIGMAJ4b5DO_bByg9c.jar' is inaccessible.
Aug 07, 2021 3:51:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-07T15:51:33.811Z: Staged package kafka-avro-serializer-5.3.2-7FAoV2nngjZLbCXD3IPBOus_R_2KrM_zffAaNlVOls4.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/kafka-avro-serializer-5.3.2-7FAoV2nngjZLbCXD3IPBOus_R_2KrM_zffAaNlVOls4.jar' is inaccessible.
Aug 07, 2021 3:51:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-07T15:51:33.859Z: Staged package kafka-schema-registry-client-5.3.2-0mGQmXEF1E2hwRsR-O-mddgC_DwcNIU4Ba9SQcw2hKY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/kafka-schema-registry-client-5.3.2-0mGQmXEF1E2hwRsR-O-mddgC_DwcNIU4Ba9SQcw2hKY.jar' is inaccessible.
Aug 07, 2021 3:51:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-07T15:51:34.743Z: Staged package spring-core-4.3.18.RELEASE-qd-hKocv03STfOrJiihBIfzwVyBqE8q9UnPLPqutu2w.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/spring-core-4.3.18.RELEASE-qd-hKocv03STfOrJiihBIfzwVyBqE8q9UnPLPqutu2w.jar' is inaccessible.
Aug 07, 2021 3:51:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-07T15:51:34.778Z: Staged package spring-expression-4.3.18.RELEASE-aB3PS8pe5rtvJwR2BF3WrB6nO_rdEY2KpkFz7bel1_A.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/spring-expression-4.3.18.RELEASE-aB3PS8pe5rtvJwR2BF3WrB6nO_rdEY2KpkFz7bel1_A.jar' is inaccessible.
Aug 07, 2021 3:51:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-07T15:51:34.869Z: Staged package zkclient-0.10-JumIuLuoOMck_YNQszHui1_8WcOpwHTfEVxMOmyEOHg.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/zkclient-0.10-JumIuLuoOMck_YNQszHui1_8WcOpwHTfEVxMOmyEOHg.jar' is inaccessible.
Aug 07, 2021 3:51:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-08-07T15:51:34.909Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Aug 07, 2021 3:54:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-08-07T15:54:34.350Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Aug 07, 2021 3:57:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-07T15:57:31.382Z: Staged package beam-sdks-java-io-kinesis-2.33.0-SNAPSHOT-l-wlPwuqL3JoShJc8MdSNmXKDFI5maGcuJPHhyiddpA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-kinesis-2.33.0-SNAPSHOT-l-wlPwuqL3JoShJc8MdSNmXKDFI5maGcuJPHhyiddpA.jar' is inaccessible.
Aug 07, 2021 3:57:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-07T15:57:31.458Z: Staged package beam-sdks-java-load-tests-2.33.0-SNAPSHOT-V9yHrJQiSt9kiJZ7EuXMHFTsASG3sfXHAWinjagDbFA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-load-tests-2.33.0-SNAPSHOT-V9yHrJQiSt9kiJZ7EuXMHFTsASG3sfXHAWinjagDbFA.jar' is inaccessible.
Aug 07, 2021 3:57:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-07T15:57:31.679Z: Staged package common-config-5.3.2-B_HQvj7POWJFHkM1OEwdiS2Y6zStZi7OXoiw-BJN7Nc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/common-config-5.3.2-B_HQvj7POWJFHkM1OEwdiS2Y6zStZi7OXoiw-BJN7Nc.jar' is inaccessible.
Aug 07, 2021 3:57:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-07T15:57:31.728Z: Staged package common-utils-5.3.2-4Rm-xdfQw0Yp9EH6ym0CZyPloNIGMAJ4b5DO_bByg9c.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/common-utils-5.3.2-4Rm-xdfQw0Yp9EH6ym0CZyPloNIGMAJ4b5DO_bByg9c.jar' is inaccessible.
Aug 07, 2021 3:57:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-07T15:57:33.474Z: Staged package kafka-avro-serializer-5.3.2-7FAoV2nngjZLbCXD3IPBOus_R_2KrM_zffAaNlVOls4.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/kafka-avro-serializer-5.3.2-7FAoV2nngjZLbCXD3IPBOus_R_2KrM_zffAaNlVOls4.jar' is inaccessible.
Aug 07, 2021 3:57:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-07T15:57:33.533Z: Staged package kafka-schema-registry-client-5.3.2-0mGQmXEF1E2hwRsR-O-mddgC_DwcNIU4Ba9SQcw2hKY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/kafka-schema-registry-client-5.3.2-0mGQmXEF1E2hwRsR-O-mddgC_DwcNIU4Ba9SQcw2hKY.jar' is inaccessible.
Aug 07, 2021 3:57:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-07T15:57:34.215Z: Staged package spring-core-4.3.18.RELEASE-qd-hKocv03STfOrJiihBIfzwVyBqE8q9UnPLPqutu2w.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/spring-core-4.3.18.RELEASE-qd-hKocv03STfOrJiihBIfzwVyBqE8q9UnPLPqutu2w.jar' is inaccessible.
Aug 07, 2021 3:57:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-07T15:57:34.247Z: Staged package spring-expression-4.3.18.RELEASE-aB3PS8pe5rtvJwR2BF3WrB6nO_rdEY2KpkFz7bel1_A.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/spring-expression-4.3.18.RELEASE-aB3PS8pe5rtvJwR2BF3WrB6nO_rdEY2KpkFz7bel1_A.jar' is inaccessible.
Aug 07, 2021 3:57:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-07T15:57:34.331Z: Staged package zkclient-0.10-JumIuLuoOMck_YNQszHui1_8WcOpwHTfEVxMOmyEOHg.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/zkclient-0.10-JumIuLuoOMck_YNQszHui1_8WcOpwHTfEVxMOmyEOHg.jar' is inaccessible.
Aug 07, 2021 3:57:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-08-07T15:57:34.371Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Aug 07, 2021 4:00:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-08-07T16:00:34.778Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Aug 07, 2021 4:00:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-07T16:00:57.629Z: Cancel request is committed for workflow job: 2021-08-07_05_48_16-15439522556927687297.
Aug 07, 2021 4:00:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-07T16:00:57.659Z: Cleaning up.
Aug 07, 2021 4:00:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-07T16:00:57.739Z: Stopping **** pool...
Aug 07, 2021 4:00:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-07T16:00:57.793Z: Stopping **** pool...
Aug 07, 2021 4:03:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-07T16:03:08.870Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Aug 07, 2021 4:03:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-07T16:03:08.909Z: Worker pool stopped.
Aug 07, 2021 4:03:15 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-08-07_05_48_16-15439522556927687297 finished with status CANCELLED.
Aug 07, 2021 4:03:15 PM org.apache.beam.sdk.testutils.metrics.MetricsReader getCounterMetric
SEVERE: Failed to get metric totalBytes.count, from namespace co_gbk
Load test results for test (ID): 23cf37be-aca6-4606-9e11-e8016dc9fc68 and timestamp: 2021-08-07T12:48:11.517000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                       0.0
dataflow_v2_java11_total_bytes_count                      -1.0
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210807124330
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:445213c32843d1e3253c25ec3d7e7f4f8d1685f24f9648672397fea2855d9125
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210807124330]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:445213c32843d1e3253c25ec3d7e7f4f8d1685f24f9648672397fea2855d9125]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210807124330] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:445213c32843d1e3253c25ec3d7e7f4f8d1685f24f9648672397fea2855d9125])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:445213c32843d1e3253c25ec3d7e7f4f8d1685f24f9648672397fea2855d9125
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:445213c32843d1e3253c25ec3d7e7f4f8d1685f24f9648672397fea2855d9125
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:445213c32843d1e3253c25ec3d7e7f4f8d1685f24f9648672397fea2855d9125].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:a49d0c85df7301a47cb162365b0da4993316b6c863dafe73412a580f220458e5
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:a49d0c85df7301a47cb162365b0da4993316b6c863dafe73412a580f220458e5
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:a49d0c85df7301a47cb162365b0da4993316b6c863dafe73412a580f220458e5].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 20m 2s
107 actionable tasks: 79 executed, 26 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/zzhcdmqtwmbcu

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 - Build # 50 - Aborted!

Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 - Build # 50 - Aborted:

Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/50/ to view the results.

Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #49

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/49/display/redirect?page=changes>

Changes:

[Etienne Chauchot] [BEAM-12591] Put Spark Structured Streaming runner sources back to main

[Etienne Chauchot] [BEAM-12629] As spark DataSourceV2 is only available for spark 2,

[Etienne Chauchot] [BEAM-12627] Deal with spark Encoders braking change between spark 2 and

[Etienne Chauchot] [BEAM-12591] move SchemaHelpers to correct package

[Etienne Chauchot] [BEAM-8470] Disable wait for termination in a streaming pipeline because

[Etienne Chauchot] [BEAM-12630] Deal with breaking change in streaming pipelines start by

[Etienne Chauchot] [BEAM-12629] Make source tests spark version agnostic and move them back

[Etienne Chauchot] [BEAM-12629] Make a spark 3 source impl

[ajamato] [BEAM-12670] Relocate bq client exception imports to try block and

[Etienne Chauchot] [BEAM-12591] Fix checkstyle and spotless

[Etienne Chauchot] [BEAM-12629] Reduce serializable to only needed classes and Fix schema

[Etienne Chauchot] [BEAM-12591] Add checkstyle exceptions for version specific classes

[Etienne Chauchot] [BEAM-12629] Fix sources javadocs and improve impl

[dpires] [BEAM-12715] Use shard number specified by user in SnowflakeIO batch

[Etienne Chauchot] [BEAM-12591] Add spark 3 to structured streaming validates runner tests

[relax] fix GroupIntoBatches

[noreply] [BEAM-12703] Fix universal metrics. (#15260)

[noreply] [BEAM-12702] Pull step unique names from pipeline for metrics. (#15261)

[noreply] [BEAM-12678] Add dependency of java jars when running go VR on portable

[noreply] [BEAM-12671] Mark known composite transforms native (#15236)

[noreply] [BEAM-6516] Fixes race condition in RabbitMqIO causing duplicate acks


------------------------------------------
[...truncated 66.00 KB...]
105cc467921f: Pushed
d7d869eafd69: Pushed
e613c09693fe: Pushed
f42aed5f7feb: Layer already exists
89819bafde36: Layer already exists
f3d5b8f65132: Layer already exists
f90211cbf29a: Pushed
ad83f0aa5c0a: Layer already exists
4b0edb23340c: Layer already exists
5a9a65095453: Layer already exists
afa3e488a0ee: Layer already exists
bafc2b387421: Pushed
52c82cddf467: Pushed
32249bc1f186: Pushed
20210805124354: digest: sha256:4b4f7c2c2a48604c17b7891a918ab489f87579c330b10965aa7eb7bc63de073c size: 4310

> Task :sdks:java:testing:load-tests:run
Aug 05, 2021 12:51:31 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Aug 05, 2021 12:51:32 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 191 files. Enable logging at DEBUG level to see which files will be staged.
Aug 05, 2021 12:51:33 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Aug 05, 2021 12:51:33 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Aug 05, 2021 12:51:36 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Aug 05, 2021 12:51:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <112699 bytes, hash 555293d1bbf87ed37bccdb58e5ce9e12073aaa40ce37a0ca450e91ceb040c25a> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-VVKT0bv4ftN7zNtY5c6eEgc6qkDON6DKRQ6RzrBAwlo.pb
Aug 05, 2021 12:51:38 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 191 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Aug 05, 2021 12:51:40 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 191 files cached, 0 files newly uploaded in 1 seconds
Aug 05, 2021 12:51:40 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Aug 05, 2021 12:51:40 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c3c4a71, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1352434e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4f9a6c2d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2b6fcb9f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@75de6341, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@74170687, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@68f0f72c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3d96fa9e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3b545206, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77bb48d5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@181d8899, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12d5c30e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@b887730, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26586b74, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@52f57666, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6e041285, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@267dc982, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@439b15f2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3aa41da1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@74fab04a]
Aug 05, 2021 12:51:40 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Aug 05, 2021 12:51:40 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Aug 05, 2021 12:51:40 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Aug 05, 2021 12:51:40 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Aug 05, 2021 12:51:40 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@44ed0a8f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@32177fa5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@a96d56c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6ab4a5b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2abe9173, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@235d29d6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1fdca564, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43f9dd56, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1d12e953, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@57cb70be, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2d4608a6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@20d87335, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2a8a4e0c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26c89563, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3bd6ba24, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58f437b0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@20f6f88c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4277127c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4c7e978c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@354e7004]
Aug 05, 2021 12:51:40 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Aug 05, 2021 12:51:40 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Aug 05, 2021 12:51:40 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Aug 05, 2021 12:51:40 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Aug 05, 2021 12:51:40 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Aug 05, 2021 12:51:40 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Aug 05, 2021 12:51:40 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Aug 05, 2021 12:51:40 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Aug 05, 2021 12:51:40 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Aug 05, 2021 12:51:40 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Aug 05, 2021 12:51:40 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Aug 05, 2021 12:51:40 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.33.0-SNAPSHOT
Aug 05, 2021 12:51:41 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-08-05_05_51_40-12932913610290942808?project=apache-beam-testing
Aug 05, 2021 12:51:41 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-08-05_05_51_40-12932913610290942808
Aug 05, 2021 12:51:41 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-08-05_05_51_40-12932913610290942808
Aug 05, 2021 12:51:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-08-05T12:51:48.483Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-08-iiph. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Aug 05, 2021 12:51:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-05T12:51:53.565Z: Worker configuration: e2-standard-2 in us-central1-a.
Aug 05, 2021 12:51:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-05T12:51:54.112Z: Expanding SplittableParDo operations into optimizable parts.
Aug 05, 2021 12:51:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-05T12:51:54.136Z: Expanding CollectionToSingleton operations into optimizable parts.
Aug 05, 2021 12:51:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-05T12:51:54.210Z: Expanding CoGroupByKey operations into optimizable parts.
Aug 05, 2021 12:51:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-05T12:51:54.298Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Aug 05, 2021 12:51:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-05T12:51:54.334Z: Expanding GroupByKey operations into streaming Read/Write steps
Aug 05, 2021 12:51:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-05T12:51:54.399Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Aug 05, 2021 12:51:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-05T12:51:54.543Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Aug 05, 2021 12:51:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-05T12:51:54.578Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Aug 05, 2021 12:51:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-05T12:51:54.602Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Aug 05, 2021 12:51:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-05T12:51:54.631Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Aug 05, 2021 12:51:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-05T12:51:54.700Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Aug 05, 2021 12:51:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-05T12:51:54.735Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Aug 05, 2021 12:51:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-05T12:51:54.767Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Aug 05, 2021 12:51:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-05T12:51:54.808Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Aug 05, 2021 12:51:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-05T12:51:54.832Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Aug 05, 2021 12:51:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-05T12:51:54.874Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Aug 05, 2021 12:51:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-05T12:51:54.909Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Aug 05, 2021 12:51:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-05T12:51:54.942Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Aug 05, 2021 12:51:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-05T12:51:54.970Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Aug 05, 2021 12:51:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-05T12:51:54.993Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Aug 05, 2021 12:51:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-05T12:51:55.022Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Aug 05, 2021 12:51:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-05T12:51:55.043Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Aug 05, 2021 12:51:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-05T12:51:55.076Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Aug 05, 2021 12:51:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-05T12:51:55.111Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Aug 05, 2021 12:51:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-05T12:51:55.145Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Aug 05, 2021 12:51:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-05T12:51:55.178Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Aug 05, 2021 12:51:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-05T12:51:55.206Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Aug 05, 2021 12:51:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-05T12:51:55.233Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Aug 05, 2021 12:51:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-05T12:51:55.265Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Aug 05, 2021 12:51:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-05T12:51:55.641Z: Starting 5 ****s in us-central1-a...
Aug 05, 2021 12:52:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-05T12:52:22.932Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Aug 05, 2021 12:52:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-05T12:52:25.368Z: Autoscaling: Raised the number of ****s to 2 so that the pipeline can catch up with its backlog and keep up with its input rate.
Aug 05, 2021 12:52:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-05T12:52:25.402Z: Resized **** pool to 2, though goal was 5.  This could be a quota issue.
Aug 05, 2021 12:52:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-05T12:52:35.681Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Aug 05, 2021 12:53:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-05T12:53:27.100Z: Workers have started successfully.
Aug 05, 2021 12:53:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-05T12:53:27.133Z: Workers have started successfully.
Aug 05, 2021 4:00:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-05T16:00:40.807Z: Cancel request is committed for workflow job: 2021-08-05_05_51_40-12932913610290942808.
Aug 05, 2021 4:00:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-05T16:00:40.905Z: Cleaning up.
Aug 05, 2021 4:00:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-05T16:00:40.987Z: Stopping **** pool...
Aug 05, 2021 4:00:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-05T16:00:41.033Z: Stopping **** pool...
Aug 05, 2021 4:03:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-05T16:03:13.474Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Aug 05, 2021 4:03:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-05T16:03:13.508Z: Worker pool stopped.
Aug 05, 2021 4:03:20 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-08-05_05_51_40-12932913610290942808 finished with status CANCELLED.
Aug 05, 2021 4:03:20 PM org.apache.beam.sdk.testutils.metrics.MetricsReader getCounterMetric
SEVERE: Failed to get metric totalBytes.count, from namespace co_gbk
Load test results for test (ID): d406f90f-42fe-4291-a9ec-1c64412dd987 and timestamp: 2021-08-05T12:51:32.497000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                       0.0
dataflow_v2_java11_total_bytes_count                      -1.0
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210805124354
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:4b4f7c2c2a48604c17b7891a918ab489f87579c330b10965aa7eb7bc63de073c
Deleted: sha256:ea0b7bf170f0af89f9bad837c551cd70ceac10305742c0242ff51a8a4876620b
Deleted: sha256:d1d6b7634994eb58e110930c0ecb35692daee7e4e864d7897a2a309632a62f3c
Deleted: sha256:14788254f7d10833d2e668f4b7b906c261fc7e534f9e1d4c9707fcd91185427f
Deleted: sha256:4c21031e25abbcfb7b6ccb841ba3f0843f359d28616b14f2e1676425af8d8bf8
Deleted: sha256:018f640d4288400e0827071abf200cb6d587298ca4a121a9e4512765eb71118b
Deleted: sha256:aadf4dfbec4a2a9fb38aaea7057d59f1a16682c98ce6af90088dd6a0aa6d1922
Deleted: sha256:51842ae6b83a47bf3100d232a9ba2763ca02089935b3cf250e8596072c8f501f
Deleted: sha256:975bf50e7fbedde807e37616eae062a68df9ab54c8d3d6147d90bef11925a8f4
Deleted: sha256:09caa155b057f85b509b879f14d0ea7c1e485ebdd9354eb74e57089b8a7566e0
Deleted: sha256:bb0d8b9e315694f241f79e54a33d9d83d2cf188f3aebe15d3200b9ca1c367c8e
Deleted: sha256:dcdde4da7bc7e5e49b4edd1a32a1990fff053629f47cc78b1d77eb1587344653
Deleted: sha256:c8ec9f92cee2ef63ffcb29202d6de50879f0c87d33a6c0ee6ca843ec1a3161da
Deleted: sha256:a1849371c370c4344f2e41fba6acc2e96d1d71f9bf24646d42fbcbdb74d55fbe
Deleted: sha256:3e2492d9515fffeefe02e7acc076bf56653bc7913704e2617fc5ca29a3c0cd14
Deleted: sha256:3b3fe95b54a404956ec23b3ef2d683a91b94ab67c7c579344b17f06ac0647bb0
Deleted: sha256:eb02cd9ead098b820cd746510d1a9596880af1210be555ee638c322644167122
Deleted: sha256:d2c6d056df3cb875caaedf879139d5fbbbabf46292568208158e82f55357825c
Deleted: sha256:7eb3f670c072a198399e5296bbc1805c71051d5f72603f596adf994b2b909cb3
Deleted: sha256:db77e4a3a3c9cfa47e1a0f168d977e11de2f97e3aa3a73794b11c8c1ec5e0c21
Deleted: sha256:7e48d05b522502f3085b4bed9be889f6291bc368f737c54c1c59357798ab6037
Deleted: sha256:21a2ac4698bc338e3b71cf690707a042ae045d844afc48a5095fffc9cb3114f0
Deleted: sha256:4a1d2eb3134109d2f4f820401cd8187b754d572b64ca6bf311689ceb3e43ae1e
Deleted: sha256:264f03ba9f489cbae86e3b76cf36f8920950634f124c2fd98933ce06773f8fb2
Deleted: sha256:b3e47cd8aec70ce65817793014bc76d8501f1e58704264e4e4862aee4205cdcd
Deleted: sha256:2c9cda29a87415606cd18851ef9e65171f93d958cde49a75f056b16bcdcaf3ac
Deleted: sha256:20742cee675c6890f2fa7bdd78b03839a70ba4596efb5650e2e501d9a3be17da
Deleted: sha256:a2664dd8a165f32220a04ea5156da94722bfd8b466de858caa7bd63e0467a0f3
Deleted: sha256:a173e282e735d93d3fab3276e1d0d5d0eafa62e21c8ca2f59d2118c6e956fc20
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210805124354]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:4b4f7c2c2a48604c17b7891a918ab489f87579c330b10965aa7eb7bc63de073c]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210805124354] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:4b4f7c2c2a48604c17b7891a918ab489f87579c330b10965aa7eb7bc63de073c])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:4b4f7c2c2a48604c17b7891a918ab489f87579c330b10965aa7eb7bc63de073c
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:4b4f7c2c2a48604c17b7891a918ab489f87579c330b10965aa7eb7bc63de073c
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:4b4f7c2c2a48604c17b7891a918ab489f87579c330b10965aa7eb7bc63de073c].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:fa3174bce374b6fc640d79bcebecc861796e8b3f403885041ff95aee6be4769c
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:fa3174bce374b6fc640d79bcebecc861796e8b3f403885041ff95aee6be4769c
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:fa3174bce374b6fc640d79bcebecc861796e8b3f403885041ff95aee6be4769c].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 59s
107 actionable tasks: 76 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/unaiwizv6756w

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #48

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/48/display/redirect?page=changes>

Changes:

[sayat.satybaldiyev] Fix issue with update schema source format

[noreply] [BEAM-12696] Update NewTaggedExternal to not require inputs (#15266)

[noreply] [BEAM-11088] Add TestStream package to Go SDK testing capabilities

[noreply] Merge pull request #15272:[BEAM-12712] Exclude runners that can't handle


------------------------------------------
[...truncated 75.94 KB...]
INFO: 2021-08-04T12:48:52.713Z: Worker configuration: e2-standard-2 in us-central1-a.
Aug 04, 2021 12:48:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-04T12:48:53.465Z: Expanding SplittableParDo operations into optimizable parts.
Aug 04, 2021 12:48:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-04T12:48:53.508Z: Expanding CollectionToSingleton operations into optimizable parts.
Aug 04, 2021 12:48:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-04T12:48:53.574Z: Expanding CoGroupByKey operations into optimizable parts.
Aug 04, 2021 12:48:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-04T12:48:53.650Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Aug 04, 2021 12:48:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-04T12:48:53.686Z: Expanding GroupByKey operations into streaming Read/Write steps
Aug 04, 2021 12:48:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-04T12:48:53.752Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Aug 04, 2021 12:48:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-04T12:48:53.860Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Aug 04, 2021 12:48:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-04T12:48:53.890Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Aug 04, 2021 12:48:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-04T12:48:53.919Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Aug 04, 2021 12:48:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-04T12:48:53.962Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Aug 04, 2021 12:48:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-04T12:48:54.000Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Aug 04, 2021 12:48:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-04T12:48:54.035Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Aug 04, 2021 12:48:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-04T12:48:54.057Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Aug 04, 2021 12:48:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-04T12:48:54.090Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Aug 04, 2021 12:48:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-04T12:48:54.134Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Aug 04, 2021 12:48:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-04T12:48:54.169Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Aug 04, 2021 12:48:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-04T12:48:54.202Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Aug 04, 2021 12:48:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-04T12:48:54.223Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Aug 04, 2021 12:48:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-04T12:48:54.248Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Aug 04, 2021 12:48:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-04T12:48:54.279Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Aug 04, 2021 12:48:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-04T12:48:54.303Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Aug 04, 2021 12:48:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-04T12:48:54.325Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Aug 04, 2021 12:48:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-04T12:48:54.353Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Aug 04, 2021 12:48:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-04T12:48:54.388Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Aug 04, 2021 12:48:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-04T12:48:54.421Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Aug 04, 2021 12:48:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-04T12:48:54.454Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Aug 04, 2021 12:48:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-04T12:48:54.485Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Aug 04, 2021 12:48:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-04T12:48:54.519Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Aug 04, 2021 12:48:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-04T12:48:54.551Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Aug 04, 2021 12:48:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-04T12:48:54.885Z: Starting 5 ****s in us-central1-a...
Aug 04, 2021 12:49:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-04T12:49:09.085Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Aug 04, 2021 12:49:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-04T12:49:41.275Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Aug 04, 2021 12:50:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-04T12:50:35.734Z: Workers have started successfully.
Aug 04, 2021 12:50:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-04T12:50:35.771Z: Workers have started successfully.
Aug 04, 2021 3:51:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-04T15:51:55.077Z: Staged package annotations-4.1.1.4-unNOHoTAnWFa9qCdMwNLTwRC-Hct7BIO-zdthqVlrhU.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/annotations-4.1.1.4-unNOHoTAnWFa9qCdMwNLTwRC-Hct7BIO-zdthqVlrhU.jar' is inaccessible.
Aug 04, 2021 3:51:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-04T15:51:55.296Z: Staged package avro-1.8.2-91SggwzmelqfpnpU7BXRA-8V4chQ17Jvr3tkfu3cgtM.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/avro-1.8.2-91SggwzmelqfpnpU7BXRA-8V4chQ17Jvr3tkfu3cgtM.jar' is inaccessible.
Aug 04, 2021 3:51:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-04T15:51:55.915Z: Staged package beam-vendor-bytebuddy-1_11_0-0.1-SneHjzQgCpxxLkvyN8dugqO_sTzwXJ2995kqFu4l2i4.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-bytebuddy-1_11_0-0.1-SneHjzQgCpxxLkvyN8dugqO_sTzwXJ2995kqFu4l2i4.jar' is inaccessible.
Aug 04, 2021 3:51:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-04T15:51:56.016Z: Staged package beam-vendor-guava-26_0-jre-0.1-8quizfXTrmcvJyOXi3zWDFtP5EofgqjksY5sIbsBqws.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-guava-26_0-jre-0.1-8quizfXTrmcvJyOXi3zWDFtP5EofgqjksY5sIbsBqws.jar' is inaccessible.
Aug 04, 2021 3:51:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-04T15:51:56.073Z: Staged package checker-compat-qual-2.5.5-EdE0skXpysxHRRTS1mtbhhj4A5oUZc3FW7wLNOAAi3o.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/checker-compat-qual-2.5.5-EdE0skXpysxHRRTS1mtbhhj4A5oUZc3FW7wLNOAAi3o.jar' is inaccessible.
Aug 04, 2021 3:51:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-04T15:51:56.198Z: Staged package commons-codec-1.14-oSjk-T-r5Tgd7WTPKHMBngYDC3GOtDzurgsOXRetM-k.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/commons-codec-1.14-oSjk-T-r5Tgd7WTPKHMBngYDC3GOtDzurgsOXRetM-k.jar' is inaccessible.
Aug 04, 2021 3:51:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-04T15:51:56.241Z: Staged package commons-compress-1.20-CutiXJSMaX6nsgUVbhEjY7We1eJVEhLNTkYL23LHwG4.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/commons-compress-1.20-CutiXJSMaX6nsgUVbhEjY7We1eJVEhLNTkYL23LHwG4.jar' is inaccessible.
Aug 04, 2021 3:51:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-04T15:51:56.321Z: Staged package commons-lang3-3.9-3i4dzc8--ReozoWGYaBnJqmpRPKOM61_ngi-pE3DwjA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/commons-lang3-3.9-3i4dzc8--ReozoWGYaBnJqmpRPKOM61_ngi-pE3DwjA.jar' is inaccessible.
Aug 04, 2021 3:51:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-04T15:51:56.362Z: Staged package commons-logging-1.2-2t3qHqC-D1aXirMAa4rJKDSv7vvZt-TmMW_KV98PpjY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/commons-logging-1.2-2t3qHqC-D1aXirMAa4rJKDSv7vvZt-TmMW_KV98PpjY.jar' is inaccessible.
Aug 04, 2021 3:51:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-04T15:51:56.415Z: Staged package commons-math3-3.6.1-HlbXsFjSi2Wr0la4RY44hbZ0wdWI-kPNfRy7nH7yswg.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/commons-math3-3.6.1-HlbXsFjSi2Wr0la4RY44hbZ0wdWI-kPNfRy7nH7yswg.jar' is inaccessible.
Aug 04, 2021 3:51:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-04T15:51:56.469Z: Staged package datastore-v1-proto-client-1.6.3-8GhVpKiAAK6wztlpNJ48AH4jyyB1KYo-5qJLXiBC6cw.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/datastore-v1-proto-client-1.6.3-8GhVpKiAAK6wztlpNJ48AH4jyyB1KYo-5qJLXiBC6cw.jar' is inaccessible.
Aug 04, 2021 3:51:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-04T15:51:57.707Z: Staged package gson-2.8.6-yPtIOQVNKAswM_gA0fWpfeLwKOuLoutFitKH5Tbz8l8.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/gson-2.8.6-yPtIOQVNKAswM_gA0fWpfeLwKOuLoutFitKH5Tbz8l8.jar' is inaccessible.
Aug 04, 2021 3:51:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-04T15:51:57.814Z: Staged package hamcrest-2.1-upOy46ViMiukMvChtTrdzFXLGIJTMZoCDtd_gk5pIFA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/hamcrest-2.1-upOy46ViMiukMvChtTrdzFXLGIJTMZoCDtd_gk5pIFA.jar' is inaccessible.
Aug 04, 2021 3:51:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-04T15:51:57.859Z: Staged package hamcrest-core-2.1-4JEJ5UoonYhQa5v-yYfd0Zn0IXyUZBMmaDUbmk8Avuk.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/hamcrest-core-2.1-4JEJ5UoonYhQa5v-yYfd0Zn0IXyUZBMmaDUbmk8Avuk.jar' is inaccessible.
Aug 04, 2021 3:51:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-04T15:51:57.976Z: Staged package j2objc-annotations-1.3-Ia8wySJnvWEiwOC00gzMtmQaN-r5VsZUDsRx1YTmSns.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/j2objc-annotations-1.3-Ia8wySJnvWEiwOC00gzMtmQaN-r5VsZUDsRx1YTmSns.jar' is inaccessible.
Aug 04, 2021 3:51:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-04T15:51:58.062Z: Staged package jackson-core-asl-1.9.13-RAqctcqVshX5U9OiCmsaENofCbUpqd3qX4pJBd2rT1o.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/jackson-core-asl-1.9.13-RAqctcqVshX5U9OiCmsaENofCbUpqd3qX4pJBd2rT1o.jar' is inaccessible.
Aug 04, 2021 3:51:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-04T15:51:58.127Z: Staged package jackson-dataformat-cbor-2.12.1-52d5rqlCfKc9f0B-X6gIoEBVeOwFZlOoZcJuTG8B1Cg.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/jackson-dataformat-cbor-2.12.1-52d5rqlCfKc9f0B-X6gIoEBVeOwFZlOoZcJuTG8B1Cg.jar' is inaccessible.
Aug 04, 2021 3:51:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-04T15:51:58.170Z: Staged package jackson-mapper-asl-1.9.13-dOegenby7breKTEqWi68z6AZEovAIezjhW12GX6b4MI.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/jackson-mapper-asl-1.9.13-dOegenby7breKTEqWi68z6AZEovAIezjhW12GX6b4MI.jar' is inaccessible.
Aug 04, 2021 3:51:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-04T15:51:58.247Z: Staged package javax.annotation-api-1.3.2-4EulGVvNVV3JVlD3zGFNFR5LzVLSmhC4qiGX86uJq5s.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/javax.annotation-api-1.3.2-4EulGVvNVV3JVlD3zGFNFR5LzVLSmhC4qiGX86uJq5s.jar' is inaccessible.
Aug 04, 2021 3:51:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-04T15:51:58.331Z: Staged package json-20200518-D_MObagvFa-vDna9bplqerVWaFEWaoP5c8ZKmg-UbS0.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/json-20200518-D_MObagvFa-vDna9bplqerVWaFEWaoP5c8ZKmg-UbS0.jar' is inaccessible.
Aug 04, 2021 3:51:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-04T15:51:58.396Z: Staged package jsr305-3.0.2-dmrSoHg_JoeWLIrXTO7MOKKLn3Ki0IXuQ4t4E-ko0Mc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/jsr305-3.0.2-dmrSoHg_JoeWLIrXTO7MOKKLn3Ki0IXuQ4t4E-ko0Mc.jar' is inaccessible.
Aug 04, 2021 3:51:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-04T15:51:58.618Z: Staged package metrics-core-3.2.6-iRF0xSMEduGVCihhaWcgApeFYuvfXIXwGZ7K4b0DD58.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/metrics-core-3.2.6-iRF0xSMEduGVCihhaWcgApeFYuvfXIXwGZ7K4b0DD58.jar' is inaccessible.
Aug 04, 2021 3:51:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-08-04T15:51:59.534Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Aug 04, 2021 3:54:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-08-04T15:54:59.600Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Aug 04, 2021 3:57:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-04T15:57:55.095Z: Staged package annotations-4.1.1.4-unNOHoTAnWFa9qCdMwNLTwRC-Hct7BIO-zdthqVlrhU.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/annotations-4.1.1.4-unNOHoTAnWFa9qCdMwNLTwRC-Hct7BIO-zdthqVlrhU.jar' is inaccessible.
Aug 04, 2021 3:57:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-04T15:57:55.224Z: Staged package avro-1.8.2-91SggwzmelqfpnpU7BXRA-8V4chQ17Jvr3tkfu3cgtM.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/avro-1.8.2-91SggwzmelqfpnpU7BXRA-8V4chQ17Jvr3tkfu3cgtM.jar' is inaccessible.
Aug 04, 2021 3:57:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-04T15:57:55.695Z: Staged package beam-vendor-bytebuddy-1_11_0-0.1-SneHjzQgCpxxLkvyN8dugqO_sTzwXJ2995kqFu4l2i4.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-bytebuddy-1_11_0-0.1-SneHjzQgCpxxLkvyN8dugqO_sTzwXJ2995kqFu4l2i4.jar' is inaccessible.
Aug 04, 2021 3:57:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-04T15:57:55.785Z: Staged package beam-vendor-guava-26_0-jre-0.1-8quizfXTrmcvJyOXi3zWDFtP5EofgqjksY5sIbsBqws.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-guava-26_0-jre-0.1-8quizfXTrmcvJyOXi3zWDFtP5EofgqjksY5sIbsBqws.jar' is inaccessible.
Aug 04, 2021 3:57:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-04T15:57:55.856Z: Staged package checker-compat-qual-2.5.5-EdE0skXpysxHRRTS1mtbhhj4A5oUZc3FW7wLNOAAi3o.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/checker-compat-qual-2.5.5-EdE0skXpysxHRRTS1mtbhhj4A5oUZc3FW7wLNOAAi3o.jar' is inaccessible.
Aug 04, 2021 3:57:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-04T15:57:56.017Z: Staged package commons-codec-1.14-oSjk-T-r5Tgd7WTPKHMBngYDC3GOtDzurgsOXRetM-k.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/commons-codec-1.14-oSjk-T-r5Tgd7WTPKHMBngYDC3GOtDzurgsOXRetM-k.jar' is inaccessible.
Aug 04, 2021 3:57:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-04T15:57:56.058Z: Staged package commons-compress-1.20-CutiXJSMaX6nsgUVbhEjY7We1eJVEhLNTkYL23LHwG4.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/commons-compress-1.20-CutiXJSMaX6nsgUVbhEjY7We1eJVEhLNTkYL23LHwG4.jar' is inaccessible.
Aug 04, 2021 3:57:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-04T15:57:56.134Z: Staged package commons-lang3-3.9-3i4dzc8--ReozoWGYaBnJqmpRPKOM61_ngi-pE3DwjA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/commons-lang3-3.9-3i4dzc8--ReozoWGYaBnJqmpRPKOM61_ngi-pE3DwjA.jar' is inaccessible.
Aug 04, 2021 3:57:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-04T15:57:56.189Z: Staged package commons-logging-1.2-2t3qHqC-D1aXirMAa4rJKDSv7vvZt-TmMW_KV98PpjY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/commons-logging-1.2-2t3qHqC-D1aXirMAa4rJKDSv7vvZt-TmMW_KV98PpjY.jar' is inaccessible.
Aug 04, 2021 3:57:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-04T15:57:56.229Z: Staged package commons-math3-3.6.1-HlbXsFjSi2Wr0la4RY44hbZ0wdWI-kPNfRy7nH7yswg.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/commons-math3-3.6.1-HlbXsFjSi2Wr0la4RY44hbZ0wdWI-kPNfRy7nH7yswg.jar' is inaccessible.
Aug 04, 2021 3:57:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-04T15:57:56.304Z: Staged package datastore-v1-proto-client-1.6.3-8GhVpKiAAK6wztlpNJ48AH4jyyB1KYo-5qJLXiBC6cw.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/datastore-v1-proto-client-1.6.3-8GhVpKiAAK6wztlpNJ48AH4jyyB1KYo-5qJLXiBC6cw.jar' is inaccessible.
Aug 04, 2021 3:57:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-04T15:57:57.473Z: Staged package gson-2.8.6-yPtIOQVNKAswM_gA0fWpfeLwKOuLoutFitKH5Tbz8l8.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/gson-2.8.6-yPtIOQVNKAswM_gA0fWpfeLwKOuLoutFitKH5Tbz8l8.jar' is inaccessible.
Aug 04, 2021 3:57:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-04T15:57:57.532Z: Staged package hamcrest-2.1-upOy46ViMiukMvChtTrdzFXLGIJTMZoCDtd_gk5pIFA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/hamcrest-2.1-upOy46ViMiukMvChtTrdzFXLGIJTMZoCDtd_gk5pIFA.jar' is inaccessible.
Aug 04, 2021 3:57:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-04T15:57:57.575Z: Staged package hamcrest-core-2.1-4JEJ5UoonYhQa5v-yYfd0Zn0IXyUZBMmaDUbmk8Avuk.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/hamcrest-core-2.1-4JEJ5UoonYhQa5v-yYfd0Zn0IXyUZBMmaDUbmk8Avuk.jar' is inaccessible.
Aug 04, 2021 3:58:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-04T15:57:57.674Z: Staged package j2objc-annotations-1.3-Ia8wySJnvWEiwOC00gzMtmQaN-r5VsZUDsRx1YTmSns.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/j2objc-annotations-1.3-Ia8wySJnvWEiwOC00gzMtmQaN-r5VsZUDsRx1YTmSns.jar' is inaccessible.
Aug 04, 2021 3:58:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-04T15:57:57.762Z: Staged package jackson-core-asl-1.9.13-RAqctcqVshX5U9OiCmsaENofCbUpqd3qX4pJBd2rT1o.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/jackson-core-asl-1.9.13-RAqctcqVshX5U9OiCmsaENofCbUpqd3qX4pJBd2rT1o.jar' is inaccessible.
Aug 04, 2021 3:58:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-04T15:57:57.822Z: Staged package jackson-dataformat-cbor-2.12.1-52d5rqlCfKc9f0B-X6gIoEBVeOwFZlOoZcJuTG8B1Cg.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/jackson-dataformat-cbor-2.12.1-52d5rqlCfKc9f0B-X6gIoEBVeOwFZlOoZcJuTG8B1Cg.jar' is inaccessible.
Aug 04, 2021 3:58:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-04T15:57:57.869Z: Staged package jackson-mapper-asl-1.9.13-dOegenby7breKTEqWi68z6AZEovAIezjhW12GX6b4MI.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/jackson-mapper-asl-1.9.13-dOegenby7breKTEqWi68z6AZEovAIezjhW12GX6b4MI.jar' is inaccessible.
Aug 04, 2021 3:58:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-04T15:57:57.931Z: Staged package javax.annotation-api-1.3.2-4EulGVvNVV3JVlD3zGFNFR5LzVLSmhC4qiGX86uJq5s.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/javax.annotation-api-1.3.2-4EulGVvNVV3JVlD3zGFNFR5LzVLSmhC4qiGX86uJq5s.jar' is inaccessible.
Aug 04, 2021 3:58:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-04T15:57:57.990Z: Staged package json-20200518-D_MObagvFa-vDna9bplqerVWaFEWaoP5c8ZKmg-UbS0.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/json-20200518-D_MObagvFa-vDna9bplqerVWaFEWaoP5c8ZKmg-UbS0.jar' is inaccessible.
Aug 04, 2021 3:58:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-04T15:57:58.037Z: Staged package jsr305-3.0.2-dmrSoHg_JoeWLIrXTO7MOKKLn3Ki0IXuQ4t4E-ko0Mc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/jsr305-3.0.2-dmrSoHg_JoeWLIrXTO7MOKKLn3Ki0IXuQ4t4E-ko0Mc.jar' is inaccessible.
Aug 04, 2021 3:58:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-04T15:57:58.208Z: Staged package metrics-core-3.2.6-iRF0xSMEduGVCihhaWcgApeFYuvfXIXwGZ7K4b0DD58.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/metrics-core-3.2.6-iRF0xSMEduGVCihhaWcgApeFYuvfXIXwGZ7K4b0DD58.jar' is inaccessible.
Aug 04, 2021 3:58:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-08-04T15:57:58.905Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Aug 04, 2021 4:00:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-04T16:00:43.163Z: Cancel request is committed for workflow job: 2021-08-04_05_48_41-6226870436249924502.
Aug 04, 2021 4:00:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-04T16:00:43.197Z: Cleaning up.
Aug 04, 2021 4:00:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-04T16:00:43.269Z: Stopping **** pool...
Aug 04, 2021 4:00:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-04T16:00:43.334Z: Stopping **** pool...
Aug 04, 2021 4:03:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-04T16:03:06.085Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Aug 04, 2021 4:03:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-04T16:03:06.125Z: Worker pool stopped.
Aug 04, 2021 4:03:12 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-08-04_05_48_41-6226870436249924502 finished with status CANCELLED.
Aug 04, 2021 4:03:12 PM org.apache.beam.sdk.testutils.metrics.MetricsReader getCounterMetric
SEVERE: Failed to get metric totalBytes.count, from namespace co_gbk
Load test results for test (ID): a593a966-6b62-49cc-8463-af3e07227ac4 and timestamp: 2021-08-04T12:48:36.070000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                       0.0
dataflow_v2_java11_total_bytes_count                      -1.0
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210804124337
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:6837e58b79c15fec79efab1f19d11efd1bbe992a27ba6c7c3f986f7c39d13ba0
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210804124337]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:6837e58b79c15fec79efab1f19d11efd1bbe992a27ba6c7c3f986f7c39d13ba0]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210804124337] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:6837e58b79c15fec79efab1f19d11efd1bbe992a27ba6c7c3f986f7c39d13ba0])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:6837e58b79c15fec79efab1f19d11efd1bbe992a27ba6c7c3f986f7c39d13ba0
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:6837e58b79c15fec79efab1f19d11efd1bbe992a27ba6c7c3f986f7c39d13ba0
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:6837e58b79c15fec79efab1f19d11efd1bbe992a27ba6c7c3f986f7c39d13ba0].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:464e1cacb22be0ad5dba573257a87ea4d324bdf1798b544ab6b76f2108e15aff
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:464e1cacb22be0ad5dba573257a87ea4d324bdf1798b544ab6b76f2108e15aff
ERROR: (gcloud.container.images.delete) Not found: response: {'status': '404', 'content-length': '168', 'x-xss-protection': '0', 'transfer-encoding': 'chunked', 'server': 'Docker Registry', '-content-encoding': 'gzip', 'docker-distribution-api-version': 'registry/2.0', 'cache-control': 'private', 'date': 'Wed, 04 Aug 2021 16:03:17 GMT', 'x-frame-options': 'SAMEORIGIN', 'content-type': 'application/json'}
Failed to compute blob liveness for manifest: 'sha256:464e1cacb22be0ad5dba573257a87ea4d324bdf1798b544ab6b76f2108e15aff': None

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages FAILED

FAILURE: Build completed with 2 failures.

1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

2: Task failed with an exception.
-----------
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src/runners/google-cloud-dataflow-java/build.gradle'> line: 282

* What went wrong:
Execution failed for task ':runners:google-cloud-dataflow-java:cleanUpDockerImages'.
> Process 'command './scripts/cleanup_untagged_gcr_images.sh'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 57s
107 actionable tasks: 77 executed, 28 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/e35kqdljnhwlm

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #47

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/47/display/redirect?page=changes>

Changes:

[relax] Ensure timer consistency in Dataflow and portable runners


------------------------------------------
[...truncated 67.24 KB...]
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Aug 03, 2021 12:48:27 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Aug 03, 2021 12:48:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <112699 bytes, hash 5bf4778ae49aeabd7c6f3448b0130ce897d8ae625df6f10057ef59ec354a928f> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-W_R3iuSa6r18bzRIsBMM6JfYrmJd9vEAV-9Z7DVKko8.pb
Aug 03, 2021 12:48:29 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 191 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Aug 03, 2021 12:48:29 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 191 files cached, 0 files newly uploaded in 0 seconds
Aug 03, 2021 12:48:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Aug 03, 2021 12:48:29 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@75de6341, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@74170687, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@68f0f72c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3d96fa9e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3b545206, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77bb48d5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@181d8899, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12d5c30e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@b887730, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26586b74, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@52f57666, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6e041285, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@267dc982, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@439b15f2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3aa41da1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@74fab04a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c83468e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@272ce069, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3d7fb838, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1c26273d]
Aug 03, 2021 12:48:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Aug 03, 2021 12:48:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Aug 03, 2021 12:48:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Aug 03, 2021 12:48:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Aug 03, 2021 12:48:29 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6ab4a5b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2abe9173, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@235d29d6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1fdca564, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43f9dd56, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1d12e953, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@57cb70be, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2d4608a6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@20d87335, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2a8a4e0c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26c89563, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3bd6ba24, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58f437b0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@20f6f88c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4277127c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4c7e978c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@354e7004, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@22ebccb9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5875de6a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@608fe01f]
Aug 03, 2021 12:48:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Aug 03, 2021 12:48:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Aug 03, 2021 12:48:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Aug 03, 2021 12:48:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Aug 03, 2021 12:48:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Aug 03, 2021 12:48:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Aug 03, 2021 12:48:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Aug 03, 2021 12:48:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Aug 03, 2021 12:48:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Aug 03, 2021 12:48:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Aug 03, 2021 12:48:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Aug 03, 2021 12:48:29 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.33.0-SNAPSHOT
Aug 03, 2021 12:48:30 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-08-03_05_48_29-15466374677597675524?project=apache-beam-testing
Aug 03, 2021 12:48:30 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-08-03_05_48_29-15466374677597675524
Aug 03, 2021 12:48:30 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-08-03_05_48_29-15466374677597675524
Aug 03, 2021 12:48:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-08-03T12:48:38.239Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-08-30f9. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Aug 03, 2021 12:48:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-03T12:48:42.689Z: Worker configuration: e2-standard-2 in us-central1-a.
Aug 03, 2021 12:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-03T12:48:43.379Z: Expanding SplittableParDo operations into optimizable parts.
Aug 03, 2021 12:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-03T12:48:43.398Z: Expanding CollectionToSingleton operations into optimizable parts.
Aug 03, 2021 12:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-03T12:48:43.467Z: Expanding CoGroupByKey operations into optimizable parts.
Aug 03, 2021 12:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-03T12:48:43.536Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Aug 03, 2021 12:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-03T12:48:43.573Z: Expanding GroupByKey operations into streaming Read/Write steps
Aug 03, 2021 12:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-03T12:48:43.638Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Aug 03, 2021 12:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-03T12:48:43.746Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Aug 03, 2021 12:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-03T12:48:43.775Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Aug 03, 2021 12:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-03T12:48:43.808Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Aug 03, 2021 12:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-03T12:48:43.850Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Aug 03, 2021 12:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-03T12:48:43.886Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Aug 03, 2021 12:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-03T12:48:43.919Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Aug 03, 2021 12:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-03T12:48:43.951Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Aug 03, 2021 12:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-03T12:48:43.977Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Aug 03, 2021 12:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-03T12:48:44.012Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Aug 03, 2021 12:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-03T12:48:44.050Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Aug 03, 2021 12:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-03T12:48:44.087Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Aug 03, 2021 12:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-03T12:48:44.121Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Aug 03, 2021 12:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-03T12:48:44.155Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Aug 03, 2021 12:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-03T12:48:44.182Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Aug 03, 2021 12:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-03T12:48:44.208Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Aug 03, 2021 12:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-03T12:48:44.236Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Aug 03, 2021 12:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-03T12:48:44.285Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Aug 03, 2021 12:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-03T12:48:44.315Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Aug 03, 2021 12:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-03T12:48:44.350Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Aug 03, 2021 12:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-03T12:48:44.395Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Aug 03, 2021 12:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-03T12:48:44.432Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Aug 03, 2021 12:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-03T12:48:44.463Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Aug 03, 2021 12:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-03T12:48:44.499Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Aug 03, 2021 12:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-03T12:48:44.874Z: Starting 5 ****s in us-central1-a...
Aug 03, 2021 12:48:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-03T12:48:56.617Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Aug 03, 2021 12:49:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-03T12:49:28.554Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Aug 03, 2021 12:50:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-03T12:50:29.747Z: Workers have started successfully.
Aug 03, 2021 12:50:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-03T12:50:29.771Z: Workers have started successfully.
Aug 03, 2021 2:24:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-03T14:24:46.429Z: Staged package conscrypt-openjdk-uber-2.5.1-AfnHQstZKhUeLmK9U5eomAYoqWcAH82s1KpHRGeGhfM.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/conscrypt-openjdk-uber-2.5.1-AfnHQstZKhUeLmK9U5eomAYoqWcAH82s1KpHRGeGhfM.jar' is inaccessible.
Aug 03, 2021 2:24:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-03T14:24:48.895Z: Staged package netty-buffer-4.1.52.Final-QcoQNa7m4PgXdZfQp2EKWqLG5vrHRKyB_eTl5Y9jlnU.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-buffer-4.1.52.Final-QcoQNa7m4PgXdZfQp2EKWqLG5vrHRKyB_eTl5Y9jlnU.jar' is inaccessible.
Aug 03, 2021 2:24:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-03T14:24:48.960Z: Staged package netty-codec-4.1.52.Final-PH543M5-NT2GrH26LCUatc3dTrhuqoic_kIpeCuCjlI.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-codec-4.1.52.Final-PH543M5-NT2GrH26LCUatc3dTrhuqoic_kIpeCuCjlI.jar' is inaccessible.
Aug 03, 2021 2:24:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-03T14:24:48.993Z: Staged package netty-codec-http-4.1.52.Final-tdoo0OqHiwfTERrPXk-vQSiO8jArJkFBkP3Rp9lzyAw.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-codec-http-4.1.52.Final-tdoo0OqHiwfTERrPXk-vQSiO8jArJkFBkP3Rp9lzyAw.jar' is inaccessible.
Aug 03, 2021 2:24:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-03T14:24:49.057Z: Staged package netty-codec-http2-4.1.52.Final-61J6QtIonRLe5tR1PKJxWXije7tBxxkVzk6VOcSfk6E.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-codec-http2-4.1.52.Final-61J6QtIonRLe5tR1PKJxWXije7tBxxkVzk6VOcSfk6E.jar' is inaccessible.
Aug 03, 2021 2:24:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-03T14:24:49.101Z: Staged package netty-codec-socks-4.1.52.Final-_sxXfgu3ch1naM_BhG7hcpEdFquA372ZldllelUdTqc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-codec-socks-4.1.52.Final-_sxXfgu3ch1naM_BhG7hcpEdFquA372ZldllelUdTqc.jar' is inaccessible.
Aug 03, 2021 2:24:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-03T14:24:49.146Z: Staged package netty-common-4.1.52.Final-DR8QHk231TDtBNfKvVfZXAdQ8Fm7az-t_tDIDUc0xxc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-common-4.1.52.Final-DR8QHk231TDtBNfKvVfZXAdQ8Fm7az-t_tDIDUc0xxc.jar' is inaccessible.
Aug 03, 2021 2:24:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-03T14:24:49.188Z: Staged package netty-handler-4.1.52.Final-LNwh-_rgSUDIWSkDl3nJP1H2GPXvshYHkxnc_PMqVcE.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-handler-4.1.52.Final-LNwh-_rgSUDIWSkDl3nJP1H2GPXvshYHkxnc_PMqVcE.jar' is inaccessible.
Aug 03, 2021 2:24:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-03T14:24:49.226Z: Staged package netty-handler-proxy-4.1.52.Final-6NB4Y860vUgG4lWh1TLIZQ8POy1NAGrW0hgS6UG0L9w.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-handler-proxy-4.1.52.Final-6NB4Y860vUgG4lWh1TLIZQ8POy1NAGrW0hgS6UG0L9w.jar' is inaccessible.
Aug 03, 2021 2:24:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-03T14:24:49.282Z: Staged package netty-resolver-4.1.52.Final-butyojOXnLRbC01tRsWkEzLFfAz4qAOx78YNu-y6CaY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-resolver-4.1.52.Final-butyojOXnLRbC01tRsWkEzLFfAz4qAOx78YNu-y6CaY.jar' is inaccessible.
Aug 03, 2021 2:24:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-03T14:24:49.354Z: Staged package netty-transport-4.1.52.Final-mj5vjA5V3jY-seoQ_ngXl-yjlOYhht8q4LTrK84LRUE.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-transport-4.1.52.Final-mj5vjA5V3jY-seoQ_ngXl-yjlOYhht8q4LTrK84LRUE.jar' is inaccessible.
Aug 03, 2021 2:24:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-03T14:24:49.416Z: Staged package opencensus-api-0.28.0-DBcj8_bTBhMjhFzouIs1_dpQCBLgp1uOtfzErYyHGpU.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/opencensus-api-0.28.0-DBcj8_bTBhMjhFzouIs1_dpQCBLgp1uOtfzErYyHGpU.jar' is inaccessible.
Aug 03, 2021 2:24:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-03T14:24:49.509Z: Staged package paranamer-2.8-aIyxGKYCHYGROOhVIIyVYDFoi-S0eiS7YVvsxjrO3wc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/paranamer-2.8-aIyxGKYCHYGROOhVIIyVYDFoi-S0eiS7YVvsxjrO3wc.jar' is inaccessible.
Aug 03, 2021 2:24:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-03T14:24:49.553Z: Staged package perfmark-api-0.23.0-xwW1wQwY_zAyuegXQrwvaw5WB_am38DIrQz_ddSRMEI.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/perfmark-api-0.23.0-xwW1wQwY_zAyuegXQrwvaw5WB_am38DIrQz_ddSRMEI.jar' is inaccessible.
Aug 03, 2021 2:24:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-08-03T14:24:49.964Z: Staged package snappy-java-1.1.8.4-JMTR_B6J4HgzGrj0AamcrWhZm95KLkUWBCy1SMUbHD4.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/snappy-java-1.1.8.4-JMTR_B6J4HgzGrj0AamcrWhZm95KLkUWBCy1SMUbHD4.jar' is inaccessible.
Aug 03, 2021 2:24:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-08-03T14:24:50.113Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Aug 03, 2021 4:00:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-03T16:00:39.148Z: Cancel request is committed for workflow job: 2021-08-03_05_48_29-15466374677597675524.
Aug 03, 2021 4:00:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-03T16:00:39.192Z: Cleaning up.
Aug 03, 2021 4:00:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-03T16:00:39.274Z: Stopping **** pool...
Aug 03, 2021 4:00:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-03T16:00:39.329Z: Stopping **** pool...
Aug 03, 2021 4:02:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-03T16:02:51.136Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Aug 03, 2021 4:02:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-03T16:02:51.165Z: Worker pool stopped.
Aug 03, 2021 4:02:57 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-08-03_05_48_29-15466374677597675524 finished with status CANCELLED.
Aug 03, 2021 4:02:57 PM org.apache.beam.sdk.testutils.metrics.MetricsReader getCounterMetric
SEVERE: Failed to get metric totalBytes.count, from namespace co_gbk
Load test results for test (ID): cdc8b56a-074d-4acc-aac6-05c82c5b7db2 and timestamp: 2021-08-03T12:48:24.892000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                       0.0
dataflow_v2_java11_total_bytes_count                      -1.0
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210803124334
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:62e77f0cbc44bb39fa72bfa484a09f1dac21032330e5e1952cded7950a91540c
Deleted: sha256:7ffa1617cc5756adca6c4e2a47b03da071e0eb277c7af3a88a83bff52d168a8c
Deleted: sha256:7de950114b527f38342b46e86256fb326d7e256892a5c82a25bd86a6295d75d9
Deleted: sha256:f22e3ae1e5a2ea6705a095f5b70272764ed5c78dd66728545fdbeb8130a1f4aa
Deleted: sha256:3d11e9cc7091c7111c4fd71ba641afa25361efd0f15fb37274fd6edf05213532
Deleted: sha256:017f298ee13d3d1b35c87dd5021e4079a52c4ac0225d66dda21cb0732a6245ea
Deleted: sha256:2c710f291e86fc92e9f5ac23def05533bab3f0c74e3ee189c5531739e37b69af
Deleted: sha256:eef629c6850e3774563785be4b0ae55691f7b6d2537f1f3d858e719fb2ee275d
Deleted: sha256:5c2a82013053a5b77495d09784ff9bced1b911046b4bb189c743d613aeb0229b
Deleted: sha256:f2a252dd325853c3527ce7fb37189a8090b3da5c8707d9b918ae61dad739e4f5
Deleted: sha256:cd47bb2764f6144ae02f6531641aff528d3fa150e61ea89da3e29760ef1daedd
Deleted: sha256:a459ef2786bb82f8812e73bd30aa5129a9766c74492f882507ed49bb02a5ce6d
Deleted: sha256:afbc2ca52062a1383376d328048aa56ce023d9393cca8178f844641c7e4e630b
Deleted: sha256:508709c06bdab7ddde681dbff9b64d95330e2cb61cb570722dd68ea045ae68b6
Deleted: sha256:235d7fcc299cf7b4a61b7bfe72485efada642bd787f698ce4782b8329ce7bef3
Deleted: sha256:9e541c110f768342858c37144b4b634b9b36015c9ab78ba63b62ca6e10e1548b
Deleted: sha256:bafff0f6f7c490cdaa3356396f0ebbb496701af271d24547e225845b4dd8fd0a
Deleted: sha256:32c190aeeb7870ce156cc4915c65ca1612feccebc1d2bac071403c681fbc196e
Deleted: sha256:4618eaaf49cddfee532a8ef759ea61ed0598e76de44781741290d8b02c481ee2
Deleted: sha256:4da66f2e88192cd8158dae2970a5fd60807f5b42494ea2b829d22fed80ac49d1
Deleted: sha256:2d8b113c9c7e410fb469aedf3e654d628aa5c244dd58d6f6c67a3f933cc517a0
Deleted: sha256:7f8c9a6337e852abc3bc18cd1c1dce52927fc5577cac5a6beca6256fe507c250
Deleted: sha256:047e56c88eb512031d6431ffc8986d638806a7640ae44bbb035837a0f5c6bda2
Deleted: sha256:dad0c445a951908af1ec4037312002f0286a2f5b4cda951e2e9cab3725fe2d6f
Deleted: sha256:a51cafecec50418e7b9d1b8fdd8f7fe2f5824e12bde026e1b1a53ea0fa048858
Deleted: sha256:d15f72703c91c4514697e888030217189f9ada3212bdf369c04857910612a4b8
Deleted: sha256:4e3ece6ee8d733dd8bcf40d3c39f734c2aa1dc3365a73e339a44fb2e07290b55
Deleted: sha256:44365386bda6993d01da0cb6335e94b1657a32531a0871adaadc866f0f10dbab
Deleted: sha256:56b2e344476804ee7dbe1414c05b211a9357569a030dd3e7696d2ab4856c3ead
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210803124334]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:62e77f0cbc44bb39fa72bfa484a09f1dac21032330e5e1952cded7950a91540c]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210803124334] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:62e77f0cbc44bb39fa72bfa484a09f1dac21032330e5e1952cded7950a91540c])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:62e77f0cbc44bb39fa72bfa484a09f1dac21032330e5e1952cded7950a91540c
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:62e77f0cbc44bb39fa72bfa484a09f1dac21032330e5e1952cded7950a91540c
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:62e77f0cbc44bb39fa72bfa484a09f1dac21032330e5e1952cded7950a91540c].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 41s
107 actionable tasks: 76 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/am3rhxrvfxz2c

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #46

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/46/display/redirect>

Changes:


------------------------------------------
[...truncated 65.75 KB...]
8f5692ae8cfa: Preparing
134147e2a04d: Preparing
2c9f04a108fb: Preparing
f6c7a58d47ca: Preparing
c54e7dfe4c6f: Preparing
616cbeb6ec16: Preparing
9f8d27faf7b2: Preparing
0f39d330030d: Preparing
b46bbadb3389: Preparing
e22329e7dec8: Preparing
385f6520b33d: Preparing
f42aed5f7feb: Preparing
89819bafde36: Preparing
f3d5b8f65132: Preparing
ad83f0aa5c0a: Preparing
5a9a65095453: Preparing
4b0edb23340c: Preparing
afa3e488a0ee: Preparing
616cbeb6ec16: Waiting
89819bafde36: Waiting
9f8d27faf7b2: Waiting
f3d5b8f65132: Waiting
ad83f0aa5c0a: Waiting
0f39d330030d: Waiting
5a9a65095453: Waiting
b46bbadb3389: Waiting
e22329e7dec8: Waiting
4b0edb23340c: Waiting
f42aed5f7feb: Waiting
385f6520b33d: Waiting
c54e7dfe4c6f: Waiting
f6c7a58d47ca: Pushed
8f5692ae8cfa: Pushed
134147e2a04d: Pushed
2c9f04a108fb: Pushed
b600ffb1c25a: Pushed
c54e7dfe4c6f: Pushed
0f39d330030d: Pushed
9f8d27faf7b2: Pushed
616cbeb6ec16: Pushed
f42aed5f7feb: Layer already exists
f3d5b8f65132: Layer already exists
89819bafde36: Layer already exists
5a9a65095453: Layer already exists
ad83f0aa5c0a: Layer already exists
4b0edb23340c: Layer already exists
afa3e488a0ee: Layer already exists
385f6520b33d: Pushed
e22329e7dec8: Pushed
b46bbadb3389: Pushed
20210802124334: digest: sha256:8963707002fd9b5dc12b7d09a0c19e297a627bcc313ad15589838ee119735116 size: 4310

> Task :sdks:java:testing:load-tests:run
Aug 02, 2021 12:48:16 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Aug 02, 2021 12:48:16 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 191 files. Enable logging at DEBUG level to see which files will be staged.
Aug 02, 2021 12:48:17 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Aug 02, 2021 12:48:17 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Aug 02, 2021 12:48:19 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Aug 02, 2021 12:48:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <112637 bytes, hash 6af3d3ba09b7b2b854e5a8aaa1bf8045e5e5fa00573b70dca01753a2cbdabca2> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-avPTugm3srhU5aiqob-AReXl-gBXO3DcoBdTosvavKI.pb
Aug 02, 2021 12:48:21 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 191 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Aug 02, 2021 12:48:21 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 191 files cached, 0 files newly uploaded in 0 seconds
Aug 02, 2021 12:48:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Aug 02, 2021 12:48:21 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c3c4a71, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1352434e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4f9a6c2d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2b6fcb9f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@75de6341, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@74170687, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@68f0f72c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3d96fa9e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3b545206, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77bb48d5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@181d8899, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12d5c30e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@b887730, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26586b74, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@52f57666, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6e041285, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@267dc982, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@439b15f2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3aa41da1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@74fab04a]
Aug 02, 2021 12:48:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Aug 02, 2021 12:48:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Aug 02, 2021 12:48:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Aug 02, 2021 12:48:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Aug 02, 2021 12:48:21 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@44ed0a8f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@32177fa5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@a96d56c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6ab4a5b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2abe9173, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@235d29d6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1fdca564, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43f9dd56, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1d12e953, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@57cb70be, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2d4608a6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@20d87335, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2a8a4e0c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26c89563, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3bd6ba24, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58f437b0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@20f6f88c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4277127c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4c7e978c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@354e7004]
Aug 02, 2021 12:48:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Aug 02, 2021 12:48:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Aug 02, 2021 12:48:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Aug 02, 2021 12:48:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Aug 02, 2021 12:48:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Aug 02, 2021 12:48:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Aug 02, 2021 12:48:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Aug 02, 2021 12:48:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Aug 02, 2021 12:48:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Aug 02, 2021 12:48:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Aug 02, 2021 12:48:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Aug 02, 2021 12:48:21 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.33.0-SNAPSHOT
Aug 02, 2021 12:48:23 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-08-02_05_48_22-15151383264437292929?project=apache-beam-testing
Aug 02, 2021 12:48:23 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-08-02_05_48_22-15151383264437292929
Aug 02, 2021 12:48:23 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-08-02_05_48_22-15151383264437292929
Aug 02, 2021 12:48:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-08-02T12:48:28.901Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-08-ptuq. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Aug 02, 2021 12:48:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-02T12:48:32.487Z: Worker configuration: e2-standard-2 in us-central1-a.
Aug 02, 2021 12:48:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-02T12:48:33.151Z: Expanding SplittableParDo operations into optimizable parts.
Aug 02, 2021 12:48:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-02T12:48:33.182Z: Expanding CollectionToSingleton operations into optimizable parts.
Aug 02, 2021 12:48:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-02T12:48:33.235Z: Expanding CoGroupByKey operations into optimizable parts.
Aug 02, 2021 12:48:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-02T12:48:33.303Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Aug 02, 2021 12:48:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-02T12:48:33.331Z: Expanding GroupByKey operations into streaming Read/Write steps
Aug 02, 2021 12:48:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-02T12:48:33.398Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Aug 02, 2021 12:48:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-02T12:48:33.511Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Aug 02, 2021 12:48:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-02T12:48:33.533Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Aug 02, 2021 12:48:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-02T12:48:33.563Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Aug 02, 2021 12:48:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-02T12:48:33.597Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Aug 02, 2021 12:48:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-02T12:48:33.630Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Aug 02, 2021 12:48:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-02T12:48:33.663Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Aug 02, 2021 12:48:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-02T12:48:33.688Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Aug 02, 2021 12:48:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-02T12:48:33.743Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Aug 02, 2021 12:48:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-02T12:48:33.768Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Aug 02, 2021 12:48:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-02T12:48:33.801Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Aug 02, 2021 12:48:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-02T12:48:33.836Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Aug 02, 2021 12:48:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-02T12:48:33.858Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Aug 02, 2021 12:48:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-02T12:48:33.893Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Aug 02, 2021 12:48:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-02T12:48:33.925Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Aug 02, 2021 12:48:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-02T12:48:33.957Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Aug 02, 2021 12:48:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-02T12:48:33.991Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Aug 02, 2021 12:48:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-02T12:48:34.025Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Aug 02, 2021 12:48:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-02T12:48:34.070Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Aug 02, 2021 12:48:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-02T12:48:34.107Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Aug 02, 2021 12:48:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-02T12:48:34.137Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Aug 02, 2021 12:48:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-02T12:48:34.172Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Aug 02, 2021 12:48:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-02T12:48:34.206Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Aug 02, 2021 12:48:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-02T12:48:34.238Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Aug 02, 2021 12:48:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-02T12:48:34.612Z: Starting 5 ****s in us-central1-a...
Aug 02, 2021 12:48:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-02T12:48:56.372Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Aug 02, 2021 12:49:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-02T12:49:16.663Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Aug 02, 2021 12:50:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-02T12:50:07.024Z: Workers have started successfully.
Aug 02, 2021 12:50:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-02T12:50:07.056Z: Workers have started successfully.
Aug 02, 2021 4:00:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-02T16:00:38.150Z: Cancel request is committed for workflow job: 2021-08-02_05_48_22-15151383264437292929.
Aug 02, 2021 4:00:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-02T16:00:39.120Z: Cleaning up.
Aug 02, 2021 4:00:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-02T16:00:39.215Z: Stopping **** pool...
Aug 02, 2021 4:00:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-02T16:00:39.260Z: Stopping **** pool...
Aug 02, 2021 4:02:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-02T16:02:52.643Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Aug 02, 2021 4:02:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-02T16:02:52.678Z: Worker pool stopped.
Aug 02, 2021 4:03:00 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-08-02_05_48_22-15151383264437292929 finished with status CANCELLED.
Aug 02, 2021 4:03:00 PM org.apache.beam.sdk.testutils.metrics.MetricsReader getCounterMetric
SEVERE: Failed to get metric totalBytes.count, from namespace co_gbk
Load test results for test (ID): 776677d0-0ae7-4418-8051-fdca1c2861ac and timestamp: 2021-08-02T12:48:17.179000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                       0.0
dataflow_v2_java11_total_bytes_count                      -1.0
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210802124334
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:8963707002fd9b5dc12b7d09a0c19e297a627bcc313ad15589838ee119735116
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210802124334]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:8963707002fd9b5dc12b7d09a0c19e297a627bcc313ad15589838ee119735116]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210802124334] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:8963707002fd9b5dc12b7d09a0c19e297a627bcc313ad15589838ee119735116])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:8963707002fd9b5dc12b7d09a0c19e297a627bcc313ad15589838ee119735116
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:8963707002fd9b5dc12b7d09a0c19e297a627bcc313ad15589838ee119735116
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:8963707002fd9b5dc12b7d09a0c19e297a627bcc313ad15589838ee119735116].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 44s
107 actionable tasks: 76 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/eqrvzcxn2avzk

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #45

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/45/display/redirect?page=changes>

Changes:

[noreply] Merge pull request #15247: [BEAM-12686] Enable self-managed relative


------------------------------------------
[...truncated 95.92 KB...]
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 404: Not Found
ERROR:root:Invalid url for org.jacoco.ant-0.8.6: https://www.eclipse.org/legal/epl-2.0/ after 9 retries.
Traceback (most recent call last):
  File "<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src/sdks/java/container/license_scripts/pull_licenses_java.py",> line 53, in pull_from_url
    url_read = urlopen(url)
  File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib/python3.5/urllib/request.py", line 472, in open
    response = meth(req, response)
  File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python3.5/urllib/request.py", line 510, in error
    return self._call_chain(*args)
  File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.5/urllib/request.py", line 590, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 404: Not Found
ERROR:root:Invalid url for org.jacoco.core-0.8.6: https://www.eclipse.org/legal/epl-2.0/. Retrying...
Traceback (most recent call last):
  File "<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src/sdks/java/container/license_scripts/pull_licenses_java.py",> line 53, in pull_from_url
    url_read = urlopen(url)
  File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib/python3.5/urllib/request.py", line 472, in open
    response = meth(req, response)
  File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python3.5/urllib/request.py", line 510, in error
    return self._call_chain(*args)
  File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.5/urllib/request.py", line 590, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 404: Not Found
ERROR:root:Invalid url for org.jacoco.core-0.8.6: https://www.eclipse.org/legal/epl-2.0/. Retrying...
Traceback (most recent call last):
  File "<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src/sdks/java/container/license_scripts/pull_licenses_java.py",> line 53, in pull_from_url
    url_read = urlopen(url)
  File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib/python3.5/urllib/request.py", line 472, in open
    response = meth(req, response)
  File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python3.5/urllib/request.py", line 510, in error
    return self._call_chain(*args)
  File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.5/urllib/request.py", line 590, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 404: Not Found
ERROR:root:Invalid url for org.jacoco.core-0.8.6: https://www.eclipse.org/legal/epl-2.0/. Retrying...
Traceback (most recent call last):
  File "<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src/sdks/java/container/license_scripts/pull_licenses_java.py",> line 53, in pull_from_url
    url_read = urlopen(url)
  File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib/python3.5/urllib/request.py", line 472, in open
    response = meth(req, response)
  File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python3.5/urllib/request.py", line 510, in error
    return self._call_chain(*args)
  File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.5/urllib/request.py", line 590, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 404: Not Found
ERROR:root:Invalid url for org.jacoco.core-0.8.6: https://www.eclipse.org/legal/epl-2.0/. Retrying...
Traceback (most recent call last):
  File "<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src/sdks/java/container/license_scripts/pull_licenses_java.py",> line 53, in pull_from_url
    url_read = urlopen(url)
  File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib/python3.5/urllib/request.py", line 472, in open
    response = meth(req, response)
  File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python3.5/urllib/request.py", line 510, in error
    return self._call_chain(*args)
  File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.5/urllib/request.py", line 590, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 404: Not Found
ERROR:root:Invalid url for org.jacoco.core-0.8.6: https://www.eclipse.org/legal/epl-2.0/. Retrying...
Traceback (most recent call last):
  File "<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src/sdks/java/container/license_scripts/pull_licenses_java.py",> line 53, in pull_from_url
    url_read = urlopen(url)
  File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib/python3.5/urllib/request.py", line 472, in open
    response = meth(req, response)
  File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python3.5/urllib/request.py", line 510, in error
    return self._call_chain(*args)
  File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.5/urllib/request.py", line 590, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 404: Not Found
ERROR:root:Invalid url for org.jacoco.core-0.8.6: https://www.eclipse.org/legal/epl-2.0/. Retrying...
Traceback (most recent call last):
  File "<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src/sdks/java/container/license_scripts/pull_licenses_java.py",> line 53, in pull_from_url
    url_read = urlopen(url)
  File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib/python3.5/urllib/request.py", line 472, in open
    response = meth(req, response)
  File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python3.5/urllib/request.py", line 510, in error
    return self._call_chain(*args)
  File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.5/urllib/request.py", line 590, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 404: Not Found
ERROR:root:Invalid url for org.jacoco.core-0.8.6: https://www.eclipse.org/legal/epl-2.0/. Retrying...

> Task :sdks:go:resolveBuildDependencies
Resolving google.golang.org/api: commit='0324d5e90dc7753607860272666845fad9ceb97e', urls=[https://code.googlesource.com/google-api-go-client]

> Task :sdks:java:container:pullLicenses
Traceback (most recent call last):
  File "<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src/sdks/java/container/license_scripts/pull_licenses_java.py",> line 53, in pull_from_url
    url_read = urlopen(url)
  File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib/python3.5/urllib/request.py", line 472, in open
    response = meth(req, response)
  File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python3.5/urllib/request.py", line 510, in error
    return self._call_chain(*args)
  File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.5/urllib/request.py", line 590, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 404: Not Found
ERROR:root:Invalid url for org.jacoco.core-0.8.6: https://www.eclipse.org/legal/epl-2.0/. Retrying...
Traceback (most recent call last):
  File "<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src/sdks/java/container/license_scripts/pull_licenses_java.py",> line 53, in pull_from_url
    url_read = urlopen(url)
  File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib/python3.5/urllib/request.py", line 472, in open
    response = meth(req, response)
  File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python3.5/urllib/request.py", line 510, in error
    return self._call_chain(*args)
  File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.5/urllib/request.py", line 590, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 404: Not Found
ERROR:root:Invalid url for org.jacoco.core-0.8.6: https://www.eclipse.org/legal/epl-2.0/ after 9 retries.

> Task :sdks:go:resolveBuildDependencies
Resolving google.golang.org/genproto: commit='4d944d34d83c502a5f761500a14d8842648415c3', urls=[https://github.com/google/go-genproto]
Resolving google.golang.org/grpc: commit='5e8f83304c0563d1ba74db05fee83d9c18ab9a58', urls=[https://github.com/grpc/grpc-go]
Resolving google.golang.org/protobuf: commit='d165be301fb1e13390ad453281ded24385fd8ebc', urls=[https://go.googlesource.com/protobuf]
Resolving cached github.com/etcd-io/etcd: commit='11214aa33bf5a47d3d9d8dafe0f6b97237dfe921', urls=[https://github.com/etcd-io/etcd.git, git@github.com:etcd-io/etcd.git]
Resolving cached github.com/etcd-io/etcd: commit='11214aa33bf5a47d3d9d8dafe0f6b97237dfe921', urls=[https://github.com/etcd-io/etcd.git, git@github.com:etcd-io/etcd.git]

> Task :sdks:go:installDependencies
> Task :sdks:go:buildLinuxAmd64
> Task :sdks:go:goBuild

> Task :sdks:java:container:pullLicenses
Traceback (most recent call last):
  File "<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src/sdks/java/container/license_scripts/pull_licenses_java.py",> line 53, in pull_from_url
    url_read = urlopen(url)
  File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib/python3.5/urllib/request.py", line 472, in open
    response = meth(req, response)
  File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python3.5/urllib/request.py", line 510, in error
    return self._call_chain(*args)
  File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.5/urllib/request.py", line 590, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 404: Not Found
ERROR:root:Invalid url for junit-4.13.1: http://www.eclipse.org/legal/epl-v10.html. Retrying...
Traceback (most recent call last):
  File "<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src/sdks/java/container/license_scripts/pull_licenses_java.py",> line 53, in pull_from_url
    url_read = urlopen(url)
  File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib/python3.5/urllib/request.py", line 472, in open
    response = meth(req, response)
  File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python3.5/urllib/request.py", line 510, in error
    return self._call_chain(*args)
  File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.5/urllib/request.py", line 590, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 404: Not Found
ERROR:root:Invalid url for junit-4.13.1: http://www.eclipse.org/legal/epl-v10.html after 9 retries.
ERROR:root:['org.jacoco.report-0.8.6', 'org.jacoco.agent-0.8.6', 'org.jacoco.ant-0.8.6', 'org.jacoco.core-0.8.6', 'junit-4.13.1']
ERROR:root:**************************************** Licenses were not able to be pulled automatically for some dependencies. Please search source code of the dependencies on the internet and add "license" and "notice" (if available) field to <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src/sdks/java/container/license_scripts/dep_urls_java.yaml> for each missing license. Dependency List: [junit-4.13.1,org.jacoco.agent-0.8.6,org.jacoco.ant-0.8.6,org.jacoco.core-0.8.6,org.jacoco.report-0.8.6]
INFO:root:pull_licenses_java.py failed. It took 357.260858 seconds with 16 threads.
Traceback (most recent call last):
  File "<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src/sdks/java/container/license_scripts/pull_licenses_java.py",> line 315, in <module>
    error_msg)
RuntimeError: ('1 error(s) occurred.', ['**************************************** Licenses were not able to be pulled automatically for some dependencies. Please search source code of the dependencies on the internet and add "license" and "notice" (if available) field to <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src/sdks/java/container/license_scripts/dep_urls_java.yaml> for each missing license. Dependency List: [junit-4.13.1,org.jacoco.agent-0.8.6,org.jacoco.ant-0.8.6,org.jacoco.core-0.8.6,org.jacoco.report-0.8.6]'])

> Task :sdks:java:container:pullLicenses FAILED

> Task :sdks:java:container:resolveBuildDependencies
Resolving ./github.com/apache/beam/sdks/go@<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src/sdks/go>

> Task :sdks:java:container:installDependencies
> Task :sdks:java:container:buildLinuxAmd64
> Task :sdks:java:container:goBuild
> Task :sdks:java:container:java11:copySdkHarnessLauncher

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:container:pullLicenses'.
> Process 'command './license_scripts/license_script.sh'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 6m 39s
102 actionable tasks: 71 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/foulihzmchh4e

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #44

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/44/display/redirect>

Changes:


------------------------------------------
[...truncated 66.31 KB...]
7674e6dadcf6: Preparing
a01751377ab7: Preparing
3f7c56606f01: Preparing
2e652fe408c0: Preparing
eff322497040: Preparing
bef10fb1d716: Preparing
461fae0d99df: Preparing
a0c3e1863e09: Preparing
5f3c9c999cf2: Preparing
7166114be0d2: Preparing
f42aed5f7feb: Preparing
89819bafde36: Preparing
f3d5b8f65132: Preparing
ad83f0aa5c0a: Preparing
5a9a65095453: Preparing
4b0edb23340c: Preparing
afa3e488a0ee: Preparing
2e652fe408c0: Waiting
7166114be0d2: Waiting
5f3c9c999cf2: Waiting
f42aed5f7feb: Waiting
a0c3e1863e09: Waiting
5a9a65095453: Waiting
461fae0d99df: Waiting
bef10fb1d716: Waiting
afa3e488a0ee: Waiting
89819bafde36: Waiting
4b0edb23340c: Waiting
eff322497040: Waiting
f3d5b8f65132: Waiting
ad83f0aa5c0a: Waiting
1e937bbf426f: Pushed
3f7c56606f01: Pushed
7674e6dadcf6: Pushed
12447b6d47f7: Pushed
2e652fe408c0: Pushed
a01751377ab7: Pushed
bef10fb1d716: Pushed
461fae0d99df: Pushed
f42aed5f7feb: Layer already exists
89819bafde36: Layer already exists
f3d5b8f65132: Layer already exists
5f3c9c999cf2: Pushed
ad83f0aa5c0a: Layer already exists
4b0edb23340c: Layer already exists
5a9a65095453: Layer already exists
afa3e488a0ee: Layer already exists
7166114be0d2: Pushed
eff322497040: Pushed
a0c3e1863e09: Pushed
20210731124337: digest: sha256:73d2dfe25326c70193421e7f65b4b69a6fc8047f7243efdbb68170ecea093133 size: 4310

> Task :sdks:java:testing:load-tests:run
Jul 31, 2021 12:48:43 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Jul 31, 2021 12:48:43 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 191 files. Enable logging at DEBUG level to see which files will be staged.
Jul 31, 2021 12:48:44 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Jul 31, 2021 12:48:44 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Jul 31, 2021 12:48:47 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Jul 31, 2021 12:48:47 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <112699 bytes, hash 985c27758ec23dc6b5621569ebda5cf5359a04305b07fa93454a7fe8d4ef94bf> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-mFwndY7CPca1YhVp69pc9TWaBDBbB_qTRUp_6NTvlL8.pb
Jul 31, 2021 12:48:49 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 191 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Jul 31, 2021 12:48:49 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 191 files cached, 0 files newly uploaded in 0 seconds
Jul 31, 2021 12:48:49 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Jul 31, 2021 12:48:49 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1352434e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4f9a6c2d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2b6fcb9f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@75de6341, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@74170687, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@68f0f72c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3d96fa9e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3b545206, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77bb48d5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@181d8899, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12d5c30e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@b887730, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26586b74, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@52f57666, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6e041285, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@267dc982, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@439b15f2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3aa41da1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@74fab04a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c83468e]
Jul 31, 2021 12:48:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Jul 31, 2021 12:48:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Jul 31, 2021 12:48:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Jul 31, 2021 12:48:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Jul 31, 2021 12:48:50 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@32177fa5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@a96d56c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6ab4a5b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2abe9173, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@235d29d6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1fdca564, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43f9dd56, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1d12e953, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@57cb70be, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2d4608a6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@20d87335, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2a8a4e0c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26c89563, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3bd6ba24, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58f437b0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@20f6f88c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4277127c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4c7e978c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@354e7004, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@22ebccb9]
Jul 31, 2021 12:48:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Jul 31, 2021 12:48:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Jul 31, 2021 12:48:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Jul 31, 2021 12:48:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Jul 31, 2021 12:48:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Jul 31, 2021 12:48:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Jul 31, 2021 12:48:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Jul 31, 2021 12:48:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Jul 31, 2021 12:48:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Jul 31, 2021 12:48:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Jul 31, 2021 12:48:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Jul 31, 2021 12:48:50 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.33.0-SNAPSHOT
Jul 31, 2021 12:48:51 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-07-31_05_48_50-17410421294700589784?project=apache-beam-testing
Jul 31, 2021 12:48:51 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-07-31_05_48_50-17410421294700589784
Jul 31, 2021 12:48:51 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-07-31_05_48_50-17410421294700589784
Jul 31, 2021 12:48:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-07-31T12:48:57.914Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-07-kr5o. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Jul 31, 2021 12:49:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-31T12:49:02.385Z: Worker configuration: e2-standard-2 in us-central1-b.
Jul 31, 2021 12:49:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-31T12:49:03.153Z: Expanding SplittableParDo operations into optimizable parts.
Jul 31, 2021 12:49:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-31T12:49:03.214Z: Expanding CollectionToSingleton operations into optimizable parts.
Jul 31, 2021 12:49:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-31T12:49:03.286Z: Expanding CoGroupByKey operations into optimizable parts.
Jul 31, 2021 12:49:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-31T12:49:03.362Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Jul 31, 2021 12:49:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-31T12:49:03.396Z: Expanding GroupByKey operations into streaming Read/Write steps
Jul 31, 2021 12:49:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-31T12:49:03.489Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jul 31, 2021 12:49:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-31T12:49:03.602Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jul 31, 2021 12:49:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-31T12:49:03.643Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Jul 31, 2021 12:49:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-31T12:49:03.686Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Jul 31, 2021 12:49:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-31T12:49:03.714Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Jul 31, 2021 12:49:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-31T12:49:03.743Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Jul 31, 2021 12:49:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-31T12:49:03.780Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jul 31, 2021 12:49:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-31T12:49:03.805Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jul 31, 2021 12:49:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-31T12:49:03.835Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jul 31, 2021 12:49:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-31T12:49:03.876Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Jul 31, 2021 12:49:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-31T12:49:03.909Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Jul 31, 2021 12:49:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-31T12:49:03.944Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Jul 31, 2021 12:49:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-31T12:49:03.981Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Jul 31, 2021 12:49:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-31T12:49:04.014Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jul 31, 2021 12:49:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-31T12:49:04.049Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jul 31, 2021 12:49:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-31T12:49:04.105Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jul 31, 2021 12:49:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-31T12:49:04.146Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Jul 31, 2021 12:49:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-31T12:49:04.183Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Jul 31, 2021 12:49:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-31T12:49:04.218Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Jul 31, 2021 12:49:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-31T12:49:04.250Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Jul 31, 2021 12:49:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-31T12:49:04.293Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Jul 31, 2021 12:49:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-31T12:49:04.326Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Jul 31, 2021 12:49:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-31T12:49:04.355Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Jul 31, 2021 12:49:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-31T12:49:04.392Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Jul 31, 2021 12:49:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-31T12:49:04.850Z: Starting 5 ****s in us-central1-b...
Jul 31, 2021 12:49:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-31T12:49:32.053Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jul 31, 2021 12:49:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-31T12:49:51.266Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Jul 31, 2021 12:50:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-31T12:50:44.313Z: Workers have started successfully.
Jul 31, 2021 12:50:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-31T12:50:44.345Z: Workers have started successfully.
Jul 31, 2021 4:00:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-31T16:00:49.959Z: Cancel request is committed for workflow job: 2021-07-31_05_48_50-17410421294700589784.
Jul 31, 2021 4:00:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-31T16:00:50.075Z: Cleaning up.
Jul 31, 2021 4:00:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-31T16:00:50.185Z: Stopping **** pool...
Jul 31, 2021 4:00:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-31T16:00:50.228Z: Stopping **** pool...
Jul 31, 2021 4:03:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-31T16:03:07.185Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Jul 31, 2021 4:03:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-31T16:03:07.231Z: Worker pool stopped.
Jul 31, 2021 4:03:14 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-07-31_05_48_50-17410421294700589784 finished with status CANCELLED.
Jul 31, 2021 4:03:15 PM org.apache.beam.sdk.testutils.metrics.MetricsReader getCounterMetric
SEVERE: Failed to get metric totalBytes.count, from namespace co_gbk
Load test results for test (ID): a1f75527-2b58-433b-ba4d-6ba3e886328c and timestamp: 2021-07-31T12:48:43.922000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                       0.0
dataflow_v2_java11_total_bytes_count                      -1.0
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210731124337
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:73d2dfe25326c70193421e7f65b4b69a6fc8047f7243efdbb68170ecea093133
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210731124337]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:73d2dfe25326c70193421e7f65b4b69a6fc8047f7243efdbb68170ecea093133]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210731124337] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:73d2dfe25326c70193421e7f65b4b69a6fc8047f7243efdbb68170ecea093133])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:73d2dfe25326c70193421e7f65b4b69a6fc8047f7243efdbb68170ecea093133
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:73d2dfe25326c70193421e7f65b4b69a6fc8047f7243efdbb68170ecea093133
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:73d2dfe25326c70193421e7f65b4b69a6fc8047f7243efdbb68170ecea093133].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 57s
107 actionable tasks: 77 executed, 28 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/iccsshfefqmfs

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #43

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/43/display/redirect?page=changes>

Changes:

[ajamato] [BEAM-11994] Update BigQueryStorageStreamSource and BigQueryServicesImpl

[ajamato] [BEAM-12670] Fix regression, properly report API call metric properly on

[noreply] [BEAM-9615] Recover from registration panics. (#15228)

[noreply] [BEAM-12609] Enable projection pushdown in SchemaIO. (#15216)

[noreply] [BEAM-12675] Fix issue with language switcher on hadoop page (#15243)


------------------------------------------
[...truncated 66.42 KB...]
7a7087e4d1b6: Waiting
a7038a1ca17d: Waiting
f42aed5f7feb: Waiting
da3ed76e2847: Waiting
89819bafde36: Waiting
29b3750e5db8: Waiting
f3d5b8f65132: Waiting
ad83f0aa5c0a: Waiting
5a9a65095453: Waiting
4b0edb23340c: Waiting
afa3e488a0ee: Waiting
787efa9f74d6: Waiting
70a184f6853a: Pushed
a62b01e7150a: Pushed
4a2c325fec4d: Pushed
ff575c09d99e: Pushed
787efa9f74d6: Pushed
f3b576e3fc83: Pushed
5543cca198e1: Pushed
29b3750e5db8: Pushed
f42aed5f7feb: Layer already exists
89819bafde36: Layer already exists
f3d5b8f65132: Layer already exists
ad83f0aa5c0a: Layer already exists
5a9a65095453: Layer already exists
4b0edb23340c: Layer already exists
afa3e488a0ee: Layer already exists
a7038a1ca17d: Pushed
0c0e0a32e5b2: Pushed
7a7087e4d1b6: Pushed
da3ed76e2847: Pushed
20210730124335: digest: sha256:e369fb15c6ef3b7d20ef9995c4604ad8f1981404422333abcc31d518de02358f size: 4310

> Task :sdks:java:testing:load-tests:run
Jul 30, 2021 12:48:47 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Jul 30, 2021 12:48:47 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 191 files. Enable logging at DEBUG level to see which files will be staged.
Jul 30, 2021 12:48:48 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Jul 30, 2021 12:48:48 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Jul 30, 2021 12:48:50 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Jul 30, 2021 12:48:50 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <112637 bytes, hash 7dc118bb3a61d0089d2163d9074763311aef39683b77fc357860ea2c09934d0a> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-fcEYuzph0AidIWPZB0djMRrvOWg7d_w1eGDqLAmTTQo.pb
Jul 30, 2021 12:48:52 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 191 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Jul 30, 2021 12:48:52 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 191 files cached, 0 files newly uploaded in 0 seconds
Jul 30, 2021 12:48:52 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Jul 30, 2021 12:48:52 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1352434e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4f9a6c2d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2b6fcb9f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@75de6341, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@74170687, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@68f0f72c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3d96fa9e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3b545206, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77bb48d5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@181d8899, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12d5c30e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@b887730, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26586b74, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@52f57666, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6e041285, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@267dc982, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@439b15f2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3aa41da1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@74fab04a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c83468e]
Jul 30, 2021 12:48:53 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Jul 30, 2021 12:48:53 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Jul 30, 2021 12:48:53 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Jul 30, 2021 12:48:53 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Jul 30, 2021 12:48:53 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@32177fa5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@a96d56c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6ab4a5b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2abe9173, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@235d29d6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1fdca564, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43f9dd56, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1d12e953, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@57cb70be, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2d4608a6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@20d87335, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2a8a4e0c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26c89563, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3bd6ba24, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58f437b0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@20f6f88c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4277127c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4c7e978c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@354e7004, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@22ebccb9]
Jul 30, 2021 12:48:53 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Jul 30, 2021 12:48:53 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Jul 30, 2021 12:48:53 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Jul 30, 2021 12:48:53 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Jul 30, 2021 12:48:53 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Jul 30, 2021 12:48:53 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Jul 30, 2021 12:48:53 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Jul 30, 2021 12:48:53 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Jul 30, 2021 12:48:53 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Jul 30, 2021 12:48:53 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Jul 30, 2021 12:48:53 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Jul 30, 2021 12:48:53 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.33.0-SNAPSHOT
Jul 30, 2021 12:48:54 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-07-30_05_48_53-9970974462263149018?project=apache-beam-testing
Jul 30, 2021 12:48:54 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-07-30_05_48_53-9970974462263149018
Jul 30, 2021 12:48:54 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-07-30_05_48_53-9970974462263149018
Jul 30, 2021 12:49:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-07-30T12:49:01.192Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-07-x93e. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Jul 30, 2021 12:49:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-30T12:49:06.113Z: Worker configuration: e2-standard-2 in us-central1-b.
Jul 30, 2021 12:49:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-30T12:49:06.863Z: Expanding SplittableParDo operations into optimizable parts.
Jul 30, 2021 12:49:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-30T12:49:06.923Z: Expanding CollectionToSingleton operations into optimizable parts.
Jul 30, 2021 12:49:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-30T12:49:07.000Z: Expanding CoGroupByKey operations into optimizable parts.
Jul 30, 2021 12:49:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-30T12:49:07.125Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Jul 30, 2021 12:49:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-30T12:49:07.294Z: Expanding GroupByKey operations into streaming Read/Write steps
Jul 30, 2021 12:49:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-30T12:49:07.352Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jul 30, 2021 12:49:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-30T12:49:07.482Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jul 30, 2021 12:49:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-30T12:49:07.527Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Jul 30, 2021 12:49:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-30T12:49:07.557Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Jul 30, 2021 12:49:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-30T12:49:07.591Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Jul 30, 2021 12:49:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-30T12:49:07.618Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Jul 30, 2021 12:49:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-30T12:49:07.656Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jul 30, 2021 12:49:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-30T12:49:07.684Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jul 30, 2021 12:49:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-30T12:49:07.748Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jul 30, 2021 12:49:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-30T12:49:07.773Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Jul 30, 2021 12:49:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-30T12:49:07.809Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Jul 30, 2021 12:49:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-30T12:49:07.845Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Jul 30, 2021 12:49:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-30T12:49:07.901Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Jul 30, 2021 12:49:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-30T12:49:07.934Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jul 30, 2021 12:49:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-30T12:49:07.978Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jul 30, 2021 12:49:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-30T12:49:08.028Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jul 30, 2021 12:49:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-30T12:49:08.064Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Jul 30, 2021 12:49:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-30T12:49:08.106Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Jul 30, 2021 12:49:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-30T12:49:08.145Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Jul 30, 2021 12:49:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-30T12:49:08.207Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Jul 30, 2021 12:49:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-30T12:49:08.248Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Jul 30, 2021 12:49:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-30T12:49:08.285Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Jul 30, 2021 12:49:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-30T12:49:08.325Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Jul 30, 2021 12:49:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-30T12:49:08.372Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Jul 30, 2021 12:49:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-30T12:49:08.767Z: Starting 5 ****s in us-central1-b...
Jul 30, 2021 12:49:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-30T12:49:31.592Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jul 30, 2021 12:49:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-30T12:49:51.284Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Jul 30, 2021 12:50:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-30T12:50:40.486Z: Workers have started successfully.
Jul 30, 2021 12:50:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-30T12:50:40.514Z: Workers have started successfully.
Jul 30, 2021 4:00:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-30T16:00:34.911Z: Cancel request is committed for workflow job: 2021-07-30_05_48_53-9970974462263149018.
Jul 30, 2021 4:00:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-30T16:00:35.003Z: Cleaning up.
Jul 30, 2021 4:00:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-30T16:00:35.080Z: Stopping **** pool...
Jul 30, 2021 4:00:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-30T16:00:35.121Z: Stopping **** pool...
Jul 30, 2021 4:03:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-30T16:02:59.874Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Jul 30, 2021 4:03:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-30T16:02:59.905Z: Worker pool stopped.
Jul 30, 2021 4:03:05 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-07-30_05_48_53-9970974462263149018 finished with status CANCELLED.
Jul 30, 2021 4:03:05 PM org.apache.beam.sdk.testutils.metrics.MetricsReader getCounterMetric
SEVERE: Failed to get metric totalBytes.count, from namespace co_gbk
Load test results for test (ID): 05b57e5f-97a9-497b-a809-2cedeb634591 and timestamp: 2021-07-30T12:48:47.914000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                       0.0
dataflow_v2_java11_total_bytes_count                      -1.0
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210730124335
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:e369fb15c6ef3b7d20ef9995c4604ad8f1981404422333abcc31d518de02358f
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210730124335]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:e369fb15c6ef3b7d20ef9995c4604ad8f1981404422333abcc31d518de02358f]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210730124335] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:e369fb15c6ef3b7d20ef9995c4604ad8f1981404422333abcc31d518de02358f])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:e369fb15c6ef3b7d20ef9995c4604ad8f1981404422333abcc31d518de02358f
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:e369fb15c6ef3b7d20ef9995c4604ad8f1981404422333abcc31d518de02358f
ERROR: (gcloud.container.images.delete) Not found: response: {'status': '404', 'content-length': '168', 'x-xss-protection': '0', 'transfer-encoding': 'chunked', 'server': 'Docker Registry', '-content-encoding': 'gzip', 'docker-distribution-api-version': 'registry/2.0', 'cache-control': 'private', 'date': 'Fri, 30 Jul 2021 16:03:09 GMT', 'x-frame-options': 'SAMEORIGIN', 'content-type': 'application/json'}
Failed to compute blob liveness for manifest: 'sha256:e369fb15c6ef3b7d20ef9995c4604ad8f1981404422333abcc31d518de02358f': None

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages FAILED

FAILURE: Build completed with 2 failures.

1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

2: Task failed with an exception.
-----------
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src/runners/google-cloud-dataflow-java/build.gradle'> line: 284

* What went wrong:
Execution failed for task ':runners:google-cloud-dataflow-java:cleanUpDockerImages'.
> Process 'command './scripts/cleanup_untagged_gcr_images.sh'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 51s
107 actionable tasks: 76 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/4mdut2adi672a

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #42

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/42/display/redirect?page=changes>

Changes:

[kawaigin] [BEAM-12506] Changed WindowedValueHolder into a Row type

[noreply] [BEAM-1833] Preserve inputs names at graph construction and through


------------------------------------------
[...truncated 65.30 KB...]
2bb13447655c: Preparing
78ed668a4852: Preparing
3d8d85c9c0de: Preparing
af62db1b1116: Preparing
b19d96f825f5: Preparing
c3a7535aa70b: Preparing
878ef81c68dc: Preparing
d370076f1798: Preparing
7980d3328a04: Preparing
5d209a104ad1: Preparing
f42aed5f7feb: Preparing
89819bafde36: Preparing
f3d5b8f65132: Preparing
ad83f0aa5c0a: Preparing
5a9a65095453: Preparing
b19d96f825f5: Waiting
4b0edb23340c: Preparing
afa3e488a0ee: Preparing
c3a7535aa70b: Waiting
7980d3328a04: Waiting
5d209a104ad1: Waiting
878ef81c68dc: Waiting
d370076f1798: Waiting
89819bafde36: Waiting
f42aed5f7feb: Waiting
ad83f0aa5c0a: Waiting
4b0edb23340c: Waiting
f3d5b8f65132: Waiting
5a9a65095453: Waiting
afa3e488a0ee: Waiting
af62db1b1116: Waiting
3d8d85c9c0de: Pushed
51019a71e8ef: Pushed
2bb13447655c: Pushed
af62db1b1116: Pushed
71379e80424f: Pushed
c3a7535aa70b: Pushed
878ef81c68dc: Pushed
7980d3328a04: Pushed
f42aed5f7feb: Layer already exists
89819bafde36: Layer already exists
f3d5b8f65132: Layer already exists
ad83f0aa5c0a: Layer already exists
78ed668a4852: Pushed
5a9a65095453: Layer already exists
4b0edb23340c: Layer already exists
afa3e488a0ee: Layer already exists
5d209a104ad1: Pushed
b19d96f825f5: Pushed
d370076f1798: Pushed
20210729124336: digest: sha256:26719ece63b94df0119620793bf549899243a0a5ce994b75b032028df3a42454 size: 4310

> Task :sdks:java:testing:load-tests:run
Jul 29, 2021 12:48:55 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Jul 29, 2021 12:48:56 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 191 files. Enable logging at DEBUG level to see which files will be staged.
Jul 29, 2021 12:48:56 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Jul 29, 2021 12:48:56 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Jul 29, 2021 12:48:58 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Jul 29, 2021 12:48:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <112699 bytes, hash 0d2e94d3eac50b0e7973d9c2d52cddabcfc331661bd4efe3c388974afd43dfcc> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-DS6U0-rFCw55c9nC1Szdq8_DMWYb1O_jw4iXSv1D38w.pb
Jul 29, 2021 12:49:00 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 191 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Jul 29, 2021 12:49:01 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 191 files cached, 0 files newly uploaded in 0 seconds
Jul 29, 2021 12:49:01 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Jul 29, 2021 12:49:01 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@75de6341, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@74170687, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@68f0f72c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3d96fa9e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3b545206, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77bb48d5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@181d8899, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12d5c30e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@b887730, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26586b74, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@52f57666, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6e041285, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@267dc982, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@439b15f2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3aa41da1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@74fab04a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c83468e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@272ce069, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3d7fb838, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1c26273d]
Jul 29, 2021 12:49:01 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Jul 29, 2021 12:49:01 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Jul 29, 2021 12:49:01 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Jul 29, 2021 12:49:01 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Jul 29, 2021 12:49:01 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2abe9173, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@235d29d6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1fdca564, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43f9dd56, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1d12e953, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@57cb70be, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2d4608a6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@20d87335, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2a8a4e0c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26c89563, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3bd6ba24, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58f437b0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@20f6f88c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4277127c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4c7e978c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@354e7004, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@22ebccb9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5875de6a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@608fe01f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2687725a]
Jul 29, 2021 12:49:01 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Jul 29, 2021 12:49:01 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Jul 29, 2021 12:49:01 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Jul 29, 2021 12:49:01 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Jul 29, 2021 12:49:01 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Jul 29, 2021 12:49:01 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Jul 29, 2021 12:49:01 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Jul 29, 2021 12:49:01 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Jul 29, 2021 12:49:01 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Jul 29, 2021 12:49:01 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Jul 29, 2021 12:49:01 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Jul 29, 2021 12:49:01 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.33.0-SNAPSHOT
Jul 29, 2021 12:49:02 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-07-29_05_49_01-16710745880911390065?project=apache-beam-testing
Jul 29, 2021 12:49:02 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-07-29_05_49_01-16710745880911390065
Jul 29, 2021 12:49:02 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-07-29_05_49_01-16710745880911390065
Jul 29, 2021 12:49:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-07-29T12:49:09.541Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-07-q6d5. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Jul 29, 2021 12:49:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-29T12:49:13.531Z: Worker configuration: e2-standard-2 in us-central1-b.
Jul 29, 2021 12:49:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-29T12:49:14.080Z: Expanding SplittableParDo operations into optimizable parts.
Jul 29, 2021 12:49:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-29T12:49:14.108Z: Expanding CollectionToSingleton operations into optimizable parts.
Jul 29, 2021 12:49:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-29T12:49:14.175Z: Expanding CoGroupByKey operations into optimizable parts.
Jul 29, 2021 12:49:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-29T12:49:14.234Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Jul 29, 2021 12:49:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-29T12:49:14.263Z: Expanding GroupByKey operations into streaming Read/Write steps
Jul 29, 2021 12:49:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-29T12:49:14.333Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jul 29, 2021 12:49:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-29T12:49:14.426Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jul 29, 2021 12:49:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-29T12:49:14.465Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Jul 29, 2021 12:49:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-29T12:49:14.495Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Jul 29, 2021 12:49:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-29T12:49:14.528Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Jul 29, 2021 12:49:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-29T12:49:14.560Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Jul 29, 2021 12:49:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-29T12:49:14.595Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jul 29, 2021 12:49:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-29T12:49:14.628Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jul 29, 2021 12:49:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-29T12:49:14.664Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jul 29, 2021 12:49:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-29T12:49:14.686Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Jul 29, 2021 12:49:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-29T12:49:14.733Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Jul 29, 2021 12:49:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-29T12:49:14.763Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Jul 29, 2021 12:49:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-29T12:49:14.798Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Jul 29, 2021 12:49:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-29T12:49:14.833Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jul 29, 2021 12:49:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-29T12:49:14.881Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jul 29, 2021 12:49:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-29T12:49:14.912Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jul 29, 2021 12:49:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-29T12:49:14.934Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Jul 29, 2021 12:49:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-29T12:49:14.970Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Jul 29, 2021 12:49:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-29T12:49:14.995Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Jul 29, 2021 12:49:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-29T12:49:15.038Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Jul 29, 2021 12:49:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-29T12:49:15.064Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Jul 29, 2021 12:49:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-29T12:49:15.097Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Jul 29, 2021 12:49:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-29T12:49:15.130Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Jul 29, 2021 12:49:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-29T12:49:15.174Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Jul 29, 2021 12:49:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-29T12:49:15.540Z: Starting 5 ****s in us-central1-b...
Jul 29, 2021 12:49:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-29T12:49:33.902Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jul 29, 2021 12:49:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-29T12:49:55.087Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Jul 29, 2021 12:50:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-29T12:50:55.335Z: Workers have started successfully.
Jul 29, 2021 12:50:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-29T12:50:55.363Z: Workers have started successfully.
Jul 29, 2021 4:00:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-29T16:00:34.272Z: Cancel request is committed for workflow job: 2021-07-29_05_49_01-16710745880911390065.
Jul 29, 2021 4:00:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-29T16:00:34.369Z: Cleaning up.
Jul 29, 2021 4:00:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-29T16:00:34.449Z: Stopping **** pool...
Jul 29, 2021 4:00:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-29T16:00:34.494Z: Stopping **** pool...
Jul 29, 2021 4:02:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-29T16:02:56.571Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Jul 29, 2021 4:02:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-29T16:02:56.615Z: Worker pool stopped.
Jul 29, 2021 4:03:02 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-07-29_05_49_01-16710745880911390065 finished with status CANCELLED.
Jul 29, 2021 4:03:02 PM org.apache.beam.sdk.testutils.metrics.MetricsReader getCounterMetric
SEVERE: Failed to get metric totalBytes.count, from namespace co_gbk
Load test results for test (ID): 2885fb4c-3198-4f44-a3bb-6aee72a34197 and timestamp: 2021-07-29T12:48:56.446000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                       0.0
dataflow_v2_java11_total_bytes_count                      -1.0
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210729124336
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:26719ece63b94df0119620793bf549899243a0a5ce994b75b032028df3a42454
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210729124336]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:26719ece63b94df0119620793bf549899243a0a5ce994b75b032028df3a42454]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210729124336] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:26719ece63b94df0119620793bf549899243a0a5ce994b75b032028df3a42454])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:26719ece63b94df0119620793bf549899243a0a5ce994b75b032028df3a42454
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:26719ece63b94df0119620793bf549899243a0a5ce994b75b032028df3a42454
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:26719ece63b94df0119620793bf549899243a0a5ce994b75b032028df3a42454].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 56s
107 actionable tasks: 76 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/4jignuxbjw2q4

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #41

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/41/display/redirect?page=changes>

Changes:

[tysonjh] [BEAM-12589] - exclude UsesStrictTimerOrdering from Dataflow

[noreply] [BEAM-12513] Add Go to 5. pipeline io. (#15225)

[noreply] [BEAM-7195] BQ BatchLoads doesn't always create new tables (#14238)

[noreply] [BEAM-12368] Updated doc nav to match wireframe (#15229)

[noreply] [BEAM-3713] Move PostCommit_Python from nosetest to pytest (#14859)

[noreply] [BEAM-12653] Fix container cleanup for Java Dataflow tests (#15213)


------------------------------------------
[...truncated 66.13 KB...]
108b0dfbe5bc: Preparing
c4c7c0e33077: Preparing
f48013b24908: Preparing
f55a7d971ec1: Preparing
a4cec4db7f26: Preparing
564b892b76d7: Preparing
543bc57a17f0: Preparing
5d6b4323fc3f: Preparing
f42aed5f7feb: Preparing
89819bafde36: Preparing
f3d5b8f65132: Preparing
ad83f0aa5c0a: Preparing
5a9a65095453: Preparing
4b0edb23340c: Preparing
afa3e488a0ee: Preparing
f55a7d971ec1: Waiting
a4cec4db7f26: Waiting
5d6b4323fc3f: Waiting
564b892b76d7: Waiting
543bc57a17f0: Waiting
f42aed5f7feb: Waiting
5a9a65095453: Waiting
89819bafde36: Waiting
c4c7c0e33077: Waiting
4b0edb23340c: Waiting
f48013b24908: Waiting
f3d5b8f65132: Waiting
ad83f0aa5c0a: Waiting
108b0dfbe5bc: Pushed
8e00564d3a9b: Pushed
88a9f96202df: Pushed
4e5fae41fdc7: Pushed
c4c7c0e33077: Pushed
0c97714e9cf5: Pushed
f55a7d971ec1: Pushed
a4cec4db7f26: Pushed
f42aed5f7feb: Layer already exists
89819bafde36: Layer already exists
f3d5b8f65132: Layer already exists
ad83f0aa5c0a: Layer already exists
5a9a65095453: Layer already exists
4b0edb23340c: Layer already exists
afa3e488a0ee: Layer already exists
543bc57a17f0: Pushed
5d6b4323fc3f: Pushed
f48013b24908: Pushed
564b892b76d7: Pushed
20210728124333: digest: sha256:1c6b7fd9bc8e334c42983a501f3ef9384e1e3f4ba9051ac035218e12df1a6063 size: 4310

> Task :sdks:java:testing:load-tests:run
Jul 28, 2021 12:48:21 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Jul 28, 2021 12:48:21 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 191 files. Enable logging at DEBUG level to see which files will be staged.
Jul 28, 2021 12:48:22 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Jul 28, 2021 12:48:22 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Jul 28, 2021 12:48:24 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Jul 28, 2021 12:48:24 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <112637 bytes, hash 203efb4815aa918443cd9a234ae80852d1fb469e0d913e2364c6ba18f68af0b5> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-ID77SBWqkYRDzZojSugIUtH7Rp4NkT4jZMa6GPaK8LU.pb
Jul 28, 2021 12:48:26 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 191 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Jul 28, 2021 12:48:26 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 191 files cached, 0 files newly uploaded in 0 seconds
Jul 28, 2021 12:48:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Jul 28, 2021 12:48:26 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4f9a6c2d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2b6fcb9f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@75de6341, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@74170687, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@68f0f72c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3d96fa9e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3b545206, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77bb48d5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@181d8899, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12d5c30e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@b887730, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26586b74, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@52f57666, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6e041285, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@267dc982, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@439b15f2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3aa41da1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@74fab04a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c83468e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@272ce069]
Jul 28, 2021 12:48:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Jul 28, 2021 12:48:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Jul 28, 2021 12:48:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Jul 28, 2021 12:48:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Jul 28, 2021 12:48:26 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@a96d56c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6ab4a5b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2abe9173, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@235d29d6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1fdca564, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43f9dd56, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1d12e953, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@57cb70be, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2d4608a6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@20d87335, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2a8a4e0c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26c89563, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3bd6ba24, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58f437b0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@20f6f88c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4277127c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4c7e978c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@354e7004, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@22ebccb9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5875de6a]
Jul 28, 2021 12:48:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Jul 28, 2021 12:48:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Jul 28, 2021 12:48:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Jul 28, 2021 12:48:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Jul 28, 2021 12:48:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Jul 28, 2021 12:48:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Jul 28, 2021 12:48:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Jul 28, 2021 12:48:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Jul 28, 2021 12:48:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Jul 28, 2021 12:48:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Jul 28, 2021 12:48:26 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Jul 28, 2021 12:48:26 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.33.0-SNAPSHOT
Jul 28, 2021 12:48:27 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-07-28_05_48_27-877179673017108577?project=apache-beam-testing
Jul 28, 2021 12:48:27 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-07-28_05_48_27-877179673017108577
Jul 28, 2021 12:48:27 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-07-28_05_48_27-877179673017108577
Jul 28, 2021 12:48:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-07-28T12:48:34.259Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-07-2akh. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Jul 28, 2021 12:48:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-28T12:48:37.809Z: Worker configuration: e2-standard-2 in us-central1-b.
Jul 28, 2021 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-28T12:48:38.501Z: Expanding SplittableParDo operations into optimizable parts.
Jul 28, 2021 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-28T12:48:38.539Z: Expanding CollectionToSingleton operations into optimizable parts.
Jul 28, 2021 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-28T12:48:38.596Z: Expanding CoGroupByKey operations into optimizable parts.
Jul 28, 2021 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-28T12:48:38.674Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Jul 28, 2021 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-28T12:48:38.708Z: Expanding GroupByKey operations into streaming Read/Write steps
Jul 28, 2021 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-28T12:48:38.774Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jul 28, 2021 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-28T12:48:38.887Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jul 28, 2021 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-28T12:48:38.916Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Jul 28, 2021 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-28T12:48:38.949Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Jul 28, 2021 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-28T12:48:38.976Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Jul 28, 2021 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-28T12:48:39.009Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Jul 28, 2021 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-28T12:48:39.035Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jul 28, 2021 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-28T12:48:39.064Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jul 28, 2021 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-28T12:48:39.095Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jul 28, 2021 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-28T12:48:39.131Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Jul 28, 2021 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-28T12:48:39.169Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Jul 28, 2021 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-28T12:48:39.199Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Jul 28, 2021 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-28T12:48:39.224Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Jul 28, 2021 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-28T12:48:39.256Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jul 28, 2021 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-28T12:48:39.287Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jul 28, 2021 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-28T12:48:39.319Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jul 28, 2021 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-28T12:48:39.343Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Jul 28, 2021 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-28T12:48:39.371Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Jul 28, 2021 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-28T12:48:39.402Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Jul 28, 2021 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-28T12:48:39.431Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Jul 28, 2021 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-28T12:48:39.457Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Jul 28, 2021 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-28T12:48:39.488Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Jul 28, 2021 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-28T12:48:39.524Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Jul 28, 2021 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-28T12:48:39.557Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Jul 28, 2021 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-28T12:48:39.944Z: Starting 5 ****s in us-central1-b...
Jul 28, 2021 12:49:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-28T12:49:00.595Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jul 28, 2021 12:49:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-28T12:49:22.745Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Jul 28, 2021 12:50:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-28T12:50:12.013Z: Workers have started successfully.
Jul 28, 2021 12:50:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-28T12:50:12.040Z: Workers have started successfully.
Jul 28, 2021 4:00:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-28T16:00:34.855Z: Cancel request is committed for workflow job: 2021-07-28_05_48_27-877179673017108577.
Jul 28, 2021 4:00:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-28T16:00:34.912Z: Cleaning up.
Jul 28, 2021 4:00:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-28T16:00:34.991Z: Stopping **** pool...
Jul 28, 2021 4:00:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-28T16:00:35.056Z: Stopping **** pool...
Jul 28, 2021 4:00:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-28T16:00:43.266Z: generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:1089
Jul 28, 2021 4:03:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-28T16:03:03.559Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Jul 28, 2021 4:03:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-28T16:03:03.647Z: Worker pool stopped.
Jul 28, 2021 4:03:11 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-07-28_05_48_27-877179673017108577 finished with status CANCELLED.
Load test results for test (ID): c6a9676f-7ef8-4992-b2dd-bd077971dfd3 and timestamp: 2021-07-28T12:48:21.887000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11387.005
dataflow_v2_java11_total_bytes_count             2.17562004E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210728124333
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:1c6b7fd9bc8e334c42983a501f3ef9384e1e3f4ba9051ac035218e12df1a6063
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210728124333]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:1c6b7fd9bc8e334c42983a501f3ef9384e1e3f4ba9051ac035218e12df1a6063]

Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210728124333] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:1c6b7fd9bc8e334c42983a501f3ef9384e1e3f4ba9051ac035218e12df1a6063])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:1c6b7fd9bc8e334c42983a501f3ef9384e1e3f4ba9051ac035218e12df1a6063
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:1c6b7fd9bc8e334c42983a501f3ef9384e1e3f4ba9051ac035218e12df1a6063
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:1c6b7fd9bc8e334c42983a501f3ef9384e1e3f4ba9051ac035218e12df1a6063].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 58s
107 actionable tasks: 77 executed, 28 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/lf2rltuzbnabi

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #40

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/40/display/redirect?page=changes>

Changes:

[noreply] [BEAM-12656] Fix go-licenses build (#15215)

[emilyye] add orjson and recommendations-ai packages to Python image

[noreply] [Beam-10166] Improve execution time errors (#15206)

[noreply] [BEAM-3878] Improve error reporting in calls.go (#15221)

[noreply] [BEAM-12661] Fix stuck GetData Windmill calls (#15224)


------------------------------------------
[...truncated 66.29 KB...]
20210727124333: digest: sha256:627cc128ed959dd39bc70f62f0b1f173dbd6b3cbf5c9ff7f346d342975e84600 size: 4310

> Task :sdks:java:testing:load-tests:run
Jul 27, 2021 12:48:12 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Jul 27, 2021 12:48:13 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 191 files. Enable logging at DEBUG level to see which files will be staged.
Jul 27, 2021 12:48:13 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Jul 27, 2021 12:48:13 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Jul 27, 2021 12:48:15 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Jul 27, 2021 12:48:15 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <112699 bytes, hash 08154425c68389c94818ac9d71af20dd30cee4423eb8720ce4eaf2f0b436549e> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-CBVEJcaDiclIGKydca8g3TDO5EI-uHIM5Ory8LQ2VJ4.pb
Jul 27, 2021 12:48:17 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 191 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Jul 27, 2021 12:48:19 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 191 files cached, 0 files newly uploaded in 2 seconds
Jul 27, 2021 12:48:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Jul 27, 2021 12:48:19 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@74170687, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@68f0f72c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3d96fa9e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3b545206, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77bb48d5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@181d8899, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12d5c30e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@b887730, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26586b74, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@52f57666, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6e041285, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@267dc982, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@439b15f2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3aa41da1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@74fab04a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c83468e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@272ce069, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3d7fb838, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1c26273d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3a37a501]
Jul 27, 2021 12:48:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Jul 27, 2021 12:48:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Jul 27, 2021 12:48:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Jul 27, 2021 12:48:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Jul 27, 2021 12:48:19 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2abe9173, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@235d29d6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1fdca564, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43f9dd56, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1d12e953, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@57cb70be, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2d4608a6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@20d87335, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2a8a4e0c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26c89563, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3bd6ba24, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58f437b0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@20f6f88c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4277127c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4c7e978c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@354e7004, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@22ebccb9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5875de6a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@608fe01f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2687725a]
Jul 27, 2021 12:48:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Jul 27, 2021 12:48:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Jul 27, 2021 12:48:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Jul 27, 2021 12:48:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Jul 27, 2021 12:48:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Jul 27, 2021 12:48:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Jul 27, 2021 12:48:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Jul 27, 2021 12:48:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Jul 27, 2021 12:48:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Jul 27, 2021 12:48:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Jul 27, 2021 12:48:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Jul 27, 2021 12:48:19 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.33.0-SNAPSHOT
Jul 27, 2021 12:48:20 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-07-27_05_48_19-11324485973340498397?project=apache-beam-testing
Jul 27, 2021 12:48:20 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-07-27_05_48_19-11324485973340498397
Jul 27, 2021 12:48:20 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-07-27_05_48_19-11324485973340498397
Jul 27, 2021 12:48:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-07-27T12:48:28.608Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-07-5wt2. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Jul 27, 2021 12:48:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-27T12:48:34.526Z: Worker configuration: e2-standard-2 in us-central1-a.
Jul 27, 2021 12:48:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-27T12:48:35.284Z: Expanding SplittableParDo operations into optimizable parts.
Jul 27, 2021 12:48:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-27T12:48:35.316Z: Expanding CollectionToSingleton operations into optimizable parts.
Jul 27, 2021 12:48:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-27T12:48:35.411Z: Expanding CoGroupByKey operations into optimizable parts.
Jul 27, 2021 12:48:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-27T12:48:35.484Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Jul 27, 2021 12:48:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-27T12:48:35.523Z: Expanding GroupByKey operations into streaming Read/Write steps
Jul 27, 2021 12:48:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-27T12:48:35.597Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jul 27, 2021 12:48:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-27T12:48:35.719Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jul 27, 2021 12:48:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-27T12:48:35.750Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Jul 27, 2021 12:48:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-27T12:48:35.789Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Jul 27, 2021 12:48:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-27T12:48:35.826Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Jul 27, 2021 12:48:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-27T12:48:35.860Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Jul 27, 2021 12:48:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-27T12:48:35.891Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jul 27, 2021 12:48:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-27T12:48:35.934Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jul 27, 2021 12:48:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-27T12:48:35.969Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jul 27, 2021 12:48:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-27T12:48:36.002Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Jul 27, 2021 12:48:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-27T12:48:36.035Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Jul 27, 2021 12:48:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-27T12:48:36.067Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Jul 27, 2021 12:48:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-27T12:48:36.137Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Jul 27, 2021 12:48:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-27T12:48:36.162Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jul 27, 2021 12:48:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-27T12:48:36.184Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jul 27, 2021 12:48:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-27T12:48:36.224Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jul 27, 2021 12:48:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-27T12:48:36.253Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Jul 27, 2021 12:48:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-27T12:48:36.273Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Jul 27, 2021 12:48:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-27T12:48:36.310Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Jul 27, 2021 12:48:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-27T12:48:36.349Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Jul 27, 2021 12:48:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-27T12:48:36.379Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Jul 27, 2021 12:48:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-27T12:48:36.418Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Jul 27, 2021 12:48:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-27T12:48:36.459Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Jul 27, 2021 12:48:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-27T12:48:36.498Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Jul 27, 2021 12:48:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-27T12:48:36.903Z: Starting 5 ****s in us-central1-a...
Jul 27, 2021 12:48:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-27T12:48:49.471Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jul 27, 2021 12:49:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-27T12:49:19.295Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Jul 27, 2021 12:50:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-27T12:50:11.354Z: Workers have started successfully.
Jul 27, 2021 12:50:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-27T12:50:11.394Z: Workers have started successfully.
Jul 27, 2021 3:54:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-27T15:54:37.206Z: Staged package aws-java-sdk-cloudwatch-1.11.974-Ub2yicrz1p27ushPndFcVBigc3SuIZ4G-Yaq45B1iUo.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/aws-java-sdk-cloudwatch-1.11.974-Ub2yicrz1p27ushPndFcVBigc3SuIZ4G-Yaq45B1iUo.jar' is inaccessible.
Jul 27, 2021 3:54:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-27T15:54:37.297Z: Staged package aws-java-sdk-core-1.11.974-s3lPhICKsq1xExGOyj03VQHRqOsuX5YUc-UUlnhiA64.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/aws-java-sdk-core-1.11.974-s3lPhICKsq1xExGOyj03VQHRqOsuX5YUc-UUlnhiA64.jar' is inaccessible.
Jul 27, 2021 3:54:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-27T15:54:37.356Z: Staged package aws-java-sdk-dynamodb-1.11.974-RLvIDrX-TDIDN06gNnaHm9dQbuHCD_XSBesOuniSTk4.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/aws-java-sdk-dynamodb-1.11.974-RLvIDrX-TDIDN06gNnaHm9dQbuHCD_XSBesOuniSTk4.jar' is inaccessible.
Jul 27, 2021 3:54:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-27T15:54:37.397Z: Staged package aws-java-sdk-kinesis-1.11.974-kbU_CjbRpkrhhFeoyIMQA81nTYmniJY77w8Smd7PLpY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/aws-java-sdk-kinesis-1.11.974-kbU_CjbRpkrhhFeoyIMQA81nTYmniJY77w8Smd7PLpY.jar' is inaccessible.
Jul 27, 2021 3:54:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-27T15:54:37.472Z: Staged package aws-java-sdk-kms-1.11.974-HWH_mHclw3Wv4_GJ56x62xFDHSPSkZWjeKAlx40XrTk.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/aws-java-sdk-kms-1.11.974-HWH_mHclw3Wv4_GJ56x62xFDHSPSkZWjeKAlx40XrTk.jar' is inaccessible.
Jul 27, 2021 3:54:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-27T15:54:37.513Z: Staged package aws-java-sdk-s3-1.11.974-Qgd0wgaOuI_uyxxQI4Ulljk9bZEq-3JZArQF0zU5yzA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/aws-java-sdk-s3-1.11.974-Qgd0wgaOuI_uyxxQI4Ulljk9bZEq-3JZArQF0zU5yzA.jar' is inaccessible.
Jul 27, 2021 3:54:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-27T15:54:38.286Z: Staged package commons-io-2.6--HfTBGYKwqFC84ZbrfyXHex-1zx0fH-NXS9ROcpzZRM.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/commons-io-2.6--HfTBGYKwqFC84ZbrfyXHex-1zx0fH-NXS9ROcpzZRM.jar' is inaccessible.
Jul 27, 2021 3:54:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-27T15:54:38.328Z: Staged package commons-lang-2.6-UPEbCfh3wpTVbyRGP0fSj5Kc9QRPZIZhwPDPuumi9Jw.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/commons-lang-2.6-UPEbCfh3wpTVbyRGP0fSj5Kc9QRPZIZhwPDPuumi9Jw.jar' is inaccessible.
Jul 27, 2021 3:54:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-27T15:54:39.720Z: Staged package ion-java-1.0.2-DRJ7IFofzgq8KjdXoEF0hlG8ZsFc9MBZusWDOyfUcaU.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/ion-java-1.0.2-DRJ7IFofzgq8KjdXoEF0hlG8ZsFc9MBZusWDOyfUcaU.jar' is inaccessible.
Jul 27, 2021 3:54:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-27T15:54:39.973Z: Staged package jmespath-java-1.11.974-QaUt5Hnw6wd7ULTQ5Oj4j_zpRr-rnMCdfGwiwkSo8-0.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/jmespath-java-1.11.974-QaUt5Hnw6wd7ULTQ5Oj4j_zpRr-rnMCdfGwiwkSo8-0.jar' is inaccessible.
Jul 27, 2021 3:54:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-07-27T15:54:41.043Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jul 27, 2021 3:57:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-07-27T15:57:41.349Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jul 27, 2021 4:00:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-27T16:00:34.666Z: Cancel request is committed for workflow job: 2021-07-27_05_48_19-11324485973340498397.
Jul 27, 2021 4:00:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-27T16:00:34.693Z: Cleaning up.
Jul 27, 2021 4:00:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-27T16:00:34.765Z: Stopping **** pool...
Jul 27, 2021 4:00:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-27T16:00:34.807Z: Stopping **** pool...
Jul 27, 2021 4:02:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-27T16:02:53.871Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Jul 27, 2021 4:02:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-27T16:02:53.916Z: Worker pool stopped.
Jul 27, 2021 4:03:00 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-07-27_05_48_19-11324485973340498397 finished with status CANCELLED.
Load test results for test (ID): e46e2d88-aebe-4ca2-bc3b-40e7cff0f419 and timestamp: 2021-07-27T12:48:13.340000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11388.972
dataflow_v2_java11_total_bytes_count             2.08686792E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210727124333
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:627cc128ed959dd39bc70f62f0b1f173dbd6b3cbf5c9ff7f346d342975e84600
Deleted: sha256:d76b72e82221434861e0ac2155d27dcfbdd10be027ec00c012d9ed7d5681f28d
Deleted: sha256:cddaef7f9f2fd4da7a1bb30ecbd657f4256816aa091f5a12ca64802482f929ac
Deleted: sha256:4c76c146932b006effef15625cbce4dc0d3b022918e58d30952fd89f332d7ab0
Deleted: sha256:b8dab4f447912db5e9a53eabebed44bfb487e0c0cfcc33f551bde8f17eda26ba
Deleted: sha256:67f04ecbd09562fc00ef34cf24241c5e9bafcccc084fbe137139f36036ba1cc5
Deleted: sha256:c555d0a69725a996cf517800e86d7714fea9f04bdd1e15d2a60c44a05460e780
Deleted: sha256:093d620248d4c38213c846af9e69234ed68969827e18520a6506b780a039e52c
Deleted: sha256:51327e2af48a6ea14ca8d27578cac417e602eb699d573801d2c670a5e24834ff
Deleted: sha256:9c34e355f7971c8ccb40da2f7327dd0484427bc1f057649445a9a6556d893164
Deleted: sha256:c5ce354148839f7d1b5843382932d5b33375d00e95d3a8b367eda001816ea3fa
Deleted: sha256:767c276872c0234cfd2007043142e7c89242b98d6150f51cc22ba2081bc8a8cc
Deleted: sha256:5d75cb09657f2d94c1516802115a1d68eb16b3de292eafc59389e67d7409af2f
Deleted: sha256:3cfae38c6e274e6cb3942f60200c260002e84960524119b781e5c2cda10d6c59
Deleted: sha256:496c3bb4c43863858f110dca590f1133d4036dec0930cd3ffeb7a187bd69c98a
Deleted: sha256:5ea75f9a44f67f4aa165688c1c4b1d15e7407426732cfea324bb1769959711da
Deleted: sha256:dd32fe9f5e38fa001b5d10dd0104ac60a3b947e19fc32d7ce01386ece7d3d3e9
Deleted: sha256:cb91b9c853e837bcb6a9843fa14b3745a5df2a2b2096ab776920fff97047b6fe
Deleted: sha256:334a430e4222ccdfd0486b7d0134206b11b42d3b7f3cf4dab1f2e72a211a8524
Deleted: sha256:bec8536a8b17d12bd79cd2e6637b75d01c2314b24253cdabb86abffb8695f881
Deleted: sha256:22aa05a294301799b2d78518837e26e9d727c9b6734e334f97928107b3a0e332
Deleted: sha256:0c32b55821e95f43d6c20808df6e1f8e6ebe47e37c227648fe32e19a4f5fc10e
Deleted: sha256:75a4776fb9cf944834d21ee2d295b0d03dc24cba5be578a9a833d1f2619926cb
Deleted: sha256:83ad183d83af04dc67fe8ef86b66c81fd9cd4ba83af4da9e26637852a9017a2b
Deleted: sha256:4e961ebbf0623bd026faa8e44c8af91b2e47492bd2511f2b9f4d1e1b0caa561c
Deleted: sha256:ae17509efe025e6999f680eecf251f28e80ed81ede9a48de9d41330b0f2b782c
Deleted: sha256:55d88347e708614cf2fa9bc5dee6e7661ef12c3562ccb1430db63528fd22694e
Deleted: sha256:b0ef5498164613138f4f7c4f923ece5ddc04713427cd7678e6e422d9cfef5c3a
Deleted: sha256:15bb05967c6b47d2721e6010d17728bfdd91805f6ec59da12e763c14f9c460cc
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:627cc128ed959dd39bc70f62f0b1f173dbd6b3cbf5c9ff7f346d342975e84600
  Associated tags:
 - 20210727124333
Tags:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210727124333
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210727124333].
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:627cc128ed959dd39bc70f62f0b1f173dbd6b3cbf5c9ff7f346d342975e84600].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 42s
107 actionable tasks: 76 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/e5theyhx73pak

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #39

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/39/display/redirect>

Changes:


------------------------------------------
[...truncated 24.68 KB...]
> Task :model:fn-execution:extractIncludeProto
> Task :model:job-management:generateProto FROM-CACHE
> Task :model:fn-execution:generateProto FROM-CACHE

> Task :release:go-licenses:java:docker
 ---> Running in fa02840b564b

> Task :model:job-management:compileJava FROM-CACHE
> Task :model:job-management:classes
> Task :model:fn-execution:compileJava FROM-CACHE
> Task :model:fn-execution:classes
> Task :model:job-management:shadowJar FROM-CACHE
> Task :model:fn-execution:shadowJar FROM-CACHE
> Task :sdks:java:core:compileJava FROM-CACHE
> Task :sdks:java:core:classes
> Task :sdks:java:core:shadowJar FROM-CACHE
> Task :sdks:java:container:generateLicenseReport
> Task :sdks:java:io:synthetic:compileJava FROM-CACHE
> Task :sdks:java:io:synthetic:classes UP-TO-DATE
> Task :sdks:java:testing:test-utils:compileJava FROM-CACHE
> Task :sdks:java:testing:test-utils:classes UP-TO-DATE
> Task :sdks:java:extensions:arrow:compileJava FROM-CACHE
> Task :sdks:java:io:kinesis:compileJava FROM-CACHE
> Task :sdks:java:extensions:arrow:classes UP-TO-DATE
> Task :sdks:java:io:kinesis:classes UP-TO-DATE
> Task :sdks:java:io:synthetic:jar
> Task :sdks:java:extensions:protobuf:extractIncludeProto
> Task :sdks:java:extensions:protobuf:generateProto NO-SOURCE
> Task :sdks:java:testing:test-utils:jar
> Task :sdks:java:extensions:arrow:jar
> Task :sdks:java:io:kinesis:jar
> Task :sdks:java:extensions:protobuf:compileJava FROM-CACHE
> Task :sdks:java:extensions:protobuf:classes UP-TO-DATE
> Task :sdks:java:extensions:protobuf:jar
> Task :sdks:java:fn-execution:compileJava FROM-CACHE
> Task :sdks:java:fn-execution:classes UP-TO-DATE
> Task :sdks:java:fn-execution:jar
> Task :runners:core-construction-java:compileJava FROM-CACHE
> Task :runners:core-construction-java:classes UP-TO-DATE
> Task :runners:core-construction-java:jar
> Task :runners:core-java:compileJava FROM-CACHE
> Task :runners:core-java:classes UP-TO-DATE
> Task :runners:core-java:jar
> Task :sdks:java:extensions:google-cloud-platform-core:compileJava FROM-CACHE
> Task :sdks:java:extensions:google-cloud-platform-core:classes UP-TO-DATE
> Task :sdks:java:extensions:google-cloud-platform-core:jar
> Task :sdks:java:harness:compileJava FROM-CACHE
> Task :sdks:java:harness:classes UP-TO-DATE
> Task :sdks:java:harness:jar

> Task :sdks:java:container:pullLicenses
Copying already-fetched licenses from <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src/sdks/java/container/build/reports/dependency-license> to <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src/sdks/java/container/build/target/java_third_party_licenses>
Already using interpreter /usr/bin/python3
Using base prefix '/usr'
New python executable in <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src/sdks/java/container/build/virtualenv/bin/python3>
Also creating executable in <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src/sdks/java/container/build/virtualenv/bin/python>
Installing setuptools, pip, wheel...

> Task :sdks:java:harness:shadowJar FROM-CACHE

> Task :sdks:go:resolveBuildDependencies
Resolving cloud.google.com/go: commit='03869a08dc16b35ad4968e92d34c5a2a2961b205', urls=[https://code.googlesource.com/gocloud]
Resolving github.com/Shopify/sarama: commit='541689b9f4212043471eb537fa72da507025d3ea', urls=[https://github.com/Shopify/sarama.git, git@github.com:Shopify/sarama.git]
Resolving github.com/armon/consul-api: commit='eb2c6b5be1b66bab83016e0b05f01b8d5496ffbd', urls=[https://github.com/armon/consul-api.git, git@github.com:armon/consul-api.git]
Resolving github.com/etcd-io/etcd: commit='11214aa33bf5a47d3d9d8dafe0f6b97237dfe921', urls=[https://github.com/etcd-io/etcd.git, git@github.com:etcd-io/etcd.git]
Resolving cached github.com/etcd-io/etcd: commit='11214aa33bf5a47d3d9d8dafe0f6b97237dfe921', urls=[https://github.com/etcd-io/etcd.git, git@github.com:etcd-io/etcd.git]
Resolving cached github.com/etcd-io/etcd: commit='11214aa33bf5a47d3d9d8dafe0f6b97237dfe921', urls=[https://github.com/etcd-io/etcd.git, git@github.com:etcd-io/etcd.git]
Resolving cached github.com/etcd-io/etcd: commit='11214aa33bf5a47d3d9d8dafe0f6b97237dfe921', urls=[https://github.com/etcd-io/etcd.git, git@github.com:etcd-io/etcd.git]
Resolving cached github.com/etcd-io/etcd: commit='11214aa33bf5a47d3d9d8dafe0f6b97237dfe921', urls=[https://github.com/etcd-io/etcd.git, git@github.com:etcd-io/etcd.git]
Resolving cached github.com/etcd-io/etcd: commit='11214aa33bf5a47d3d9d8dafe0f6b97237dfe921', urls=[https://github.com/etcd-io/etcd.git, git@github.com:etcd-io/etcd.git]
Resolving github.com/cpuguy83/go-md2man: commit='dc9f53734905c233adfc09fd4f063dce63ce3daf', urls=[https://github.com/cpuguy83/go-md2man.git, git@github.com:cpuguy83/go-md2man.git]
Resolving github.com/davecgh/go-spew: commit='87df7c60d5820d0f8ae11afede5aa52325c09717', urls=[https://github.com/davecgh/go-spew.git, git@github.com:davecgh/go-spew.git]
Resolving cached github.com/etcd-io/etcd: commit='11214aa33bf5a47d3d9d8dafe0f6b97237dfe921', urls=[https://github.com/etcd-io/etcd.git, git@github.com:etcd-io/etcd.git]
Resolving cached github.com/etcd-io/etcd: commit='11214aa33bf5a47d3d9d8dafe0f6b97237dfe921', urls=[https://github.com/etcd-io/etcd.git, git@github.com:etcd-io/etcd.git]
Resolving github.com/eapache/go-resiliency: commit='ef9aaa7ea8bd2448429af1a77cf41b2b3b34bdd6', urls=[https://github.com/eapache/go-resiliency.git, git@github.com:eapache/go-resiliency.git]
Resolving github.com/eapache/go-xerial-snappy: commit='bb955e01b9346ac19dc29eb16586c90ded99a98c', urls=[https://github.com/eapache/go-xerial-snappy.git, git@github.com:eapache/go-xerial-snappy.git]
Resolving github.com/eapache/queue: commit='44cc805cf13205b55f69e14bcb69867d1ae92f98', urls=[https://github.com/eapache/queue.git, git@github.com:eapache/queue.git]
Resolving github.com/etcd-io/etcd: commit='11214aa33bf5a47d3d9d8dafe0f6b97237dfe921', urls=[https://github.com/etcd-io/etcd.git, git@github.com:etcd-io/etcd.git]
Resolving github.com/fsnotify/fsnotify: commit='c2828203cd70a50dcccfb2761f8b1f8ceef9a8e9', urls=[https://github.com/fsnotify/fsnotify.git, git@github.com:fsnotify/fsnotify.git]
Resolving cached github.com/etcd-io/etcd: commit='11214aa33bf5a47d3d9d8dafe0f6b97237dfe921', urls=[https://github.com/etcd-io/etcd.git, git@github.com:etcd-io/etcd.git]
Resolving cached github.com/etcd-io/etcd: commit='11214aa33bf5a47d3d9d8dafe0f6b97237dfe921', urls=[https://github.com/etcd-io/etcd.git, git@github.com:etcd-io/etcd.git]
Resolving github.com/golang/glog: commit='23def4e6c14b4da8ac2ed8007337bc5eb5007998', urls=[https://github.com/golang/glog.git, git@github.com:golang/glog.git]
Resolving cached github.com/etcd-io/etcd: commit='11214aa33bf5a47d3d9d8dafe0f6b97237dfe921', urls=[https://github.com/etcd-io/etcd.git, git@github.com:etcd-io/etcd.git]
Resolving github.com/golang/mock: commit='b3e60bcdc577185fce3cf625fc96b62857ce5574', urls=[https://github.com/golang/mock.git, git@github.com:golang/mock.git]
Resolving github.com/golang/protobuf: commit='d04d7b157bb510b1e0c10132224b616ac0e26b17', urls=[https://github.com/golang/protobuf.git, git@github.com:golang/protobuf.git]
Resolving github.com/golang/snappy: commit='553a641470496b2327abcac10b36396bd98e45c9', urls=[https://github.com/golang/snappy.git, git@github.com:golang/snappy.git]
Resolving cached github.com/etcd-io/etcd: commit='11214aa33bf5a47d3d9d8dafe0f6b97237dfe921', urls=[https://github.com/etcd-io/etcd.git, git@github.com:etcd-io/etcd.git]
Resolving github.com/google/go-cmp: commit='9680bfaf28748393e28e00238d94070fb9972fd8', urls=[https://github.com/google/go-cmp.git, git@github.com:google/go-cmp.git]
Resolving github.com/google/pprof: commit='a8f279b7952b27edbcb72e5a6c69ee9be4c8ad93', urls=[https://github.com/google/pprof.git, git@github.com:google/pprof.git]
Resolving github.com/google/uuid: commit='bfb86fa49a73e4194d93bea18d7acfe3694438ce', urls=[https://github.com/google/uuid.git, git@github.com:google/uuid.git]
Resolving github.com/googleapis/gax-go: commit='bd5b16380fd03dc758d11cef74ba2e3bc8b0e8c2', urls=[https://github.com/googleapis/gax-go.git, git@github.com:googleapis/gax-go.git]
Resolving cached github.com/etcd-io/etcd: commit='11214aa33bf5a47d3d9d8dafe0f6b97237dfe921', urls=[https://github.com/etcd-io/etcd.git, git@github.com:etcd-io/etcd.git]
Resolving cached github.com/etcd-io/etcd: commit='11214aa33bf5a47d3d9d8dafe0f6b97237dfe921', urls=[https://github.com/etcd-io/etcd.git, git@github.com:etcd-io/etcd.git]
Resolving cached github.com/etcd-io/etcd: commit='11214aa33bf5a47d3d9d8dafe0f6b97237dfe921', urls=[https://github.com/etcd-io/etcd.git, git@github.com:etcd-io/etcd.git]
Resolving github.com/hashicorp/hcl: commit='23c074d0eceb2b8a5bfdbb271ab780cde70f05a8', urls=[https://github.com/hashicorp/hcl.git, git@github.com:hashicorp/hcl.git]
Resolving github.com/ianlancetaylor/demangle: commit='4883227f66371e02c4948937d3e2be1664d9be38', urls=[https://github.com/ianlancetaylor/demangle.git, git@github.com:ianlancetaylor/demangle.git]
Resolving cached github.com/etcd-io/etcd: commit='11214aa33bf5a47d3d9d8dafe0f6b97237dfe921', urls=[https://github.com/etcd-io/etcd.git, git@github.com:etcd-io/etcd.git]
Resolving cached github.com/etcd-io/etcd: commit='11214aa33bf5a47d3d9d8dafe0f6b97237dfe921', urls=[https://github.com/etcd-io/etcd.git, git@github.com:etcd-io/etcd.git]
Resolving github.com/kr/fs: commit='2788f0dbd16903de03cb8186e5c7d97b69ad387b', urls=[https://github.com/kr/fs.git, git@github.com:kr/fs.git]
Resolving cached github.com/etcd-io/etcd: commit='11214aa33bf5a47d3d9d8dafe0f6b97237dfe921', urls=[https://github.com/etcd-io/etcd.git, git@github.com:etcd-io/etcd.git]

> Task :runners:java-fn-execution:compileJava FROM-CACHE
> Task :runners:java-fn-execution:classes UP-TO-DATE
> Task :runners:java-fn-execution:jar

> Task :release:go-licenses:java:docker
package embed: unrecognized import path "embed": import path does not begin with hostname
package io/fs: unrecognized import path "io/fs": import path does not begin with hostname

> Task :sdks:java:expansion-service:compileJava FROM-CACHE
> Task :sdks:java:expansion-service:classes UP-TO-DATE
> Task :sdks:java:expansion-service:jar
> Task :sdks:java:io:kafka:compileJava FROM-CACHE
> Task :sdks:java:io:kafka:classes UP-TO-DATE

> Task :sdks:java:container:pullLicenses
done.
DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality.
Collecting beautifulsoup4<5.0,>=4.9.0
  Using cached beautifulsoup4-4.9.3-py3-none-any.whl (115 kB)
Collecting future<1.0.0,>=0.16.0
  Using cached future-0.18.2-py3-none-any.whl

> Task :sdks:java:io:kafka:jar
> Task :sdks:java:io:google-cloud-platform:compileJava FROM-CACHE
> Task :sdks:java:io:google-cloud-platform:classes UP-TO-DATE
> Task :sdks:java:io:google-cloud-platform:jar
> Task :sdks:java:testing:load-tests:compileJava FROM-CACHE
> Task :sdks:java:testing:load-tests:classes UP-TO-DATE
> Task :sdks:java:testing:load-tests:jar
> Task :runners:google-cloud-dataflow-java:compileJava FROM-CACHE
> Task :runners:google-cloud-dataflow-java:classes
> Task :runners:google-cloud-dataflow-java:jar
> Task :sdks:java:container:java11:copyDockerfileDependencies

> Task :sdks:go:resolveBuildDependencies
Resolving github.com/linkedin/goavro: commit='94a7a9db615f35a39dd2b82089398b92fabad2ba', urls=[https://github.com/linkedin/goavro.git, git@github.com:linkedin/goavro.git]
Resolving github.com/magiconair/properties: commit='49d762b9817ba1c2e9d0c69183c2b4a8b8f1d934', urls=[https://github.com/magiconair/properties.git, git@github.com:magiconair/properties.git]
Resolving cached github.com/etcd-io/etcd: commit='11214aa33bf5a47d3d9d8dafe0f6b97237dfe921', urls=[https://github.com/etcd-io/etcd.git, git@github.com:etcd-io/etcd.git]
Resolving cached github.com/etcd-io/etcd: commit='11214aa33bf5a47d3d9d8dafe0f6b97237dfe921', urls=[https://github.com/etcd-io/etcd.git, git@github.com:etcd-io/etcd.git]
Resolving github.com/mitchellh/go-homedir: commit='b8bc1bf767474819792c23f32d8286a45736f1c6', urls=[https://github.com/mitchellh/go-homedir.git, git@github.com:mitchellh/go-homedir.git]
Resolving github.com/mitchellh/mapstructure: commit='a4e142e9c047c904fa2f1e144d9a84e6133024bc', urls=[https://github.com/mitchellh/mapstructure.git, git@github.com:mitchellh/mapstructure.git]
Resolving github.com/nightlyone/lockfile: commit='0ad87eef1443f64d3d8c50da647e2b1552851124', urls=[https://github.com/nightlyone/lockfile, git@github.com:nightlyone/lockfile.git]
Resolving cached github.com/etcd-io/etcd: commit='11214aa33bf5a47d3d9d8dafe0f6b97237dfe921', urls=[https://github.com/etcd-io/etcd.git, git@github.com:etcd-io/etcd.git]
Resolving github.com/openzipkin/zipkin-go: commit='3741243b287094fda649c7f0fa74bd51f37dc122', urls=[https://github.com/openzipkin/zipkin-go.git, git@github.com:openzipkin/zipkin-go.git]
Resolving github.com/pelletier/go-toml: commit='acdc4509485b587f5e675510c4f2c63e90ff68a8', urls=[https://github.com/pelletier/go-toml.git, git@github.com:pelletier/go-toml.git]
Resolving cached github.com/etcd-io/etcd: commit='11214aa33bf5a47d3d9d8dafe0f6b97237dfe921', urls=[https://github.com/etcd-io/etcd.git, git@github.com:etcd-io/etcd.git]
Resolving github.com/pierrec/lz4: commit='ed8d4cc3b461464e69798080a0092bd028910298', urls=[https://github.com/pierrec/lz4.git, git@github.com:pierrec/lz4.git]
Resolving github.com/pierrec/xxHash: commit='a0006b13c722f7f12368c00a3d3c2ae8a999a0c6', urls=[https://github.com/pierrec/xxHash.git, git@github.com:pierrec/xxHash.git]
Resolving github.com/pkg/errors: commit='30136e27e2ac8d167177e8a583aa4c3fea5be833', urls=[https://github.com/pkg/errors.git, git@github.com:pkg/errors.git]
Resolving github.com/pkg/sftp: commit='22e9c1ccc02fc1b9fa3264572e49109b68a86947', urls=[https://github.com/pkg/sftp.git, git@github.com:pkg/sftp.git]
Resolving github.com/prometheus/client_golang: commit='9bb6ab929dcbe1c8393cd9ef70387cb69811bd1c', urls=[https://github.com/prometheus/client_golang.git, git@github.com:prometheus/client_golang.git]
Resolving cached github.com/etcd-io/etcd: commit='11214aa33bf5a47d3d9d8dafe0f6b97237dfe921', urls=[https://github.com/etcd-io/etcd.git, git@github.com:etcd-io/etcd.git]
Resolving cached github.com/etcd-io/etcd: commit='11214aa33bf5a47d3d9d8dafe0f6b97237dfe921', urls=[https://github.com/etcd-io/etcd.git, git@github.com:etcd-io/etcd.git]
Resolving github.com/prometheus/procfs: commit='cb4147076ac75738c9a7d279075a253c0cc5acbd', urls=[https://github.com/prometheus/procfs.git, git@github.com:prometheus/procfs.git]
Resolving github.com/rcrowley/go-metrics: commit='8732c616f52954686704c8645fe1a9d59e9df7c1', urls=[https://github.com/rcrowley/go-metrics.git, git@github.com:rcrowley/go-metrics.git]
Resolving github.com/cpuguy83/go-md2man: commit='dc9f53734905c233adfc09fd4f063dce63ce3daf', urls=[https://github.com/cpuguy83/go-md2man.git, git@github.com:cpuguy83/go-md2man.git]
Resolving cached github.com/cpuguy83/go-md2man: commit='dc9f53734905c233adfc09fd4f063dce63ce3daf', urls=[https://github.com/cpuguy83/go-md2man.git, git@github.com:cpuguy83/go-md2man.git]
Resolving cached github.com/etcd-io/etcd: commit='11214aa33bf5a47d3d9d8dafe0f6b97237dfe921', urls=[https://github.com/etcd-io/etcd.git, git@github.com:etcd-io/etcd.git]
Resolving cached github.com/etcd-io/etcd: commit='11214aa33bf5a47d3d9d8dafe0f6b97237dfe921', urls=[https://github.com/etcd-io/etcd.git, git@github.com:etcd-io/etcd.git]
Resolving github.com/spf13/afero: commit='bb8f1927f2a9d3ab41c9340aa034f6b803f4359c', urls=[https://github.com/spf13/afero.git, git@github.com:spf13/afero.git]
Resolving github.com/spf13/cast: commit='acbeb36b902d72a7a4c18e8f3241075e7ab763e4', urls=[https://github.com/spf13/cast.git, git@github.com:spf13/cast.git]
Resolving github.com/spf13/cobra: commit='93959269ad99e80983c9ba742a7e01203a4c0e4f', urls=[https://github.com/spf13/cobra.git, git@github.com:spf13/cobra.git]
Resolving github.com/spf13/jwalterweatherman: commit='7c0cea34c8ece3fbeb2b27ab9b59511d360fb394', urls=[https://github.com/spf13/jwalterweatherman.git, git@github.com:spf13/jwalterweatherman.git]
Resolving cached github.com/etcd-io/etcd: commit='11214aa33bf5a47d3d9d8dafe0f6b97237dfe921', urls=[https://github.com/etcd-io/etcd.git, git@github.com:etcd-io/etcd.git]
Resolving github.com/spf13/viper: commit='aafc9e6bc7b7bb53ddaa75a5ef49a17d6e654be5', urls=[https://github.com/spf13/viper.git, git@github.com:spf13/viper.git]
Resolving github.com/stathat/go: commit='74669b9f388d9d788c97399a0824adbfee78400e', urls=[https://github.com/stathat/go.git, git@github.com:stathat/go.git]
Resolving cached github.com/etcd-io/etcd: commit='11214aa33bf5a47d3d9d8dafe0f6b97237dfe921', urls=[https://github.com/etcd-io/etcd.git, git@github.com:etcd-io/etcd.git]
Resolving cached github.com/etcd-io/etcd: commit='11214aa33bf5a47d3d9d8dafe0f6b97237dfe921', urls=[https://github.com/etcd-io/etcd.git, git@github.com:etcd-io/etcd.git]
Resolving cached github.com/etcd-io/etcd: commit='11214aa33bf5a47d3d9d8dafe0f6b97237dfe921', urls=[https://github.com/etcd-io/etcd.git, git@github.com:etcd-io/etcd.git]
Resolving cached github.com/etcd-io/etcd: commit='11214aa33bf5a47d3d9d8dafe0f6b97237dfe921', urls=[https://github.com/etcd-io/etcd.git, git@github.com:etcd-io/etcd.git]
Resolving github.com/xordataexchange/crypt: commit='b2862e3d0a775f18c7cfe02273500ae307b61218', urls=[https://github.com/xordataexchange/crypt.git, git@github.com:xordataexchange/crypt.git]
Resolving go.opencensus.io: commit='3fb168f674736c026e623310bfccb0691e6dec8a', urls=[https://github.com/census-instrumentation/opencensus-go]
Resolving golang.org/x/crypto: commit='d9133f5469342136e669e85192a26056b587f503', urls=[https://go.googlesource.com/crypto]
Resolving golang.org/x/debug: commit='95515998a8a4bd7448134b2cb5971dbeb12e0b77', urls=[https://go.googlesource.com/debug]

> Task :sdks:java:container:pullLicenses
Collecting pyyaml<6.0.0,>=3.12
  Using cached PyYAML-5.3.1-cp35-cp35m-linux_x86_64.whl
Collecting tenacity<6.0,>=5.0.2
  Using cached tenacity-5.1.5-py2.py3-none-any.whl (34 kB)
Collecting soupsieve>1.2
  Using cached soupsieve-2.1-py3-none-any.whl (32 kB)
Collecting six>=1.9.0
  Using cached six-1.16.0-py2.py3-none-any.whl (11 kB)
Installing collected packages: soupsieve, six, tenacity, pyyaml, future, beautifulsoup4
Successfully installed beautifulsoup4-4.9.3 future-0.18.2 pyyaml-5.3.1 six-1.16.0 soupsieve-2.1 tenacity-5.1.5
Executing <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src/sdks/java/container/build/virtualenv/bin/python> <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src/sdks/java/container/license_scripts/pull_licenses_java.py> --license_index=<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src/sdks/java/container/build/reports/dependency-license/index.json>        --output_dir=<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src/sdks/java/container/build/target/java_third_party_licenses>        --dep_url_yaml=<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src/sdks/java/container/license_scripts/dep_urls_java.yaml> 
INFO:root:Pulling license for 207 dependencies using 16 threads.
INFO:root:pull_licenses_java.py succeed. It took 1.791423 seconds with 16 threads.
Copying licenses from <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src/sdks/java/container/build/target/java_third_party_licenses> to <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src/sdks/java/container/build/target/third_party_licenses.>
Finished license_scripts.sh

> Task :sdks:java:container:java11:copyJavaThirdPartyLicenses NO-SOURCE

> Task :sdks:go:resolveBuildDependencies
Resolving golang.org/x/net: commit='2fb46b16b8dda405028c50f7c7f0f9dd1fa6bfb1', urls=[https://go.googlesource.com/net]
Resolving golang.org/x/oauth2: commit='5d25da1a8d43b66f2898c444f899c7bcfd6a407e', urls=[https://go.googlesource.com/oauth2]
Resolving golang.org/x/sync: commit='fd80eb99c8f653c847d294a001bdf2a3a6f768f5', urls=[https://go.googlesource.com/sync]
Resolving golang.org/x/sys: commit='fde4db37ae7ad8191b03d30d27f258b5291ae4e3', urls=[https://go.googlesource.com/sys]

> Task :release:go-licenses:java:docker FAILED
The command '/bin/sh -c go get github.com/google/go-licenses' returned a non-zero code: 1

> Task :sdks:go:resolveBuildDependencies
Resolving golang.org/x/text: commit='23ae387dee1f90d29a23c0e87ee0b46038fbed0e', urls=[https://go.googlesource.com/text]
Resolving cached github.com/etcd-io/etcd: commit='11214aa33bf5a47d3d9d8dafe0f6b97237dfe921', urls=[https://github.com/etcd-io/etcd.git, git@github.com:etcd-io/etcd.git]
Resolving google.golang.org/api: commit='0324d5e90dc7753607860272666845fad9ceb97e', urls=[https://code.googlesource.com/google-api-go-client]
Resolving google.golang.org/genproto: commit='4d944d34d83c502a5f761500a14d8842648415c3', urls=[https://github.com/google/go-genproto]
Resolving google.golang.org/grpc: commit='5e8f83304c0563d1ba74db05fee83d9c18ab9a58', urls=[https://github.com/grpc/grpc-go]
Resolving google.golang.org/protobuf: commit='d165be301fb1e13390ad453281ded24385fd8ebc', urls=[https://go.googlesource.com/protobuf]
Resolving cached github.com/etcd-io/etcd: commit='11214aa33bf5a47d3d9d8dafe0f6b97237dfe921', urls=[https://github.com/etcd-io/etcd.git, git@github.com:etcd-io/etcd.git]
Resolving cached github.com/etcd-io/etcd: commit='11214aa33bf5a47d3d9d8dafe0f6b97237dfe921', urls=[https://github.com/etcd-io/etcd.git, git@github.com:etcd-io/etcd.git]

> Task :sdks:go:installDependencies
> Task :sdks:go:buildLinuxAmd64
> Task :sdks:go:goBuild

> Task :sdks:java:container:resolveBuildDependencies
Resolving ./github.com/apache/beam/sdks/go@<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src/sdks/go>

> Task :sdks:java:container:installDependencies
> Task :sdks:java:container:buildLinuxAmd64
> Task :sdks:java:container:goBuild
> Task :sdks:java:container:java11:copySdkHarnessLauncher

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':release:go-licenses:java:docker'.
> Process 'command 'docker'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 4m 18s
100 actionable tasks: 69 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/ub6r53g5mcls6

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #38

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/38/display/redirect>

Changes:


------------------------------------------
[...truncated 65.77 KB...]
d1663742fc42: Preparing
b5846832c46f: Preparing
15825d62a048: Preparing
351ee27efeaa: Preparing
6b54a67bba71: Preparing
1bb87576a5a8: Preparing
ce3b203a940f: Preparing
efd581501a69: Preparing
f42aed5f7feb: Preparing
89819bafde36: Preparing
f3d5b8f65132: Preparing
ad83f0aa5c0a: Preparing
5a9a65095453: Preparing
4b0edb23340c: Preparing
afa3e488a0ee: Preparing
b5846832c46f: Waiting
6b54a67bba71: Waiting
1bb87576a5a8: Waiting
15825d62a048: Waiting
351ee27efeaa: Waiting
ce3b203a940f: Waiting
f42aed5f7feb: Waiting
4b0edb23340c: Waiting
afa3e488a0ee: Waiting
89819bafde36: Waiting
ad83f0aa5c0a: Waiting
f3d5b8f65132: Waiting
5a9a65095453: Waiting
efd581501a69: Waiting
34a3e4d7f142: Pushed
d1663742fc42: Pushed
214bba85d905: Pushed
f68943217278: Pushed
b5846832c46f: Pushed
febcfe5f8d23: Pushed
351ee27efeaa: Pushed
6b54a67bba71: Pushed
f42aed5f7feb: Layer already exists
89819bafde36: Layer already exists
f3d5b8f65132: Layer already exists
ad83f0aa5c0a: Layer already exists
ce3b203a940f: Pushed
5a9a65095453: Layer already exists
efd581501a69: Pushed
4b0edb23340c: Layer already exists
afa3e488a0ee: Layer already exists
15825d62a048: Pushed
1bb87576a5a8: Pushed
20210725124332: digest: sha256:0ab47272970dcbe95abaa4fea9947088ca43f788520782ff0cbbcdbc5b17489f size: 4310

> Task :sdks:java:testing:load-tests:run
Jul 25, 2021 12:48:29 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Jul 25, 2021 12:48:30 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 191 files. Enable logging at DEBUG level to see which files will be staged.
Jul 25, 2021 12:48:30 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Jul 25, 2021 12:48:30 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Jul 25, 2021 12:48:33 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Jul 25, 2021 12:48:33 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <112637 bytes, hash e28ab6fde1e6b2ce0374d36bc24158d8c5e697bccf57533a23a291b68902de50> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-4oq2_eHmss4DdNNrwkFY2MXml7zPV1M6I6KRtokC3lA.pb
Jul 25, 2021 12:48:34 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 191 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Jul 25, 2021 12:48:35 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 191 files cached, 0 files newly uploaded in 0 seconds
Jul 25, 2021 12:48:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Jul 25, 2021 12:48:35 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@75de6341, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@74170687, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@68f0f72c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3d96fa9e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3b545206, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77bb48d5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@181d8899, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12d5c30e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@b887730, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26586b74, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@52f57666, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6e041285, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@267dc982, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@439b15f2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3aa41da1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@74fab04a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c83468e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@272ce069, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3d7fb838, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1c26273d]
Jul 25, 2021 12:48:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Jul 25, 2021 12:48:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Jul 25, 2021 12:48:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Jul 25, 2021 12:48:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Jul 25, 2021 12:48:35 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2abe9173, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@235d29d6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1fdca564, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43f9dd56, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1d12e953, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@57cb70be, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2d4608a6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@20d87335, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2a8a4e0c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26c89563, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3bd6ba24, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58f437b0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@20f6f88c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4277127c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4c7e978c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@354e7004, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@22ebccb9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5875de6a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@608fe01f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2687725a]
Jul 25, 2021 12:48:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Jul 25, 2021 12:48:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Jul 25, 2021 12:48:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Jul 25, 2021 12:48:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Jul 25, 2021 12:48:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Jul 25, 2021 12:48:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Jul 25, 2021 12:48:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Jul 25, 2021 12:48:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Jul 25, 2021 12:48:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Jul 25, 2021 12:48:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Jul 25, 2021 12:48:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Jul 25, 2021 12:48:35 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.33.0-SNAPSHOT
Jul 25, 2021 12:48:36 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-07-25_05_48_35-6223462627856517545?project=apache-beam-testing
Jul 25, 2021 12:48:36 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-07-25_05_48_35-6223462627856517545
Jul 25, 2021 12:48:36 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-07-25_05_48_35-6223462627856517545
Jul 25, 2021 12:48:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-07-25T12:48:42.673Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-07-5ho8. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Jul 25, 2021 12:48:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-25T12:48:47.679Z: Worker configuration: e2-standard-2 in us-central1-a.
Jul 25, 2021 12:48:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-25T12:48:48.342Z: Expanding SplittableParDo operations into optimizable parts.
Jul 25, 2021 12:48:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-25T12:48:48.375Z: Expanding CollectionToSingleton operations into optimizable parts.
Jul 25, 2021 12:48:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-25T12:48:48.439Z: Expanding CoGroupByKey operations into optimizable parts.
Jul 25, 2021 12:48:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-25T12:48:48.510Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Jul 25, 2021 12:48:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-25T12:48:48.541Z: Expanding GroupByKey operations into streaming Read/Write steps
Jul 25, 2021 12:48:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-25T12:48:48.590Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jul 25, 2021 12:48:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-25T12:48:48.699Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jul 25, 2021 12:48:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-25T12:48:48.728Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Jul 25, 2021 12:48:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-25T12:48:48.761Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Jul 25, 2021 12:48:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-25T12:48:48.795Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Jul 25, 2021 12:48:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-25T12:48:48.840Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Jul 25, 2021 12:48:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-25T12:48:48.874Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jul 25, 2021 12:48:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-25T12:48:48.909Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jul 25, 2021 12:48:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-25T12:48:48.940Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jul 25, 2021 12:48:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-25T12:48:48.963Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Jul 25, 2021 12:48:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-25T12:48:48.999Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Jul 25, 2021 12:48:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-25T12:48:49.032Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Jul 25, 2021 12:48:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-25T12:48:49.057Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Jul 25, 2021 12:48:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-25T12:48:49.083Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jul 25, 2021 12:48:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-25T12:48:49.114Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jul 25, 2021 12:48:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-25T12:48:49.152Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jul 25, 2021 12:48:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-25T12:48:49.180Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Jul 25, 2021 12:48:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-25T12:48:49.204Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Jul 25, 2021 12:48:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-25T12:48:49.226Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Jul 25, 2021 12:48:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-25T12:48:49.250Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Jul 25, 2021 12:48:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-25T12:48:49.272Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Jul 25, 2021 12:48:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-25T12:48:49.292Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Jul 25, 2021 12:48:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-25T12:48:49.322Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Jul 25, 2021 12:48:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-25T12:48:49.379Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Jul 25, 2021 12:48:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-25T12:48:49.740Z: Starting 5 ****s in us-central1-a...
Jul 25, 2021 12:48:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-25T12:48:57.698Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jul 25, 2021 12:49:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-25T12:49:25.548Z: Autoscaling: Raised the number of ****s to 2 so that the pipeline can catch up with its backlog and keep up with its input rate.
Jul 25, 2021 12:49:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-25T12:49:25.590Z: Resized **** pool to 2, though goal was 5.  This could be a quota issue.
Jul 25, 2021 12:49:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-25T12:49:35.854Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Jul 25, 2021 12:50:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-25T12:50:30.575Z: Workers have started successfully.
Jul 25, 2021 12:50:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-25T12:50:30.607Z: Workers have started successfully.
Jul 25, 2021 4:00:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-25T16:00:47.643Z: Cancel request is committed for workflow job: 2021-07-25_05_48_35-6223462627856517545.
Jul 25, 2021 4:00:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-25T16:00:47.722Z: Cleaning up.
Jul 25, 2021 4:00:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-25T16:00:47.798Z: Stopping **** pool...
Jul 25, 2021 4:00:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-25T16:00:47.844Z: Stopping **** pool...
Jul 25, 2021 4:03:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-25T16:03:04.710Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Jul 25, 2021 4:03:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-25T16:03:04.745Z: Worker pool stopped.
Jul 25, 2021 4:03:10 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-07-25_05_48_35-6223462627856517545 finished with status CANCELLED.
Load test results for test (ID): 20a5bafa-54bf-417d-9558-d42ab1967c0e and timestamp: 2021-07-25T12:48:30.465000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                  11384.88
dataflow_v2_java11_total_bytes_count             2.62775332E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210725124332
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:0ab47272970dcbe95abaa4fea9947088ca43f788520782ff0cbbcdbc5b17489f
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:0ab47272970dcbe95abaa4fea9947088ca43f788520782ff0cbbcdbc5b17489f
  Associated tags:
 - 20210725124332
Tags:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210725124332
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210725124332].
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:0ab47272970dcbe95abaa4fea9947088ca43f788520782ff0cbbcdbc5b17489f].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 55s
107 actionable tasks: 76 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/vq2uxhw2crwdq

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #37

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/37/display/redirect?page=changes>

Changes:

[noreply] [BEAM-12652] revving JS filename to bust cache (#15209)

[noreply] [BEAM-12399] Move CPython license to own file. (#15201)

[noreply] Revert "Default to Runner v2 for Python Streaming jobs. (#15140)"


------------------------------------------
[...truncated 65.80 KB...]
ac3b0cecf471: Preparing
33db017afd3a: Preparing
2874d6784b86: Preparing
f41e8728296d: Preparing
c8d212339350: Preparing
061ab49f48fb: Preparing
1dd7188f6d8c: Preparing
f42aed5f7feb: Preparing
89819bafde36: Preparing
f3d5b8f65132: Preparing
ad83f0aa5c0a: Preparing
5a9a65095453: Preparing
4b0edb23340c: Preparing
c8d212339350: Waiting
061ab49f48fb: Waiting
f42aed5f7feb: Waiting
afa3e488a0ee: Preparing
89819bafde36: Waiting
2874d6784b86: Waiting
33db017afd3a: Waiting
5a9a65095453: Waiting
ac3b0cecf471: Waiting
4b0edb23340c: Waiting
f41e8728296d: Waiting
ad83f0aa5c0a: Waiting
612afc6ceb19: Pushed
738c9f46f4aa: Pushed
a3c153287b80: Pushed
ac3b0cecf471: Pushed
3cf95861a689: Pushed
1f1662d433b5: Pushed
2874d6784b86: Pushed
f41e8728296d: Pushed
f42aed5f7feb: Layer already exists
89819bafde36: Layer already exists
061ab49f48fb: Pushed
f3d5b8f65132: Layer already exists
ad83f0aa5c0a: Layer already exists
5a9a65095453: Layer already exists
4b0edb23340c: Layer already exists
afa3e488a0ee: Layer already exists
1dd7188f6d8c: Pushed
33db017afd3a: Pushed
c8d212339350: Pushed
20210724124333: digest: sha256:17a2d01bad9e44f5dbac64ad2779c297de834feac005d7d99e6bf96f2904e6ff size: 4310

> Task :sdks:java:testing:load-tests:run
Jul 24, 2021 12:48:07 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Jul 24, 2021 12:48:08 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 191 files. Enable logging at DEBUG level to see which files will be staged.
Jul 24, 2021 12:48:08 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Jul 24, 2021 12:48:08 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Jul 24, 2021 12:48:10 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Jul 24, 2021 12:48:10 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <112637 bytes, hash b9d4e30d33141b0f416737bfee3d0014a7077f473d79b390d54c74948b414e72> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-udTjDTMUGw9BZze_7j0AFKcHf0c9ebOQ1Ux0lItBTnI.pb
Jul 24, 2021 12:48:12 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 191 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Jul 24, 2021 12:48:12 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 191 files cached, 0 files newly uploaded in 0 seconds
Jul 24, 2021 12:48:12 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Jul 24, 2021 12:48:12 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@75de6341, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@74170687, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@68f0f72c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3d96fa9e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3b545206, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77bb48d5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@181d8899, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12d5c30e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@b887730, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26586b74, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@52f57666, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6e041285, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@267dc982, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@439b15f2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3aa41da1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@74fab04a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c83468e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@272ce069, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3d7fb838, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1c26273d]
Jul 24, 2021 12:48:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Jul 24, 2021 12:48:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Jul 24, 2021 12:48:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Jul 24, 2021 12:48:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Jul 24, 2021 12:48:13 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2abe9173, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@235d29d6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1fdca564, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43f9dd56, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1d12e953, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@57cb70be, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2d4608a6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@20d87335, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2a8a4e0c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26c89563, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3bd6ba24, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58f437b0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@20f6f88c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4277127c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4c7e978c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@354e7004, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@22ebccb9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5875de6a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@608fe01f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2687725a]
Jul 24, 2021 12:48:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Jul 24, 2021 12:48:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Jul 24, 2021 12:48:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Jul 24, 2021 12:48:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Jul 24, 2021 12:48:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Jul 24, 2021 12:48:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Jul 24, 2021 12:48:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Jul 24, 2021 12:48:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Jul 24, 2021 12:48:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Jul 24, 2021 12:48:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Jul 24, 2021 12:48:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Jul 24, 2021 12:48:13 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.33.0-SNAPSHOT
Jul 24, 2021 12:48:14 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-07-24_05_48_13-6322797474676764186?project=apache-beam-testing
Jul 24, 2021 12:48:14 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-07-24_05_48_13-6322797474676764186
Jul 24, 2021 12:48:14 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-07-24_05_48_13-6322797474676764186
Jul 24, 2021 12:48:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-07-24T12:48:20.395Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-07-eyg2. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Jul 24, 2021 12:48:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-24T12:48:24.441Z: Worker configuration: e2-standard-2 in us-central1-a.
Jul 24, 2021 12:48:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-24T12:48:25.130Z: Expanding SplittableParDo operations into optimizable parts.
Jul 24, 2021 12:48:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-24T12:48:25.156Z: Expanding CollectionToSingleton operations into optimizable parts.
Jul 24, 2021 12:48:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-24T12:48:25.231Z: Expanding CoGroupByKey operations into optimizable parts.
Jul 24, 2021 12:48:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-24T12:48:25.300Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Jul 24, 2021 12:48:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-24T12:48:25.332Z: Expanding GroupByKey operations into streaming Read/Write steps
Jul 24, 2021 12:48:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-24T12:48:25.392Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jul 24, 2021 12:48:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-24T12:48:25.494Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jul 24, 2021 12:48:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-24T12:48:25.541Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Jul 24, 2021 12:48:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-24T12:48:25.565Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Jul 24, 2021 12:48:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-24T12:48:25.606Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Jul 24, 2021 12:48:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-24T12:48:25.627Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Jul 24, 2021 12:48:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-24T12:48:25.647Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jul 24, 2021 12:48:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-24T12:48:25.682Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jul 24, 2021 12:48:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-24T12:48:25.717Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jul 24, 2021 12:48:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-24T12:48:25.740Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Jul 24, 2021 12:48:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-24T12:48:25.771Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Jul 24, 2021 12:48:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-24T12:48:25.802Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Jul 24, 2021 12:48:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-24T12:48:25.827Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Jul 24, 2021 12:48:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-24T12:48:25.868Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jul 24, 2021 12:48:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-24T12:48:25.900Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jul 24, 2021 12:48:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-24T12:48:25.928Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jul 24, 2021 12:48:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-24T12:48:25.957Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Jul 24, 2021 12:48:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-24T12:48:25.987Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Jul 24, 2021 12:48:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-24T12:48:26.031Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Jul 24, 2021 12:48:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-24T12:48:26.062Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Jul 24, 2021 12:48:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-24T12:48:26.102Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Jul 24, 2021 12:48:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-24T12:48:26.133Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Jul 24, 2021 12:48:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-24T12:48:26.161Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Jul 24, 2021 12:48:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-24T12:48:26.220Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Jul 24, 2021 12:48:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-24T12:48:26.622Z: Starting 5 ****s in us-central1-a...
Jul 24, 2021 12:48:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-24T12:48:34.899Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jul 24, 2021 12:49:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-24T12:49:10.260Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Jul 24, 2021 12:50:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-24T12:50:04.292Z: Workers have started successfully.
Jul 24, 2021 12:50:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-24T12:50:04.326Z: Workers have started successfully.
Jul 24, 2021 3:15:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-24T15:15:28.304Z: Staged package google-cloud-firestore-2.2.5-JXnHRpqqW5CurHmPM8Z6G1bA49BsaTZbsct5Cf_o2G0.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-cloud-firestore-2.2.5-JXnHRpqqW5CurHmPM8Z6G1bA49BsaTZbsct5Cf_o2G0.jar' is inaccessible.
Jul 24, 2021 3:15:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-24T15:15:30.216Z: Staged package proto-google-cloud-firestore-bundle-v1-2.2.5-_y7wfckUZrXkLGsbpYLW_Scz7IHOBjgLMQ35PIA_MVo.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/proto-google-cloud-firestore-bundle-v1-2.2.5-_y7wfckUZrXkLGsbpYLW_Scz7IHOBjgLMQ35PIA_MVo.jar' is inaccessible.
Jul 24, 2021 3:15:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-24T15:15:30.253Z: Staged package proto-google-cloud-firestore-v1-2.2.5-WUHaG1T-GdkEyeFxo3A5G1Y3GCGjqzSMA4TTW3u0JxM.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/proto-google-cloud-firestore-v1-2.2.5-WUHaG1T-GdkEyeFxo3A5G1Y3GCGjqzSMA4TTW3u0JxM.jar' is inaccessible.
Jul 24, 2021 3:15:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-07-24T15:15:30.617Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jul 24, 2021 4:00:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-24T16:00:25.549Z: Cancel request is committed for workflow job: 2021-07-24_05_48_13-6322797474676764186.
Jul 24, 2021 4:00:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-24T16:00:25.596Z: Cleaning up.
Jul 24, 2021 4:00:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-24T16:00:25.653Z: Stopping **** pool...
Jul 24, 2021 4:00:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-24T16:00:25.696Z: Stopping **** pool...
Jul 24, 2021 4:02:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-24T16:02:45.135Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Jul 24, 2021 4:02:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-24T16:02:45.169Z: Worker pool stopped.
Jul 24, 2021 4:02:50 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-07-24_05_48_13-6322797474676764186 finished with status CANCELLED.
Load test results for test (ID): 81ce2639-04fc-46ef-9063-60af2d77a32d and timestamp: 2021-07-24T12:48:08.355000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11382.625
dataflow_v2_java11_total_bytes_count              3.2530754E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210724124333
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:17a2d01bad9e44f5dbac64ad2779c297de834feac005d7d99e6bf96f2904e6ff
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:17a2d01bad9e44f5dbac64ad2779c297de834feac005d7d99e6bf96f2904e6ff
  Associated tags:
 - 20210724124333
Tags:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210724124333
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210724124333].
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:17a2d01bad9e44f5dbac64ad2779c297de834feac005d7d99e6bf96f2904e6ff].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 33s
107 actionable tasks: 76 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/6ghsecvxbaeoy

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #36

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/36/display/redirect?page=changes>

Changes:

[noreply] [BEAM-12548] Fix malformed comment for EqualsFloat (#15200)

[noreply] [BEAM-4152] Implement session window merging in the Go direct runner

[Kyle Weaver] [BEAM-12625] Annotate

[noreply] [BEAM-12643] Resolved the concurrent writes issue for parallel tests

[noreply] [BEAM-10212] Add caching state client wrapper (#15170)

[noreply] [BEAM-12640] Enable adding JHM to Java based projects and add an example


------------------------------------------
[...truncated 71.60 KB...]
Jul 23, 2021 12:48:36 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2abe9173, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@235d29d6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1fdca564, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43f9dd56, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1d12e953, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@57cb70be, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2d4608a6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@20d87335, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2a8a4e0c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26c89563, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3bd6ba24, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58f437b0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@20f6f88c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4277127c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4c7e978c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@354e7004, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@22ebccb9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5875de6a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@608fe01f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2687725a]
Jul 23, 2021 12:48:36 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Jul 23, 2021 12:48:36 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Jul 23, 2021 12:48:36 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Jul 23, 2021 12:48:36 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Jul 23, 2021 12:48:36 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Jul 23, 2021 12:48:36 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Jul 23, 2021 12:48:36 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Jul 23, 2021 12:48:36 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Jul 23, 2021 12:48:36 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Jul 23, 2021 12:48:36 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Jul 23, 2021 12:48:36 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Jul 23, 2021 12:48:36 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.33.0-SNAPSHOT
Jul 23, 2021 12:48:37 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-07-23_05_48_36-14790543457681577798?project=apache-beam-testing
Jul 23, 2021 12:48:37 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-07-23_05_48_36-14790543457681577798
Jul 23, 2021 12:48:37 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-07-23_05_48_36-14790543457681577798
Jul 23, 2021 12:48:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-07-23T12:48:43.529Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-07-7570. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Jul 23, 2021 12:48:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-23T12:48:48.420Z: Worker configuration: e2-standard-2 in us-central1-a.
Jul 23, 2021 12:48:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-23T12:48:49.081Z: Expanding SplittableParDo operations into optimizable parts.
Jul 23, 2021 12:48:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-23T12:48:49.104Z: Expanding CollectionToSingleton operations into optimizable parts.
Jul 23, 2021 12:48:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-23T12:48:49.167Z: Expanding CoGroupByKey operations into optimizable parts.
Jul 23, 2021 12:48:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-23T12:48:49.228Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Jul 23, 2021 12:48:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-23T12:48:49.258Z: Expanding GroupByKey operations into streaming Read/Write steps
Jul 23, 2021 12:48:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-23T12:48:49.305Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jul 23, 2021 12:48:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-23T12:48:49.397Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jul 23, 2021 12:48:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-23T12:48:49.423Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Jul 23, 2021 12:48:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-23T12:48:49.457Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Jul 23, 2021 12:48:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-23T12:48:49.489Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Jul 23, 2021 12:48:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-23T12:48:49.522Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Jul 23, 2021 12:48:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-23T12:48:49.544Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jul 23, 2021 12:48:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-23T12:48:49.568Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jul 23, 2021 12:48:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-23T12:48:49.609Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jul 23, 2021 12:48:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-23T12:48:49.643Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Jul 23, 2021 12:48:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-23T12:48:49.680Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Jul 23, 2021 12:48:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-23T12:48:49.703Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Jul 23, 2021 12:48:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-23T12:48:49.725Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Jul 23, 2021 12:48:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-23T12:48:49.751Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jul 23, 2021 12:48:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-23T12:48:49.788Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jul 23, 2021 12:48:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-23T12:48:49.826Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jul 23, 2021 12:48:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-23T12:48:49.853Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Jul 23, 2021 12:48:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-23T12:48:49.887Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Jul 23, 2021 12:48:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-23T12:48:49.920Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Jul 23, 2021 12:48:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-23T12:48:49.954Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Jul 23, 2021 12:48:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-23T12:48:49.988Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Jul 23, 2021 12:48:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-23T12:48:50.021Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Jul 23, 2021 12:48:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-23T12:48:50.053Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Jul 23, 2021 12:48:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-23T12:48:50.089Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Jul 23, 2021 12:48:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-23T12:48:50.460Z: Starting 5 ****s in us-central1-a...
Jul 23, 2021 12:49:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-23T12:49:05.301Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jul 23, 2021 12:49:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-23T12:49:39.699Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Jul 23, 2021 12:50:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-23T12:50:33.059Z: Workers have started successfully.
Jul 23, 2021 12:50:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-23T12:50:33.091Z: Workers have started successfully.
Jul 23, 2021 12:54:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-23T12:54:50.408Z: generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:1089
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:1089
Jul 23, 2021 3:15:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-23T15:15:51.367Z: Staged package beam-vendor-grpc-1_36_0-0.2-2X-EhRNYA0GgyvffOgWohko1OyyyPyTMuiHHvaPG9es.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-grpc-1_36_0-0.2-2X-EhRNYA0GgyvffOgWohko1OyyyPyTMuiHHvaPG9es.jar' is inaccessible.
Jul 23, 2021 3:15:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-07-23T15:15:54.545Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jul 23, 2021 3:18:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-07-23T15:18:55.010Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jul 23, 2021 3:21:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-23T15:21:51.376Z: Staged package beam-vendor-grpc-1_36_0-0.2-2X-EhRNYA0GgyvffOgWohko1OyyyPyTMuiHHvaPG9es.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-grpc-1_36_0-0.2-2X-EhRNYA0GgyvffOgWohko1OyyyPyTMuiHHvaPG9es.jar' is inaccessible.
Jul 23, 2021 3:21:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-07-23T15:21:54.954Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jul 23, 2021 3:24:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-07-23T15:24:54.626Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jul 23, 2021 3:27:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-23T15:27:51.508Z: Staged package beam-vendor-grpc-1_36_0-0.2-2X-EhRNYA0GgyvffOgWohko1OyyyPyTMuiHHvaPG9es.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-grpc-1_36_0-0.2-2X-EhRNYA0GgyvffOgWohko1OyyyPyTMuiHHvaPG9es.jar' is inaccessible.
Jul 23, 2021 3:27:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-07-23T15:27:55.038Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jul 23, 2021 3:30:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-07-23T15:30:55.820Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jul 23, 2021 3:33:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-23T15:33:51.489Z: Staged package beam-vendor-grpc-1_36_0-0.2-2X-EhRNYA0GgyvffOgWohko1OyyyPyTMuiHHvaPG9es.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-grpc-1_36_0-0.2-2X-EhRNYA0GgyvffOgWohko1OyyyPyTMuiHHvaPG9es.jar' is inaccessible.
Jul 23, 2021 3:33:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-07-23T15:33:54.733Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jul 23, 2021 3:36:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-07-23T15:36:54.738Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jul 23, 2021 3:39:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-23T15:39:51.490Z: Staged package beam-vendor-grpc-1_36_0-0.2-2X-EhRNYA0GgyvffOgWohko1OyyyPyTMuiHHvaPG9es.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-grpc-1_36_0-0.2-2X-EhRNYA0GgyvffOgWohko1OyyyPyTMuiHHvaPG9es.jar' is inaccessible.
Jul 23, 2021 3:39:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-07-23T15:39:54.407Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jul 23, 2021 3:42:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-07-23T15:42:54.875Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jul 23, 2021 3:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-23T15:45:51.353Z: Staged package beam-vendor-grpc-1_36_0-0.2-2X-EhRNYA0GgyvffOgWohko1OyyyPyTMuiHHvaPG9es.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-grpc-1_36_0-0.2-2X-EhRNYA0GgyvffOgWohko1OyyyPyTMuiHHvaPG9es.jar' is inaccessible.
Jul 23, 2021 3:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-07-23T15:45:54.230Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jul 23, 2021 3:48:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-07-23T15:48:54.424Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jul 23, 2021 3:51:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-23T15:51:51.341Z: Staged package beam-vendor-grpc-1_36_0-0.2-2X-EhRNYA0GgyvffOgWohko1OyyyPyTMuiHHvaPG9es.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-grpc-1_36_0-0.2-2X-EhRNYA0GgyvffOgWohko1OyyyPyTMuiHHvaPG9es.jar' is inaccessible.
Jul 23, 2021 3:51:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-07-23T15:51:54.080Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jul 23, 2021 3:54:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-07-23T15:54:54.815Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jul 23, 2021 3:57:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-23T15:57:51.363Z: Staged package beam-vendor-grpc-1_36_0-0.2-2X-EhRNYA0GgyvffOgWohko1OyyyPyTMuiHHvaPG9es.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-grpc-1_36_0-0.2-2X-EhRNYA0GgyvffOgWohko1OyyyPyTMuiHHvaPG9es.jar' is inaccessible.
Jul 23, 2021 3:57:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-07-23T15:57:54.243Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jul 23, 2021 4:00:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-23T16:00:44.914Z: Cancel request is committed for workflow job: 2021-07-23_05_48_36-14790543457681577798.
Jul 23, 2021 4:00:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-23T16:00:44.947Z: Cleaning up.
Jul 23, 2021 4:00:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-23T16:00:45.024Z: Stopping **** pool...
Jul 23, 2021 4:00:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-23T16:00:45.077Z: Stopping **** pool...
Jul 23, 2021 4:02:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-23T16:02:54.470Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Jul 23, 2021 4:02:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-23T16:02:54.513Z: Worker pool stopped.
Jul 23, 2021 4:03:00 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-07-23_05_48_36-14790543457681577798 finished with status CANCELLED.
Load test results for test (ID): e2419e44-9ec4-4673-83fe-3970ea60fd35 and timestamp: 2021-07-23T12:48:31.121000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11332.718
dataflow_v2_java11_total_bytes_count             2.00647729E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210723124335
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:93c43cebd5b376c560abe994f16ea2e5e60d94718b6ebc60d51c4cd403adbd0a
Deleted: sha256:aefd2aef2d319c2273827c2238d9d2ae181b8aecdd0195601077ac6cfbba26f2
Deleted: sha256:2d3ece848e1b4dbd657f129ccc03a453867a9f9551d923d1907aaa8758f08a11
Deleted: sha256:e28845cf1ac3645bdc1d2c55429a514c2c161271ce3d8f6cc9238d8d479a1372
Deleted: sha256:06b7e6cacc5f68aefec37ce2f69be04baa316f11ebd07fc4ff98811b0c7afd20
Deleted: sha256:465e1c9dc8c89ed8177a4f9e4d46048824788a2c088c432dc64d88da3601d6a3
Deleted: sha256:19e04d65290e5759a6840c1a1697809e4d71e426e86f3595126d246954f7755b
Deleted: sha256:ea77f2d7b3e5976d1a2261f775e9ebedac40b9bb12d9960ba6d32c03312b1ecc
Deleted: sha256:fbed93152be1fdd93040c65c40ebb0a1473ae4da3ad534844b707e5e02adeba5
Deleted: sha256:07eaec73b190ba7f5dc16ef9e7607ce5dc445925ca7c32a72059a3d7af76dcb9
Deleted: sha256:32d8403fd8088df36ea2db038aad19f5e3d6fe3f55aa888eac9f07e53e34a5ac
Deleted: sha256:3fdb5b96ad2e1637e5fa7e15c4fbb9595d926c0cb484b05820daaa1b0c2c41f8
Deleted: sha256:b916782aac782f61d73efca2e80c5cd69c14f4cd12330aeb7aeb05499cba24d6
Deleted: sha256:aaee8c8cf8b3af39fc483f590c8bcd476f98af18a90be73c3a239460eb254cf4
Deleted: sha256:e5d519ef43e3c12148339a0d1eab1451c43cee7685ec3a68056be6ead19d7247
Deleted: sha256:5e3b21c08cc73ecff173e51eb5cbdcc238e3ee171026a67257190d7aad3cfef1
Deleted: sha256:be73a56922026153a3968da69eb8fc80f69541b859370e88390f99637272a4b8
Deleted: sha256:088dedb9000d202f1cb64f64fd8c6a7db77225d706e05d88a50bab9cdc1a9034
Deleted: sha256:5da8feec1aff623e2f1f68fb500e1ffc74d24e777e161ed4f3e5e86ac1895a08
Deleted: sha256:ff8d631b693d390a12ca4c16072595e91e8db5707828e01037d3db93b7c268ad
Deleted: sha256:1424513a9a54d72e27b50c437dcd871cbb62ae7fa91c49d3db099019750aada9
Deleted: sha256:648114a26318c8764b02bad42c870ad33adbd890ff98fcb537bba8180d616c27
Deleted: sha256:c20a52dd3689b216855d4f51e77f83d7dfe49714a950a7e80814e4fd6548746b
Deleted: sha256:6249dfff00ea66267de47095c35db8648d9897512afac0b8b6b3a09ea59072c4
Deleted: sha256:01b926ceda0cf05c48f12579d68c247e0a87276fe00c37a77516861dfca640fd
Deleted: sha256:5981672529b3df709909bc6c0077e23a4eea2d867becf8f4fd3bb2033c2da1af
Deleted: sha256:018e275d8640d9f609a29d0f560c152e0adf72ad7391ca2a3e9d8fd4b988ffa7
Deleted: sha256:aad5bd8772da4e9b26ac6ceaa96ac5839e687ad8dfade502b2634877207d6ade
Deleted: sha256:8ec087beb12087fec4e1321ffa31c32e377b9185517991f98cfac15ef852d9ee
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:93c43cebd5b376c560abe994f16ea2e5e60d94718b6ebc60d51c4cd403adbd0a
  Associated tags:
 - 20210723124335
Tags:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210723124335
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210723124335].
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:93c43cebd5b376c560abe994f16ea2e5e60d94718b6ebc60d51c4cd403adbd0a].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 43s
107 actionable tasks: 78 executed, 27 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/dq5ihu7ftb7la

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #35

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/35/display/redirect?page=changes>

Changes:

[dhuntsperger] updated site docs with troubleshooting info

[noreply] Default to Runner v2 for Python Streaming jobs. (#15140)

[ankurgoenka] Moving to 2.33.0-SNAPSHOT on master branch.


------------------------------------------
[...truncated 65.86 KB...]
9e92e4aa007e: Preparing
74ddce2746bf: Preparing
e5c757747c96: Preparing
c47f8e016290: Preparing
c0848348e2f7: Preparing
79c550eb7bd2: Preparing
7095af798ace: Preparing
fe6a4fdbedc0: Preparing
e4d0e810d54a: Preparing
4e006334a6fd: Preparing
adb2ed38ba39: Waiting
a6879de94b70: Waiting
7a91641c70a5: Waiting
cb70f3fcb7cc: Waiting
9e92e4aa007e: Waiting
74ddce2746bf: Waiting
e5c757747c96: Waiting
c47f8e016290: Waiting
c0848348e2f7: Waiting
79c550eb7bd2: Waiting
7095af798ace: Waiting
fe6a4fdbedc0: Waiting
e4d0e810d54a: Waiting
4e006334a6fd: Waiting
474a959ff641: Pushed
f9331bdb793b: Pushed
fc39f2120ae3: Pushed
adb2ed38ba39: Pushed
23b0411dbeb6: Pushed
7a91641c70a5: Pushed
c5f74c5d3bbe: Pushed
cb70f3fcb7cc: Pushed
c47f8e016290: Layer already exists
c0848348e2f7: Layer already exists
79c550eb7bd2: Layer already exists
74ddce2746bf: Pushed
7095af798ace: Layer already exists
e4d0e810d54a: Layer already exists
fe6a4fdbedc0: Layer already exists
a6879de94b70: Pushed
4e006334a6fd: Layer already exists
e5c757747c96: Pushed
9e92e4aa007e: Pushed
20210722124331: digest: sha256:c318eeff2768e0b8d1d85989d2a8bc622a401bc1014cc14bbfb2f82315e35a13 size: 4310

> Task :sdks:java:testing:load-tests:run
Jul 22, 2021 12:48:14 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Jul 22, 2021 12:48:15 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 191 files. Enable logging at DEBUG level to see which files will be staged.
Jul 22, 2021 12:48:15 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Jul 22, 2021 12:48:15 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Jul 22, 2021 12:48:18 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Jul 22, 2021 12:48:18 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <112637 bytes, hash c8cafefc682ebb76b799f0da27d924381d5a3e21f3c5eae76476b186647bd013> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-yMr-_Gguu3a3mfDaJ9kkOB1aPiHzxernZHaxhmR70BM.pb
Jul 22, 2021 12:48:20 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 191 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Jul 22, 2021 12:48:20 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 191 files cached, 0 files newly uploaded in 0 seconds
Jul 22, 2021 12:48:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Jul 22, 2021 12:48:20 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c3c4a71, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1352434e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4f9a6c2d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2b6fcb9f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@75de6341, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@74170687, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@68f0f72c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3d96fa9e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3b545206, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77bb48d5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@181d8899, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12d5c30e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@b887730, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26586b74, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@52f57666, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6e041285, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@267dc982, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@439b15f2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3aa41da1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@74fab04a]
Jul 22, 2021 12:48:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Jul 22, 2021 12:48:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Jul 22, 2021 12:48:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Jul 22, 2021 12:48:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Jul 22, 2021 12:48:20 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@44ed0a8f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@32177fa5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@a96d56c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6ab4a5b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2abe9173, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@235d29d6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1fdca564, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43f9dd56, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1d12e953, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@57cb70be, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2d4608a6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@20d87335, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2a8a4e0c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26c89563, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3bd6ba24, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58f437b0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@20f6f88c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4277127c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4c7e978c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@354e7004]
Jul 22, 2021 12:48:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Jul 22, 2021 12:48:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Jul 22, 2021 12:48:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Jul 22, 2021 12:48:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Jul 22, 2021 12:48:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Jul 22, 2021 12:48:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Jul 22, 2021 12:48:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Jul 22, 2021 12:48:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Jul 22, 2021 12:48:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Jul 22, 2021 12:48:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Jul 22, 2021 12:48:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Jul 22, 2021 12:48:20 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.33.0-SNAPSHOT
Jul 22, 2021 12:48:21 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-07-22_05_48_21-17285135585842088076?project=apache-beam-testing
Jul 22, 2021 12:48:21 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-07-22_05_48_21-17285135585842088076
Jul 22, 2021 12:48:21 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-07-22_05_48_21-17285135585842088076
Jul 22, 2021 12:48:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-07-22T12:48:27.753Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-07-qpgs. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Jul 22, 2021 12:48:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-22T12:48:32.804Z: Worker configuration: e2-standard-2 in us-central1-a.
Jul 22, 2021 12:48:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-22T12:48:33.545Z: Expanding SplittableParDo operations into optimizable parts.
Jul 22, 2021 12:48:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-22T12:48:33.577Z: Expanding CollectionToSingleton operations into optimizable parts.
Jul 22, 2021 12:48:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-22T12:48:33.625Z: Expanding CoGroupByKey operations into optimizable parts.
Jul 22, 2021 12:48:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-22T12:48:33.704Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Jul 22, 2021 12:48:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-22T12:48:33.736Z: Expanding GroupByKey operations into streaming Read/Write steps
Jul 22, 2021 12:48:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-22T12:48:33.787Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jul 22, 2021 12:48:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-22T12:48:33.895Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jul 22, 2021 12:48:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-22T12:48:33.932Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Jul 22, 2021 12:48:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-22T12:48:33.969Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Jul 22, 2021 12:48:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-22T12:48:33.998Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Jul 22, 2021 12:48:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-22T12:48:34.020Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Jul 22, 2021 12:48:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-22T12:48:34.048Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jul 22, 2021 12:48:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-22T12:48:34.076Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jul 22, 2021 12:48:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-22T12:48:34.123Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jul 22, 2021 12:48:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-22T12:48:34.159Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Jul 22, 2021 12:48:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-22T12:48:34.184Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Jul 22, 2021 12:48:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-22T12:48:34.212Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Jul 22, 2021 12:48:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-22T12:48:34.253Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Jul 22, 2021 12:48:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-22T12:48:34.280Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jul 22, 2021 12:48:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-22T12:48:34.328Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jul 22, 2021 12:48:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-22T12:48:34.353Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jul 22, 2021 12:48:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-22T12:48:34.385Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Jul 22, 2021 12:48:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-22T12:48:34.423Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Jul 22, 2021 12:48:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-22T12:48:34.444Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Jul 22, 2021 12:48:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-22T12:48:34.479Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Jul 22, 2021 12:48:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-22T12:48:34.516Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Jul 22, 2021 12:48:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-22T12:48:34.555Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Jul 22, 2021 12:48:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-22T12:48:34.591Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Jul 22, 2021 12:48:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-22T12:48:34.616Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Jul 22, 2021 12:48:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-22T12:48:34.995Z: Starting 5 ****s in us-central1-a...
Jul 22, 2021 12:48:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-22T12:48:56.778Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jul 22, 2021 12:49:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-22T12:49:16.066Z: Autoscaling: Raised the number of ****s to 4 so that the pipeline can catch up with its backlog and keep up with its input rate.
Jul 22, 2021 12:49:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-22T12:49:16.092Z: Resized **** pool to 4, though goal was 5.  This could be a quota issue.
Jul 22, 2021 12:49:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-22T12:49:36.618Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Jul 22, 2021 12:50:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-22T12:50:06.691Z: Workers have started successfully.
Jul 22, 2021 12:50:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-22T12:50:06.728Z: Workers have started successfully.
Jul 22, 2021 3:58:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-22T15:57:58.917Z: generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
    dist_proc/dax/workflow/****/streaming/streaming_rpc_windmill_service_server.cc:1089
Jul 22, 2021 4:00:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-22T16:00:39.515Z: Cancel request is committed for workflow job: 2021-07-22_05_48_21-17285135585842088076.
Jul 22, 2021 4:00:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-22T16:00:45.458Z: Cleaning up.
Jul 22, 2021 4:00:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-22T16:00:45.742Z: Stopping **** pool...
Jul 22, 2021 4:00:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-22T16:00:45.834Z: Stopping **** pool...
Jul 22, 2021 4:03:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-22T16:03:14.019Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Jul 22, 2021 4:03:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-22T16:03:14.049Z: Worker pool stopped.
Jul 22, 2021 4:03:22 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-07-22_05_48_21-17285135585842088076 finished with status CANCELLED.
Load test results for test (ID): b304beb5-48a9-49b3-bb78-21dbe7767ff0 and timestamp: 2021-07-22T12:48:15.473000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11380.866
dataflow_v2_java11_total_bytes_count             2.40897675E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210722124331
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:c318eeff2768e0b8d1d85989d2a8bc622a401bc1014cc14bbfb2f82315e35a13
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:c318eeff2768e0b8d1d85989d2a8bc622a401bc1014cc14bbfb2f82315e35a13
  Associated tags:
 - 20210722124331
Tags:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210722124331
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210722124331].
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:c318eeff2768e0b8d1d85989d2a8bc622a401bc1014cc14bbfb2f82315e35a13].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 20m 7s
107 actionable tasks: 76 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/xgv7l2t7ephte

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #34

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/34/display/redirect?page=changes>

Changes:

[dhuntsperger] fixed broken tabs for runner and shell in java quickstart

[noreply] [BEAM-10955] Revert f077ba51cb0790c403297833d5378b399b7248df (#15119)

[noreply] [BEAM-12596] Ensure that SDF restriction size is always greater then

[noreply] [BEAM-12639] Allow for zero splits in SplitAndSizeRestriction (#15191)

[noreply] [BEAM-12559] Enable CrossLanguageValidateRunner test for Samza (#15195)

[dhuntsperger] fix remaining runner and shell snippets

[noreply] Update Container beam-master-20210720 (#15197)

[noreply] [BEAM-12445] Handle none bq_client (#15194)


------------------------------------------
[...truncated 66.94 KB...]
29d6a5ce2fd2: Pushed
edc4b6146fd1: Pushed
a32859fda512: Pushed
6ea995e9b7d3: Layer already exists
c0848348e2f7: Layer already exists
79c550eb7bd2: Layer already exists
7095af798ace: Layer already exists
fe6a4fdbedc0: Layer already exists
e4d0e810d54a: Layer already exists
4e006334a6fd: Layer already exists
a30664f74bb4: Pushed
d99f4998ca2c: Pushed
ae97fe29b213: Pushed
a5adbddbfef0: Pushed
20210721124334: digest: sha256:23b6b7f7a422ec962a94472ba5a7d083081b928f79d84b5e23a3fd349c03dfdd size: 4310

> Task :sdks:java:testing:load-tests:run
Jul 21, 2021 12:48:44 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Jul 21, 2021 12:48:45 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 191 files. Enable logging at DEBUG level to see which files will be staged.
Jul 21, 2021 12:48:45 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Jul 21, 2021 12:48:45 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Jul 21, 2021 12:48:48 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Jul 21, 2021 12:48:48 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <112699 bytes, hash 111d3c12f5bf4e5a7d160c9be9467fb850c7295c89483617a9a8a7e74ff111be> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-ER08EvW_Tlp9Fgyb6UZ_uFDHKVyJSDYXqain50_xEb4.pb
Jul 21, 2021 12:48:49 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 191 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Jul 21, 2021 12:48:50 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 191 files cached, 0 files newly uploaded in 0 seconds
Jul 21, 2021 12:48:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Jul 21, 2021 12:48:50 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@75de6341, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@74170687, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@68f0f72c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3d96fa9e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3b545206, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77bb48d5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@181d8899, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12d5c30e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@b887730, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26586b74, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@52f57666, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6e041285, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@267dc982, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@439b15f2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3aa41da1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@74fab04a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c83468e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@272ce069, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3d7fb838, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1c26273d]
Jul 21, 2021 12:48:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Jul 21, 2021 12:48:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Jul 21, 2021 12:48:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Jul 21, 2021 12:48:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Jul 21, 2021 12:48:50 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2abe9173, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@235d29d6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1fdca564, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43f9dd56, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1d12e953, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@57cb70be, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2d4608a6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@20d87335, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2a8a4e0c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26c89563, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3bd6ba24, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58f437b0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@20f6f88c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4277127c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4c7e978c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@354e7004, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@22ebccb9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5875de6a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@608fe01f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2687725a]
Jul 21, 2021 12:48:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Jul 21, 2021 12:48:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Jul 21, 2021 12:48:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Jul 21, 2021 12:48:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Jul 21, 2021 12:48:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Jul 21, 2021 12:48:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Jul 21, 2021 12:48:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Jul 21, 2021 12:48:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Jul 21, 2021 12:48:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Jul 21, 2021 12:48:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Jul 21, 2021 12:48:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Jul 21, 2021 12:48:50 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.32.0-SNAPSHOT
Jul 21, 2021 12:48:51 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-07-21_05_48_50-12852398544501993323?project=apache-beam-testing
Jul 21, 2021 12:48:51 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-07-21_05_48_50-12852398544501993323
Jul 21, 2021 12:48:51 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-07-21_05_48_50-12852398544501993323
Jul 21, 2021 12:48:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-07-21T12:48:57.981Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-07-s5hj. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Jul 21, 2021 12:49:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-21T12:49:02.581Z: Worker configuration: e2-standard-2 in us-central1-a.
Jul 21, 2021 12:49:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-21T12:49:03.233Z: Expanding SplittableParDo operations into optimizable parts.
Jul 21, 2021 12:49:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-21T12:49:03.263Z: Expanding CollectionToSingleton operations into optimizable parts.
Jul 21, 2021 12:49:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-21T12:49:03.332Z: Expanding CoGroupByKey operations into optimizable parts.
Jul 21, 2021 12:49:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-21T12:49:03.411Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Jul 21, 2021 12:49:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-21T12:49:03.440Z: Expanding GroupByKey operations into streaming Read/Write steps
Jul 21, 2021 12:49:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-21T12:49:03.509Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jul 21, 2021 12:49:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-21T12:49:03.617Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jul 21, 2021 12:49:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-21T12:49:03.651Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Jul 21, 2021 12:49:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-21T12:49:03.683Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Jul 21, 2021 12:49:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-21T12:49:03.709Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Jul 21, 2021 12:49:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-21T12:49:03.752Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Jul 21, 2021 12:49:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-21T12:49:03.783Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jul 21, 2021 12:49:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-21T12:49:03.817Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jul 21, 2021 12:49:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-21T12:49:03.843Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jul 21, 2021 12:49:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-21T12:49:03.878Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Jul 21, 2021 12:49:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-21T12:49:03.910Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Jul 21, 2021 12:49:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-21T12:49:03.947Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Jul 21, 2021 12:49:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-21T12:49:03.977Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Jul 21, 2021 12:49:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-21T12:49:04.013Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jul 21, 2021 12:49:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-21T12:49:04.034Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jul 21, 2021 12:49:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-21T12:49:04.058Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jul 21, 2021 12:49:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-21T12:49:04.080Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Jul 21, 2021 12:49:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-21T12:49:04.118Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Jul 21, 2021 12:49:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-21T12:49:04.152Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Jul 21, 2021 12:49:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-21T12:49:04.186Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Jul 21, 2021 12:49:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-21T12:49:04.226Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Jul 21, 2021 12:49:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-21T12:49:04.256Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Jul 21, 2021 12:49:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-21T12:49:04.288Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Jul 21, 2021 12:49:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-21T12:49:04.312Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Jul 21, 2021 12:49:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-21T12:49:04.689Z: Starting 5 ****s in us-central1-a...
Jul 21, 2021 12:49:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-21T12:49:29.990Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jul 21, 2021 12:49:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-21T12:49:50.356Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Jul 21, 2021 12:50:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-21T12:50:45.131Z: Workers have started successfully.
Jul 21, 2021 12:50:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-21T12:50:45.171Z: Workers have started successfully.
Jul 21, 2021 1:07:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-21T13:07:07.494Z: Staged package jackson-dataformat-cbor-2.12.1-52d5rqlCfKc9f0B-X6gIoEBVeOwFZlOoZcJuTG8B1Cg.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/jackson-dataformat-cbor-2.12.1-52d5rqlCfKc9f0B-X6gIoEBVeOwFZlOoZcJuTG8B1Cg.jar' is inaccessible.
Jul 21, 2021 1:07:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-07-21T13:07:08.668Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jul 21, 2021 1:10:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-07-21T13:10:08.476Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jul 21, 2021 1:13:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-21T13:13:07.368Z: Staged package jackson-dataformat-cbor-2.12.1-52d5rqlCfKc9f0B-X6gIoEBVeOwFZlOoZcJuTG8B1Cg.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/jackson-dataformat-cbor-2.12.1-52d5rqlCfKc9f0B-X6gIoEBVeOwFZlOoZcJuTG8B1Cg.jar' is inaccessible.
Jul 21, 2021 1:13:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-07-21T13:13:08.521Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jul 21, 2021 4:00:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-21T16:00:34.117Z: Cancel request is committed for workflow job: 2021-07-21_05_48_50-12852398544501993323.
Jul 21, 2021 4:00:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-21T16:00:34.153Z: Cleaning up.
Jul 21, 2021 4:00:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-21T16:00:34.233Z: Stopping **** pool...
Jul 21, 2021 4:00:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-21T16:00:34.290Z: Stopping **** pool...
Jul 21, 2021 4:02:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-21T16:02:55.281Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Jul 21, 2021 4:02:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-21T16:02:55.332Z: Worker pool stopped.
Jul 21, 2021 4:03:01 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-07-21_05_48_50-12852398544501993323 finished with status CANCELLED.
Load test results for test (ID): ad4ee3ca-e452-4174-9dca-da084306295c and timestamp: 2021-07-21T12:48:45.530000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11315.361
dataflow_v2_java11_total_bytes_count             2.48966186E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210721124334
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:23b6b7f7a422ec962a94472ba5a7d083081b928f79d84b5e23a3fd349c03dfdd
Deleted: sha256:bd05cb84b8d2dc452b750ee3657cc6c4df136cbbc8c7b33eee140ed2134629d5
Deleted: sha256:46850aebf9697d727d04597b44e8c2327ec01b2caa16e5715fe656fff4310e56
Deleted: sha256:7f2bdea2265ae970919d9c35439fcc93d9f09f69b6f968ac66853a6b7a484355
Deleted: sha256:ae5057082a76f0c4e622fca5ada01f4e7cbbf80f761a8ecdddcf233fcfd6a7da
Deleted: sha256:870effe4c47469d313b4848b9de339ed81b19b76603d7460d33a93b3abf34d6c
Deleted: sha256:188a0aa329a25a88d1511509085cd1745a6bcca92a87eb0ee6accbdcd40ab71b
Deleted: sha256:e066e210a15264bd487b345291e8e03263f04da3d6b6ba6524c3baf783ec7a8c
Deleted: sha256:b3ecbd99352d02d42f7f1570f703c291e5d9d0cd05cd42522f84418b0fa88f89
Deleted: sha256:4cae11db87ada4a788629e87ce84232dbf6fb844deec9852e091d8b0339ca8c9
Deleted: sha256:9ff61a9497c29d500c4e079a2a1b000d7f3604ef0360681982bd83c5f32a1894
Deleted: sha256:a3fabfd3a20d0aa595c33a58800224927f6009e949ec53df8e0b9004e5235c41
Deleted: sha256:d0b3cc1aedd3c69898c6379e2c02c5f2cfc553e8a3b79c4da6b6bcf0013528fd
Deleted: sha256:fb87eaf5021406a059d7d0ce142688fab2a6a1cf9d20760a4e3e3eb20e8cf780
Deleted: sha256:14f605e2e304e37bd2e07a86262905b169e7738b123b8cd1d16667547d5f1ea1
Deleted: sha256:8c3962651351b554e913dbf126465fb5f8da734792a9359f5e9f9de1360b7c2e
Deleted: sha256:860e2e631ba8e55a7922c6cb906461618862998acbc4aa6323305359eb090ef2
Deleted: sha256:1fc607197153b28aaf8e1af6b127f069f31dddb3581ea66cc3ab987ac50cd5a3
Deleted: sha256:50f6519b66e1d6ed916016874a1e765daee0cbaa2e273632529c258884a72aaf
Deleted: sha256:d8f0dabf6b1113d5e4a7e332147292dfef0332c27eaab34c725f217197f0d739
Deleted: sha256:173ced8af7cc371a593fec10d01070e82d270d219d345396bb43f577c786fd0a
Deleted: sha256:ada30f5bb1fcec16747307c22f4c787dd9e1798a227ea9708adbaee593a99b45
Deleted: sha256:d8939fbb07ed51972fc37e2405d643cd4deaff47450a5fb16b13d1b4d2016f4f
Deleted: sha256:cad02f09c3721cee506d8104baf97740476062829bcdb0ecdf917096034a4ec6
Deleted: sha256:fdf0d1cb9dc1d635af540a6bab47c5988007686d9219d64bba60ff95791d89bb
Deleted: sha256:1293107bedb393c5853a8fcca3323039a61570c9fcaea62e3a90699f78a975da
Deleted: sha256:dd6648cbd5312b2960deb0d0d6c7ce6c27eb5ca0fbf4aaccf43401810fafbc61
Deleted: sha256:7282baad33e26bd41c23ff1cdea987cbdbe73c34c9f57de8e6191a4c75394b49
Deleted: sha256:15300ae95679c931a2f3fa2176bf91c0da4e5e62e467e56ff73d5ce333d87e71
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:23b6b7f7a422ec962a94472ba5a7d083081b928f79d84b5e23a3fd349c03dfdd
  Associated tags:
 - 20210721124334
Tags:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210721124334
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210721124334].
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:23b6b7f7a422ec962a94472ba5a7d083081b928f79d84b5e23a3fd349c03dfdd].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 43s
107 actionable tasks: 76 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/cpebqyfognsyq

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #33

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/33/display/redirect?page=changes>

Changes:

[zhoufek] [BEAM-12474] Write PubsubIO parsing errors to dead-letter topic

[noreply] [BEAM-4152] Disable window tests on Dataflow (#15188)

[noreply] [BEAM-12548] Implement EqualsFloat test helper (#15175)

[noreply] Fix formatting using go fmt (#15189)

[noreply] [BEAM-4152] Add merging strategy for sessions to WindowingStrategy proto

[noreply] [BEAM-12613] Enable Python build tests for Samza (#15169)

[noreply] Fix spelling.


------------------------------------------
[...truncated 65.99 KB...]
See: https://cloud.google.com/container-registry/docs/support/deprecation-notices#gcloud-docker

The push refers to repository [us.gcr.io/apache-beam-testing/java-postcommit-it/java]
9b42aa0d18fc: Preparing
93821ce7b4d4: Preparing
b5dff7010038: Preparing
fa1948845804: Preparing
fb0606e647ba: Preparing
0a6cc7fa3603: Preparing
743ef93eef7b: Preparing
0ee94ef367e5: Preparing
0bc160da78ab: Preparing
d39bc526141e: Preparing
c65143949103: Preparing
62ab8cd4222c: Preparing
6ea995e9b7d3: Preparing
c0848348e2f7: Preparing
79c550eb7bd2: Preparing
7095af798ace: Preparing
fe6a4fdbedc0: Preparing
e4d0e810d54a: Preparing
4e006334a6fd: Preparing
62ab8cd4222c: Waiting
6ea995e9b7d3: Waiting
c0848348e2f7: Waiting
0a6cc7fa3603: Waiting
e4d0e810d54a: Waiting
743ef93eef7b: Waiting
4e006334a6fd: Waiting
0ee94ef367e5: Waiting
7095af798ace: Waiting
fe6a4fdbedc0: Waiting
79c550eb7bd2: Waiting
93821ce7b4d4: Pushed
fb0606e647ba: Pushed
b5dff7010038: Pushed
9b42aa0d18fc: Pushed
0a6cc7fa3603: Pushed
fa1948845804: Pushed
0ee94ef367e5: Pushed
0bc160da78ab: Pushed
6ea995e9b7d3: Layer already exists
c65143949103: Pushed
c0848348e2f7: Layer already exists
79c550eb7bd2: Layer already exists
7095af798ace: Layer already exists
fe6a4fdbedc0: Layer already exists
e4d0e810d54a: Layer already exists
4e006334a6fd: Layer already exists
62ab8cd4222c: Pushed
743ef93eef7b: Pushed
d39bc526141e: Pushed
20210720124333: digest: sha256:adcb6c4a7a0914befe640d4aeafb8eb13134c29a6fe8ba57b013a7d3abe63941 size: 4310

> Task :sdks:java:testing:load-tests:run
Jul 20, 2021 12:48:35 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Jul 20, 2021 12:48:35 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 191 files. Enable logging at DEBUG level to see which files will be staged.
Jul 20, 2021 12:48:36 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Jul 20, 2021 12:48:36 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Jul 20, 2021 12:48:39 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Jul 20, 2021 12:48:39 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <112637 bytes, hash 5aa1afeeab873967d94ee4769ca84dbd3263cde611c61403f1d092d8dad5cdb0> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-WqGv7quHOWfZTuR2nKhNvTJjzeYRxhQD8dCS2NrVzbA.pb
Jul 20, 2021 12:48:40 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 191 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Jul 20, 2021 12:48:41 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 191 files cached, 0 files newly uploaded in 0 seconds
Jul 20, 2021 12:48:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Jul 20, 2021 12:48:41 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c3c4a71, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1352434e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4f9a6c2d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2b6fcb9f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@75de6341, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@74170687, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@68f0f72c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3d96fa9e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3b545206, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77bb48d5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@181d8899, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12d5c30e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@b887730, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26586b74, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@52f57666, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6e041285, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@267dc982, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@439b15f2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3aa41da1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@74fab04a]
Jul 20, 2021 12:48:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Jul 20, 2021 12:48:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Jul 20, 2021 12:48:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Jul 20, 2021 12:48:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Jul 20, 2021 12:48:41 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@44ed0a8f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@32177fa5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@a96d56c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6ab4a5b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2abe9173, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@235d29d6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1fdca564, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43f9dd56, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1d12e953, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@57cb70be, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2d4608a6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@20d87335, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2a8a4e0c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26c89563, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3bd6ba24, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58f437b0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@20f6f88c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4277127c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4c7e978c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@354e7004]
Jul 20, 2021 12:48:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Jul 20, 2021 12:48:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Jul 20, 2021 12:48:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Jul 20, 2021 12:48:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Jul 20, 2021 12:48:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Jul 20, 2021 12:48:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Jul 20, 2021 12:48:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Jul 20, 2021 12:48:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Jul 20, 2021 12:48:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Jul 20, 2021 12:48:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Jul 20, 2021 12:48:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Jul 20, 2021 12:48:41 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.32.0-SNAPSHOT
Jul 20, 2021 12:48:42 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-07-20_05_48_41-13777248125947355154?project=apache-beam-testing
Jul 20, 2021 12:48:42 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-07-20_05_48_41-13777248125947355154
Jul 20, 2021 12:48:42 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-07-20_05_48_41-13777248125947355154
Jul 20, 2021 12:48:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-07-20T12:48:49.293Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-07-tg76. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Jul 20, 2021 12:48:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-20T12:48:54.232Z: Worker configuration: e2-standard-2 in us-central1-a.
Jul 20, 2021 12:48:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-20T12:48:54.793Z: Expanding SplittableParDo operations into optimizable parts.
Jul 20, 2021 12:48:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-20T12:48:54.830Z: Expanding CollectionToSingleton operations into optimizable parts.
Jul 20, 2021 12:48:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-20T12:48:54.924Z: Expanding CoGroupByKey operations into optimizable parts.
Jul 20, 2021 12:48:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-20T12:48:55.091Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Jul 20, 2021 12:48:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-20T12:48:55.122Z: Expanding GroupByKey operations into streaming Read/Write steps
Jul 20, 2021 12:48:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-20T12:48:55.188Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jul 20, 2021 12:48:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-20T12:48:55.303Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jul 20, 2021 12:48:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-20T12:48:55.347Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Jul 20, 2021 12:48:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-20T12:48:55.379Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Jul 20, 2021 12:48:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-20T12:48:55.414Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Jul 20, 2021 12:48:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-20T12:48:55.445Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Jul 20, 2021 12:48:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-20T12:48:55.473Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jul 20, 2021 12:48:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-20T12:48:55.506Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jul 20, 2021 12:48:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-20T12:48:55.532Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jul 20, 2021 12:48:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-20T12:48:55.571Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Jul 20, 2021 12:48:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-20T12:48:55.608Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Jul 20, 2021 12:48:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-20T12:48:55.644Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Jul 20, 2021 12:48:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-20T12:48:55.676Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Jul 20, 2021 12:48:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-20T12:48:55.711Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jul 20, 2021 12:48:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-20T12:48:55.747Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jul 20, 2021 12:48:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-20T12:48:55.780Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jul 20, 2021 12:48:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-20T12:48:55.815Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Jul 20, 2021 12:48:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-20T12:48:55.846Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Jul 20, 2021 12:48:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-20T12:48:55.884Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Jul 20, 2021 12:48:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-20T12:48:55.921Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Jul 20, 2021 12:48:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-20T12:48:55.967Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Jul 20, 2021 12:48:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-20T12:48:56.001Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Jul 20, 2021 12:48:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-20T12:48:56.063Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Jul 20, 2021 12:48:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-20T12:48:56.102Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Jul 20, 2021 12:48:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-20T12:48:56.492Z: Starting 5 ****s in us-central1-a...
Jul 20, 2021 12:49:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-20T12:49:25.038Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jul 20, 2021 12:49:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-20T12:49:45.323Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Jul 20, 2021 12:50:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-20T12:50:34.403Z: Workers have started successfully.
Jul 20, 2021 12:50:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-20T12:50:34.437Z: Workers have started successfully.
Jul 20, 2021 4:00:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-20T16:00:39.194Z: Cancel request is committed for workflow job: 2021-07-20_05_48_41-13777248125947355154.
Jul 20, 2021 4:00:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-20T16:00:39.273Z: Cleaning up.
Jul 20, 2021 4:00:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-20T16:00:39.367Z: Stopping **** pool...
Jul 20, 2021 4:00:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-20T16:00:39.438Z: Stopping **** pool...
Jul 20, 2021 4:03:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-20T16:03:00.466Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Jul 20, 2021 4:03:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-20T16:03:00.512Z: Worker pool stopped.
Jul 20, 2021 4:03:08 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-07-20_05_48_41-13777248125947355154 finished with status CANCELLED.
Load test results for test (ID): a114d546-e5f4-4b74-bfd3-5dec9b673326 and timestamp: 2021-07-20T12:48:35.740000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11376.586
dataflow_v2_java11_total_bytes_count               7.2702688E9
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210720124333
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:adcb6c4a7a0914befe640d4aeafb8eb13134c29a6fe8ba57b013a7d3abe63941
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:adcb6c4a7a0914befe640d4aeafb8eb13134c29a6fe8ba57b013a7d3abe63941
  Associated tags:
 - 20210720124333
Tags:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210720124333
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210720124333].
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:adcb6c4a7a0914befe640d4aeafb8eb13134c29a6fe8ba57b013a7d3abe63941].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 51s
107 actionable tasks: 78 executed, 27 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/6gh5zodkf6qbw

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #32

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/32/display/redirect>

Changes:


------------------------------------------
[...truncated 65.65 KB...]
1c55082f6996: Preparing
21bed95af9d1: Preparing
f6e329fa7a21: Preparing
28ed330a4974: Preparing
3915db800ca6: Preparing
8cda1d8170c9: Preparing
30ff783beb17: Preparing
6dcd649e6a82: Preparing
f3aceae111b6: Preparing
8a0ae97220e2: Preparing
a0bdb1180144: Preparing
e414c4f63fd6: Preparing
6ea995e9b7d3: Preparing
c0848348e2f7: Preparing
30ff783beb17: Waiting
79c550eb7bd2: Preparing
7095af798ace: Preparing
6dcd649e6a82: Waiting
fe6a4fdbedc0: Preparing
e4d0e810d54a: Preparing
4e006334a6fd: Preparing
f3aceae111b6: Waiting
7095af798ace: Waiting
fe6a4fdbedc0: Waiting
6ea995e9b7d3: Waiting
e4d0e810d54a: Waiting
c0848348e2f7: Waiting
4e006334a6fd: Waiting
79c550eb7bd2: Waiting
a0bdb1180144: Waiting
8cda1d8170c9: Waiting
8a0ae97220e2: Waiting
e414c4f63fd6: Waiting
f6e329fa7a21: Pushed
3915db800ca6: Pushed
21bed95af9d1: Pushed
1c55082f6996: Pushed
28ed330a4974: Pushed
8cda1d8170c9: Pushed
6dcd649e6a82: Pushed
f3aceae111b6: Pushed
6ea995e9b7d3: Layer already exists
c0848348e2f7: Layer already exists
79c550eb7bd2: Layer already exists
7095af798ace: Layer already exists
fe6a4fdbedc0: Layer already exists
a0bdb1180144: Pushed
e4d0e810d54a: Layer already exists
4e006334a6fd: Layer already exists
e414c4f63fd6: Pushed
30ff783beb17: Pushed
8a0ae97220e2: Pushed
20210719124332: digest: sha256:2065e4982210ed032f1aca768a809f398772e52513c0370727e4ddcd1920c8aa size: 4310

> Task :sdks:java:testing:load-tests:run
Jul 19, 2021 12:48:10 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Jul 19, 2021 12:48:10 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 191 files. Enable logging at DEBUG level to see which files will be staged.
Jul 19, 2021 12:48:11 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Jul 19, 2021 12:48:11 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Jul 19, 2021 12:48:14 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Jul 19, 2021 12:48:14 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <112637 bytes, hash 0ae3fac772a680d3ba7d8846a61b0d4645fc3b1a2e61d22b34eb4ab7c0c3b646> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-CuP6x3KmgNO6fYhGphsNRkX8OxouYdIrNOtKt8DDtkY.pb
Jul 19, 2021 12:48:15 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 191 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Jul 19, 2021 12:48:16 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 191 files cached, 0 files newly uploaded in 0 seconds
Jul 19, 2021 12:48:16 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Jul 19, 2021 12:48:16 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@75de6341, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@74170687, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@68f0f72c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3d96fa9e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3b545206, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77bb48d5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@181d8899, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12d5c30e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@b887730, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26586b74, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@52f57666, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6e041285, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@267dc982, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@439b15f2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3aa41da1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@74fab04a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c83468e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@272ce069, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3d7fb838, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1c26273d]
Jul 19, 2021 12:48:16 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Jul 19, 2021 12:48:16 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Jul 19, 2021 12:48:16 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Jul 19, 2021 12:48:16 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Jul 19, 2021 12:48:16 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2abe9173, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@235d29d6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1fdca564, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43f9dd56, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1d12e953, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@57cb70be, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2d4608a6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@20d87335, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2a8a4e0c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26c89563, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3bd6ba24, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58f437b0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@20f6f88c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4277127c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4c7e978c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@354e7004, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@22ebccb9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5875de6a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@608fe01f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2687725a]
Jul 19, 2021 12:48:16 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Jul 19, 2021 12:48:16 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Jul 19, 2021 12:48:16 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Jul 19, 2021 12:48:16 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Jul 19, 2021 12:48:16 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Jul 19, 2021 12:48:16 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Jul 19, 2021 12:48:16 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Jul 19, 2021 12:48:16 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Jul 19, 2021 12:48:16 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Jul 19, 2021 12:48:16 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Jul 19, 2021 12:48:16 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Jul 19, 2021 12:48:16 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.32.0-SNAPSHOT
Jul 19, 2021 12:48:17 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-07-19_05_48_16-18360856457159799989?project=apache-beam-testing
Jul 19, 2021 12:48:17 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-07-19_05_48_16-18360856457159799989
Jul 19, 2021 12:48:17 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-07-19_05_48_16-18360856457159799989
Jul 19, 2021 12:48:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-07-19T12:48:24.053Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-07-w01x. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Jul 19, 2021 12:48:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-19T12:48:28.686Z: Worker configuration: e2-standard-2 in us-central1-a.
Jul 19, 2021 12:48:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-19T12:48:29.485Z: Expanding SplittableParDo operations into optimizable parts.
Jul 19, 2021 12:48:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-19T12:48:29.517Z: Expanding CollectionToSingleton operations into optimizable parts.
Jul 19, 2021 12:48:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-19T12:48:29.577Z: Expanding CoGroupByKey operations into optimizable parts.
Jul 19, 2021 12:48:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-19T12:48:29.627Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Jul 19, 2021 12:48:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-19T12:48:29.656Z: Expanding GroupByKey operations into streaming Read/Write steps
Jul 19, 2021 12:48:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-19T12:48:29.730Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jul 19, 2021 12:48:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-19T12:48:29.833Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jul 19, 2021 12:48:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-19T12:48:29.871Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Jul 19, 2021 12:48:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-19T12:48:29.904Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Jul 19, 2021 12:48:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-19T12:48:29.927Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Jul 19, 2021 12:48:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-19T12:48:29.948Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Jul 19, 2021 12:48:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-19T12:48:29.983Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jul 19, 2021 12:48:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-19T12:48:30.019Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jul 19, 2021 12:48:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-19T12:48:30.050Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jul 19, 2021 12:48:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-19T12:48:30.086Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Jul 19, 2021 12:48:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-19T12:48:30.116Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Jul 19, 2021 12:48:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-19T12:48:30.153Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Jul 19, 2021 12:48:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-19T12:48:30.189Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Jul 19, 2021 12:48:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-19T12:48:30.221Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jul 19, 2021 12:48:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-19T12:48:30.257Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jul 19, 2021 12:48:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-19T12:48:30.292Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jul 19, 2021 12:48:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-19T12:48:30.322Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Jul 19, 2021 12:48:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-19T12:48:30.356Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Jul 19, 2021 12:48:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-19T12:48:30.410Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Jul 19, 2021 12:48:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-19T12:48:30.448Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Jul 19, 2021 12:48:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-19T12:48:30.487Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Jul 19, 2021 12:48:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-19T12:48:30.512Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Jul 19, 2021 12:48:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-19T12:48:30.545Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Jul 19, 2021 12:48:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-19T12:48:30.567Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Jul 19, 2021 12:48:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-19T12:48:30.885Z: Starting 5 ****s in us-central1-a...
Jul 19, 2021 12:48:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-19T12:48:54.893Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jul 19, 2021 12:49:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-19T12:49:15.609Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Jul 19, 2021 12:50:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-19T12:50:16.714Z: Workers have started successfully.
Jul 19, 2021 12:50:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-19T12:50:16.742Z: Workers have started successfully.
Jul 19, 2021 4:00:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-19T16:00:37.857Z: Cancel request is committed for workflow job: 2021-07-19_05_48_16-18360856457159799989.
Jul 19, 2021 4:00:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-19T16:00:37.914Z: Cleaning up.
Jul 19, 2021 4:00:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-19T16:00:37.996Z: Stopping **** pool...
Jul 19, 2021 4:00:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-19T16:00:38.041Z: Stopping **** pool...
Jul 19, 2021 4:03:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-19T16:03:09.224Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Jul 19, 2021 4:03:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-19T16:03:09.260Z: Worker pool stopped.
Jul 19, 2021 4:03:16 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-07-19_05_48_16-18360856457159799989 finished with status CANCELLED.
Load test results for test (ID): d30c874b-e32a-4409-be8b-e66d0da3ed7a and timestamp: 2021-07-19T12:48:11.104000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11406.172
dataflow_v2_java11_total_bytes_count               6.3569824E9
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210719124332
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:2065e4982210ed032f1aca768a809f398772e52513c0370727e4ddcd1920c8aa
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:2065e4982210ed032f1aca768a809f398772e52513c0370727e4ddcd1920c8aa
  Associated tags:
 - 20210719124332
Tags:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210719124332
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210719124332].
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:2065e4982210ed032f1aca768a809f398772e52513c0370727e4ddcd1920c8aa].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 20m 1s
107 actionable tasks: 76 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/iccwxsnpwugq6

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #31

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/31/display/redirect>

Changes:


------------------------------------------
[...truncated 65.65 KB...]
b6f1e2d38e2f: Preparing
6051c5bed6b7: Preparing
6ede542f0478: Preparing
706d8eb1ff60: Preparing
a2fca52c6a8a: Preparing
d08575abb294: Preparing
f251e24bea9a: Preparing
8ece28f396ff: Preparing
813b1f0074dc: Preparing
0c7ab0fa1d8d: Preparing
bc625cdeb393: Preparing
a82e325aa510: Preparing
6ea995e9b7d3: Preparing
c0848348e2f7: Preparing
79c550eb7bd2: Preparing
7095af798ace: Preparing
fe6a4fdbedc0: Preparing
e4d0e810d54a: Preparing
4e006334a6fd: Preparing
8ece28f396ff: Waiting
4e006334a6fd: Waiting
813b1f0074dc: Waiting
fe6a4fdbedc0: Waiting
79c550eb7bd2: Waiting
e4d0e810d54a: Waiting
7095af798ace: Waiting
a82e325aa510: Waiting
6ea995e9b7d3: Waiting
d08575abb294: Waiting
f251e24bea9a: Waiting
bc625cdeb393: Waiting
0c7ab0fa1d8d: Waiting
c0848348e2f7: Waiting
6ede542f0478: Pushed
6051c5bed6b7: Pushed
a2fca52c6a8a: Pushed
b6f1e2d38e2f: Pushed
d08575abb294: Pushed
706d8eb1ff60: Pushed
8ece28f396ff: Pushed
813b1f0074dc: Pushed
6ea995e9b7d3: Layer already exists
c0848348e2f7: Layer already exists
79c550eb7bd2: Layer already exists
bc625cdeb393: Pushed
7095af798ace: Layer already exists
fe6a4fdbedc0: Layer already exists
e4d0e810d54a: Layer already exists
4e006334a6fd: Layer already exists
a82e325aa510: Pushed
f251e24bea9a: Pushed
0c7ab0fa1d8d: Pushed
20210718124333: digest: sha256:80e3b4322aed089881be9fe13aeda52d9fb08280c60b5ed7d80f5114a9d8910b size: 4310

> Task :sdks:java:testing:load-tests:run
Jul 18, 2021 12:48:21 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Jul 18, 2021 12:48:21 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 191 files. Enable logging at DEBUG level to see which files will be staged.
Jul 18, 2021 12:48:22 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Jul 18, 2021 12:48:22 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Jul 18, 2021 12:48:25 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Jul 18, 2021 12:48:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <112637 bytes, hash 68f08c8f7295a5382ea1735e9182a6354317f50a012ccc2c6f48f02206aa48cb> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-aPCMj3KVpTguoXNekYKmNUMX9QoBLMwsb0jwIgaqSMs.pb
Jul 18, 2021 12:48:27 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 191 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Jul 18, 2021 12:48:27 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 191 files cached, 0 files newly uploaded in 0 seconds
Jul 18, 2021 12:48:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Jul 18, 2021 12:48:27 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1352434e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4f9a6c2d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2b6fcb9f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@75de6341, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@74170687, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@68f0f72c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3d96fa9e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3b545206, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77bb48d5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@181d8899, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12d5c30e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@b887730, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26586b74, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@52f57666, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6e041285, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@267dc982, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@439b15f2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3aa41da1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@74fab04a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c83468e]
Jul 18, 2021 12:48:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Jul 18, 2021 12:48:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Jul 18, 2021 12:48:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Jul 18, 2021 12:48:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Jul 18, 2021 12:48:27 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@32177fa5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@a96d56c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6ab4a5b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2abe9173, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@235d29d6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1fdca564, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43f9dd56, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1d12e953, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@57cb70be, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2d4608a6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@20d87335, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2a8a4e0c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26c89563, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3bd6ba24, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58f437b0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@20f6f88c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4277127c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4c7e978c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@354e7004, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@22ebccb9]
Jul 18, 2021 12:48:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Jul 18, 2021 12:48:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Jul 18, 2021 12:48:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Jul 18, 2021 12:48:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Jul 18, 2021 12:48:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Jul 18, 2021 12:48:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Jul 18, 2021 12:48:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Jul 18, 2021 12:48:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Jul 18, 2021 12:48:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Jul 18, 2021 12:48:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Jul 18, 2021 12:48:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Jul 18, 2021 12:48:28 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.32.0-SNAPSHOT
Jul 18, 2021 12:48:28 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-07-18_05_48_28-17961249875087085063?project=apache-beam-testing
Jul 18, 2021 12:48:28 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-07-18_05_48_28-17961249875087085063
Jul 18, 2021 12:48:28 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-07-18_05_48_28-17961249875087085063
Jul 18, 2021 12:48:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-07-18T12:48:35.341Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-07-2yqn. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Jul 18, 2021 12:48:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-18T12:48:40.065Z: Worker configuration: e2-standard-2 in us-central1-a.
Jul 18, 2021 12:48:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-18T12:48:40.743Z: Expanding SplittableParDo operations into optimizable parts.
Jul 18, 2021 12:48:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-18T12:48:40.792Z: Expanding CollectionToSingleton operations into optimizable parts.
Jul 18, 2021 12:48:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-18T12:48:40.849Z: Expanding CoGroupByKey operations into optimizable parts.
Jul 18, 2021 12:48:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-18T12:48:40.920Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Jul 18, 2021 12:48:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-18T12:48:40.949Z: Expanding GroupByKey operations into streaming Read/Write steps
Jul 18, 2021 12:48:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-18T12:48:41.007Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jul 18, 2021 12:48:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-18T12:48:41.113Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jul 18, 2021 12:48:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-18T12:48:41.147Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Jul 18, 2021 12:48:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-18T12:48:41.173Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Jul 18, 2021 12:48:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-18T12:48:41.198Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Jul 18, 2021 12:48:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-18T12:48:41.227Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Jul 18, 2021 12:48:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-18T12:48:41.252Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jul 18, 2021 12:48:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-18T12:48:41.286Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jul 18, 2021 12:48:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-18T12:48:41.319Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jul 18, 2021 12:48:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-18T12:48:41.345Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Jul 18, 2021 12:48:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-18T12:48:41.379Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Jul 18, 2021 12:48:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-18T12:48:41.406Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Jul 18, 2021 12:48:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-18T12:48:41.439Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Jul 18, 2021 12:48:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-18T12:48:41.474Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jul 18, 2021 12:48:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-18T12:48:41.529Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jul 18, 2021 12:48:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-18T12:48:41.561Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jul 18, 2021 12:48:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-18T12:48:41.589Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Jul 18, 2021 12:48:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-18T12:48:41.619Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Jul 18, 2021 12:48:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-18T12:48:41.644Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Jul 18, 2021 12:48:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-18T12:48:41.679Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Jul 18, 2021 12:48:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-18T12:48:41.712Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Jul 18, 2021 12:48:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-18T12:48:41.737Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Jul 18, 2021 12:48:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-18T12:48:41.769Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Jul 18, 2021 12:48:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-18T12:48:41.795Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Jul 18, 2021 12:48:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-18T12:48:42.124Z: Starting 5 ****s in us-central1-a...
Jul 18, 2021 12:48:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-18T12:48:49.962Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jul 18, 2021 12:49:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-18T12:49:29.475Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Jul 18, 2021 12:50:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-18T12:50:17.601Z: Workers have started successfully.
Jul 18, 2021 12:50:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-18T12:50:17.637Z: Workers have started successfully.
Jul 18, 2021 4:00:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-18T16:00:32.201Z: Cancel request is committed for workflow job: 2021-07-18_05_48_28-17961249875087085063.
Jul 18, 2021 4:00:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-18T16:00:32.265Z: Cleaning up.
Jul 18, 2021 4:00:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-18T16:00:32.346Z: Stopping **** pool...
Jul 18, 2021 4:00:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-18T16:00:32.421Z: Stopping **** pool...
Jul 18, 2021 4:02:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-18T16:02:52.840Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Jul 18, 2021 4:02:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-18T16:02:52.871Z: Worker pool stopped.
Jul 18, 2021 4:03:00 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-07-18_05_48_28-17961249875087085063 finished with status CANCELLED.
Load test results for test (ID): dc9e8764-7149-41f8-a15c-1fa95b78f99f and timestamp: 2021-07-18T12:48:22.202000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11380.011
dataflow_v2_java11_total_bytes_count               6.9648185E9
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210718124333
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:80e3b4322aed089881be9fe13aeda52d9fb08280c60b5ed7d80f5114a9d8910b
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:80e3b4322aed089881be9fe13aeda52d9fb08280c60b5ed7d80f5114a9d8910b
  Associated tags:
 - 20210718124333
Tags:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210718124333
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210718124333].
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:80e3b4322aed089881be9fe13aeda52d9fb08280c60b5ed7d80f5114a9d8910b].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 42s
107 actionable tasks: 76 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/varvztsgradmk

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #30

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/30/display/redirect?page=changes>

Changes:

[Luke Cwik] [BEAM-12596] Ensure that SDFs always report a size >= 0.

[noreply] [BEAM-2914] Add TimestampPrefixingWindowCoder as

[noreply] [BEAM-11934] Remove Dataflow override of streaming WriteFiles with


------------------------------------------
[...truncated 66.32 KB...]
fe6a4fdbedc0: Waiting
e4d0e810d54a: Waiting
79c550eb7bd2: Waiting
bc6c81dcc6ca: Waiting
a92926ced994: Waiting
dda53d9c1a11: Pushed
a073c68bb67d: Pushed
c8440f3187d9: Pushed
aa442e406acb: Pushed
a92926ced994: Pushed
b7a3db87e2c9: Pushed
bce8bcb5d0c4: Pushed
ad280a8d6c30: Pushed
6ea995e9b7d3: Layer already exists
c0848348e2f7: Layer already exists
79c550eb7bd2: Layer already exists
7095af798ace: Layer already exists
bc6c81dcc6ca: Pushed
fe6a4fdbedc0: Layer already exists
e4d0e810d54a: Layer already exists
6dffe5eabcc0: Pushed
4e006334a6fd: Layer already exists
5bef82b88264: Pushed
146a0de8a36f: Pushed
20210717124332: digest: sha256:5484b61b93b8e0706b0cea3cdffe3855eb244da5593d741a6089bb44ed7cbe42 size: 4310

> Task :sdks:java:testing:load-tests:run
Jul 17, 2021 12:48:24 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Jul 17, 2021 12:48:25 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 191 files. Enable logging at DEBUG level to see which files will be staged.
Jul 17, 2021 12:48:26 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Jul 17, 2021 12:48:26 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Jul 17, 2021 12:48:28 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Jul 17, 2021 12:48:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <112699 bytes, hash 3498cdc7336301c0e6edb693710bf9cbc0a5fb32d72e02c9b6e33542a24de106> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-NJjNxzNjAcDm7baTcQv5y8Cl-zLXLgLJtuM1QqJN4QY.pb
Jul 17, 2021 12:48:29 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 191 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Jul 17, 2021 12:48:30 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 191 files cached, 0 files newly uploaded in 0 seconds
Jul 17, 2021 12:48:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Jul 17, 2021 12:48:30 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c3c4a71, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1352434e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4f9a6c2d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2b6fcb9f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@75de6341, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@74170687, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@68f0f72c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3d96fa9e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3b545206, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77bb48d5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@181d8899, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12d5c30e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@b887730, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26586b74, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@52f57666, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6e041285, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@267dc982, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@439b15f2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3aa41da1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@74fab04a]
Jul 17, 2021 12:48:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Jul 17, 2021 12:48:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Jul 17, 2021 12:48:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Jul 17, 2021 12:48:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Jul 17, 2021 12:48:30 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@44ed0a8f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@32177fa5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@a96d56c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6ab4a5b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2abe9173, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@235d29d6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1fdca564, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43f9dd56, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1d12e953, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@57cb70be, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2d4608a6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@20d87335, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2a8a4e0c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26c89563, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3bd6ba24, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58f437b0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@20f6f88c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4277127c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4c7e978c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@354e7004]
Jul 17, 2021 12:48:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Jul 17, 2021 12:48:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Jul 17, 2021 12:48:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Jul 17, 2021 12:48:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Jul 17, 2021 12:48:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Jul 17, 2021 12:48:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Jul 17, 2021 12:48:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Jul 17, 2021 12:48:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Jul 17, 2021 12:48:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Jul 17, 2021 12:48:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Jul 17, 2021 12:48:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Jul 17, 2021 12:48:30 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.32.0-SNAPSHOT
Jul 17, 2021 12:48:31 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-07-17_05_48_30-1666522210892877877?project=apache-beam-testing
Jul 17, 2021 12:48:31 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-07-17_05_48_30-1666522210892877877
Jul 17, 2021 12:48:31 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-07-17_05_48_30-1666522210892877877
Jul 17, 2021 12:48:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-07-17T12:48:36.614Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-07-52wi. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Jul 17, 2021 12:48:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-17T12:48:41.187Z: Worker configuration: e2-standard-2 in us-central1-a.
Jul 17, 2021 12:48:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-17T12:48:41.826Z: Expanding SplittableParDo operations into optimizable parts.
Jul 17, 2021 12:48:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-17T12:48:41.849Z: Expanding CollectionToSingleton operations into optimizable parts.
Jul 17, 2021 12:48:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-17T12:48:41.913Z: Expanding CoGroupByKey operations into optimizable parts.
Jul 17, 2021 12:48:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-17T12:48:41.978Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Jul 17, 2021 12:48:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-17T12:48:42.011Z: Expanding GroupByKey operations into streaming Read/Write steps
Jul 17, 2021 12:48:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-17T12:48:42.068Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jul 17, 2021 12:48:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-17T12:48:42.176Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jul 17, 2021 12:48:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-17T12:48:42.199Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Jul 17, 2021 12:48:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-17T12:48:42.233Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Jul 17, 2021 12:48:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-17T12:48:42.277Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Jul 17, 2021 12:48:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-17T12:48:42.326Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Jul 17, 2021 12:48:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-17T12:48:42.351Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jul 17, 2021 12:48:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-17T12:48:42.376Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jul 17, 2021 12:48:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-17T12:48:42.412Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jul 17, 2021 12:48:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-17T12:48:42.445Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Jul 17, 2021 12:48:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-17T12:48:42.469Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Jul 17, 2021 12:48:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-17T12:48:42.498Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Jul 17, 2021 12:48:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-17T12:48:42.534Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Jul 17, 2021 12:48:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-17T12:48:42.558Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jul 17, 2021 12:48:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-17T12:48:42.589Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jul 17, 2021 12:48:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-17T12:48:42.636Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jul 17, 2021 12:48:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-17T12:48:42.672Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Jul 17, 2021 12:48:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-17T12:48:42.699Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Jul 17, 2021 12:48:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-17T12:48:42.731Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Jul 17, 2021 12:48:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-17T12:48:42.765Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Jul 17, 2021 12:48:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-17T12:48:42.796Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Jul 17, 2021 12:48:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-17T12:48:42.831Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Jul 17, 2021 12:48:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-17T12:48:42.877Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Jul 17, 2021 12:48:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-17T12:48:42.912Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Jul 17, 2021 12:48:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-17T12:48:43.327Z: Starting 5 ****s in us-central1-a...
Jul 17, 2021 12:49:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-17T12:49:00.468Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jul 17, 2021 12:49:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-17T12:49:27.771Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Jul 17, 2021 12:50:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-17T12:50:17.169Z: Workers have started successfully.
Jul 17, 2021 12:50:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-17T12:50:17.196Z: Workers have started successfully.
Jul 17, 2021 4:00:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-17T16:00:34.933Z: Cancel request is committed for workflow job: 2021-07-17_05_48_30-1666522210892877877.
Jul 17, 2021 4:00:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-17T16:00:34.986Z: Cleaning up.
Jul 17, 2021 4:00:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-17T16:00:35.061Z: Stopping **** pool...
Jul 17, 2021 4:00:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-17T16:00:35.123Z: Stopping **** pool...
Jul 17, 2021 4:02:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-17T16:02:53.154Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Jul 17, 2021 4:02:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-17T16:02:53.190Z: Worker pool stopped.
Jul 17, 2021 4:02:58 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-07-17_05_48_30-1666522210892877877 finished with status CANCELLED.
Load test results for test (ID): 3f046189-2045-4be5-861b-40e5e81ed5a4 and timestamp: 2021-07-17T12:48:25.635000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11389.307
dataflow_v2_java11_total_bytes_count               8.1641618E9
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210717124332
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:5484b61b93b8e0706b0cea3cdffe3855eb244da5593d741a6089bb44ed7cbe42
Deleted: sha256:95efd1e876d9074ed650ba0da86ff0e6ea80312bc92288ba729b05ea094aacdc
Deleted: sha256:b8b0e76e3954958f89cdc4215a8e386370e4825569265bd787f4612649b4fe37
Deleted: sha256:ad125c9db6685625db5b663cff330ee4e7bf5193a5efadef5e9d13199cd2d0d2
Deleted: sha256:8a2c6041fafad8302e701966cc3ad30423c933997a3ec1cc9454b38f287164be
Deleted: sha256:2e172a836a5ed046097567a4022f39fa33ff4eacc5a3584ee13ba0ace77b5a1b
Deleted: sha256:b3bdea5b36051c0f9ff0e062ead1cb1163592d6ded7c52826ab2037abc2e91c3
Deleted: sha256:13e19e8bfa32fdf34d086806d323cf8addcf2a7dd41504555a5a420d6d81b0d3
Deleted: sha256:6e0bf9fe0e692474d4a2df1af6a6f1bda1530bf93e2ed3ae04e218992bf56d17
Deleted: sha256:75b0cfebf48fa011616abcc86d294d5bbc31e39501424a7333e7730eb6a6d491
Deleted: sha256:e14257f92c27cc718a492009f493c5f8fcedcead741bc4f11a345a501e1286a9
Deleted: sha256:a38390176b20278c769e1e4dbc164ab2ee42d7616c0b97a641199f30fd6dc088
Deleted: sha256:1b81df8e109727710dd89e372f70eabb07d2c96ecc1d49e708e5e257e01e3e2d
Deleted: sha256:09373127cb7bf9a916288ad343e70a3954601d5b9cbf4812d3af34cf2abc29e0
Deleted: sha256:a550a646b8e5e659588004716b3f20afa1e5f6cd4406ab0f2bd25d395d1d28ed
Deleted: sha256:f27ac035b43d830648b4eca0027a38ca2028372f7a3035e687f426a238e3147f
Deleted: sha256:87cd4a9f34dfb78b59c774816b77fc2b3ef1d1facde990823181de1525f7b495
Deleted: sha256:f2f390434ed4f425691136ace14d14035484cbf950d6bd8c3bf31839f732f76b
Deleted: sha256:f4d93ac65f1fa50f897d567682989cdf4ffa42db04d903459dba54ea4ed0fbe1
Deleted: sha256:9945a3ec210cff6aac995ec85777191aa79939bb7e55d6a2dacfa84f08299086
Deleted: sha256:61d4ba18a67c06efb7964949ddf6aed6b9ac9721c8cc2762f121b5895384c6c1
Deleted: sha256:59f4c872e2ea9dd6ec8508c73cf5ddc0760187426630d3834f96123683d6c150
Deleted: sha256:63c6d73222b646086677ccb77c926b10b3f53dc0eefe209354a0bba5b6b91cfe
Deleted: sha256:366a34d48858ee4c973c1b0939e49e34b759f30d776e54af90c9843197a7aefc
Deleted: sha256:72c37679eec0c0af0bbda8adb601fbc723aa65699ac9ef3cf18f6dce587a675a
Deleted: sha256:966844de7ed7f5d3f21311c790be38757f2573018e592bfb1f1ca1dac97393a7
Deleted: sha256:11a29667e908ae9d3d8d018d5cec7a8c2e8dba8dc494810268736b4d255e9740
Deleted: sha256:b22f5c5eec1c06ce2b5a2a4df5b43b6320a36f12ee8044ac809c16dd28c94a96
Deleted: sha256:988de39561020a3f5827aaa7044f4ff4f7fb1d2ab78ea19dae2a666c0575114d
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:5484b61b93b8e0706b0cea3cdffe3855eb244da5593d741a6089bb44ed7cbe42
  Associated tags:
 - 20210717124332
Tags:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210717124332
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210717124332].
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:5484b61b93b8e0706b0cea3cdffe3855eb244da5593d741a6089bb44ed7cbe42].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 42s
107 actionable tasks: 78 executed, 27 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/hlb2n4iywxuje

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #29

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/29/display/redirect?page=changes>

Changes:

[Pablo Estrada] BigQueryIO returns successful rows for streaming inserts.

[Pablo Estrada] fixup

[Pablo Estrada] Fixup

[Pablo Estrada] Address comment on method versions

[Jan Lukavský] Fix help of apache_beam.trigger.

[rohde.samuel] [BEAM-12388] Add caching to deferred dataframes

[noreply] [BEAM-4152] Finish plumbing for session windowing in Go SDK (#15168)

[noreply] [BEAM-12626] Reject session windows in side inputs (#15173)


------------------------------------------
[...truncated 57.79 KB...]
INFO:root:pull_licenses_java.py succeed. It took 5.136712 seconds with 16 threads.

> Task :sdks:java:container:java11:copyJavaThirdPartyLicenses NO-SOURCE

> Task :sdks:go:resolveBuildDependencies
Resolving golang.org/x/text: commit='23ae387dee1f90d29a23c0e87ee0b46038fbed0e', urls=[https://go.googlesource.com/text]
Resolving cached github.com/etcd-io/etcd: commit='11214aa33bf5a47d3d9d8dafe0f6b97237dfe921', urls=[https://github.com/etcd-io/etcd.git, git@github.com:etcd-io/etcd.git]

> Task :release:go-licenses:java:dockerRun
+ go-licenses save github.com/apache/beam/sdks/java/container --save_path=/output/licenses
+ go-licenses csv github.com/apache/beam/sdks/java/container
+ tee /output/licenses/list.csv
github.com/apache/beam/sdks/java/container,https://github.com/apache/beam/blob/master/LICENSE,Apache-2.0
github.com/apache/beam/sdks/go/pkg/beam,https://github.com/apache/beam/blob/master/sdks/go/README.md,Apache-2.0
google.golang.org/protobuf,https://go.googlesource.com/protobuf/+/refs/heads/master/LICENSE,BSD-3-Clause
golang.org/x/net,https://go.googlesource.com/net/+/refs/heads/master/LICENSE,BSD-3-Clause
golang.org/x/sys,https://go.googlesource.com/sys/+/refs/heads/master/LICENSE,BSD-3-Clause
github.com/golang/protobuf,https://github.com/golang/protobuf/blob/master/LICENSE,BSD-3-Clause
google.golang.org/grpc,https://github.com/grpc/grpc-go/blob/master/LICENSE,Apache-2.0
google.golang.org/genproto/googleapis/rpc/status,https://github.com/googleapis/go-genproto/blob/master/LICENSE,Apache-2.0
golang.org/x/text,https://go.googlesource.com/text/+/refs/heads/master/LICENSE,BSD-3-Clause
+ chmod -R a+w /output/licenses

> Task :release:go-licenses:java:createLicenses
> Task :sdks:java:container:java11:copyGolangLicenses

> Task :sdks:go:resolveBuildDependencies
Resolving google.golang.org/api: commit='0324d5e90dc7753607860272666845fad9ceb97e', urls=[https://code.googlesource.com/google-api-go-client]
Resolving google.golang.org/genproto: commit='4d944d34d83c502a5f761500a14d8842648415c3', urls=[https://github.com/google/go-genproto]
Resolving google.golang.org/grpc: commit='5e8f83304c0563d1ba74db05fee83d9c18ab9a58', urls=[https://github.com/grpc/grpc-go]
Resolving google.golang.org/protobuf: commit='d165be301fb1e13390ad453281ded24385fd8ebc', urls=[https://go.googlesource.com/protobuf]
Resolving cached github.com/etcd-io/etcd: commit='11214aa33bf5a47d3d9d8dafe0f6b97237dfe921', urls=[https://github.com/etcd-io/etcd.git, git@github.com:etcd-io/etcd.git]
Resolving cached github.com/etcd-io/etcd: commit='11214aa33bf5a47d3d9d8dafe0f6b97237dfe921', urls=[https://github.com/etcd-io/etcd.git, git@github.com:etcd-io/etcd.git]

> Task :sdks:go:installDependencies
> Task :sdks:go:buildLinuxAmd64
> Task :sdks:go:goBuild

> Task :sdks:java:container:resolveBuildDependencies
Resolving ./github.com/apache/beam/sdks/go@<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src/sdks/go>

> Task :sdks:java:container:installDependencies
> Task :sdks:java:container:buildLinuxAmd64
> Task :sdks:java:container:goBuild
> Task :sdks:java:container:java11:copySdkHarnessLauncher
> Task :sdks:java:container:java11:dockerPrepare
> Task :sdks:java:container:java11:docker

> Task :runners:google-cloud-dataflow-java:buildAndPushDockerContainer
WARNING: `gcloud docker` will not be supported for Docker client versions above 18.03.

As an alternative, use `gcloud auth configure-docker` to configure `docker` to
use `gcloud` as a credential helper, then use `docker` as you would for non-GCR
registries, e.g. `docker pull gcr.io/project-id/my-image`. Add
`--verbosity=error` to silence this warning: `gcloud docker
--verbosity=error -- pull gcr.io/project-id/my-image`.

See: https://cloud.google.com/container-registry/docs/support/deprecation-notices#gcloud-docker


> Task :sdks:java:testing:load-tests:run
Jul 16, 2021 12:48:16 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Jul 16, 2021 12:48:17 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 191 files. Enable logging at DEBUG level to see which files will be staged.
Jul 16, 2021 12:48:17 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Jul 16, 2021 12:48:17 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Jul 16, 2021 12:48:20 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Jul 16, 2021 12:48:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <112608 bytes, hash 04c83710f8b266d086bf4d2b6b35b9acec71f52ea4a066e3b8ff774348588e91> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-BMg3EPiyZtCGv00razW5rOxx9S6koGbjuP93Q0hYjpE.pb
Jul 16, 2021 12:48:21 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 191 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Jul 16, 2021 12:48:22 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 191 files cached, 0 files newly uploaded in 0 seconds
Jul 16, 2021 12:48:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Jul 16, 2021 12:48:22 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4f9a6c2d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2b6fcb9f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@75de6341, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@74170687, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@68f0f72c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3d96fa9e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3b545206, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77bb48d5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@181d8899, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12d5c30e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@b887730, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26586b74, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@52f57666, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6e041285, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@267dc982, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@439b15f2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3aa41da1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@74fab04a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c83468e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@272ce069]
Jul 16, 2021 12:48:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Jul 16, 2021 12:48:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Jul 16, 2021 12:48:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Jul 16, 2021 12:48:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Jul 16, 2021 12:48:22 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@44ed0a8f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@32177fa5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@a96d56c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6ab4a5b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2abe9173, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@235d29d6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1fdca564, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43f9dd56, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1d12e953, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@57cb70be, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2d4608a6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@20d87335, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2a8a4e0c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26c89563, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3bd6ba24, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58f437b0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@20f6f88c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4277127c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4c7e978c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@354e7004]
Jul 16, 2021 12:48:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Jul 16, 2021 12:48:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Jul 16, 2021 12:48:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Jul 16, 2021 12:48:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Jul 16, 2021 12:48:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Jul 16, 2021 12:48:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Jul 16, 2021 12:48:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Jul 16, 2021 12:48:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Jul 16, 2021 12:48:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Jul 16, 2021 12:48:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Jul 16, 2021 12:48:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Jul 16, 2021 12:48:22 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.32.0-SNAPSHOT
Jul 16, 2021 12:48:23 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-07-16_05_48_22-13047063686675425281?project=apache-beam-testing
Jul 16, 2021 12:48:23 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-07-16_05_48_22-13047063686675425281
Jul 16, 2021 12:48:23 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-07-16_05_48_22-13047063686675425281
Jul 16, 2021 12:48:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-07-16T12:48:29.971Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-07-ehy2. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Jul 16, 2021 12:48:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-16T12:48:32.700Z: Worker configuration: e2-standard-2 in us-central1-a.
Jul 16, 2021 12:48:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-16T12:48:33.387Z: Expanding SplittableParDo operations into optimizable parts.
Jul 16, 2021 12:48:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-16T12:48:33.420Z: Expanding CollectionToSingleton operations into optimizable parts.
Jul 16, 2021 12:48:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-16T12:48:33.480Z: Expanding CoGroupByKey operations into optimizable parts.
Jul 16, 2021 12:48:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-16T12:48:33.564Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Jul 16, 2021 12:48:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-16T12:48:33.585Z: Expanding GroupByKey operations into streaming Read/Write steps
Jul 16, 2021 12:48:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-16T12:48:33.644Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jul 16, 2021 12:48:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-16T12:48:33.734Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jul 16, 2021 12:48:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-16T12:48:33.762Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Jul 16, 2021 12:48:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-16T12:48:33.785Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Jul 16, 2021 12:48:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-16T12:48:33.808Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Jul 16, 2021 12:48:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-16T12:48:33.834Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Jul 16, 2021 12:48:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-16T12:48:33.862Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jul 16, 2021 12:48:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-16T12:48:33.897Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jul 16, 2021 12:48:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-16T12:48:33.925Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jul 16, 2021 12:48:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-16T12:48:33.952Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Jul 16, 2021 12:48:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-16T12:48:34.002Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Jul 16, 2021 12:48:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-16T12:48:34.034Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Jul 16, 2021 12:48:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-16T12:48:34.063Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Jul 16, 2021 12:48:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-16T12:48:34.092Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jul 16, 2021 12:48:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-16T12:48:34.118Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jul 16, 2021 12:48:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-16T12:48:34.141Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jul 16, 2021 12:48:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-16T12:48:34.170Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Jul 16, 2021 12:48:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-16T12:48:34.198Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Jul 16, 2021 12:48:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-16T12:48:34.223Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Jul 16, 2021 12:48:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-16T12:48:34.247Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Jul 16, 2021 12:48:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-16T12:48:34.277Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Jul 16, 2021 12:48:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-16T12:48:34.311Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Jul 16, 2021 12:48:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-16T12:48:34.345Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Jul 16, 2021 12:48:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-16T12:48:34.374Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Jul 16, 2021 12:48:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-16T12:48:34.714Z: Starting 5 ****s in us-central1-a...
Jul 16, 2021 12:48:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-16T12:48:52.573Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jul 16, 2021 12:49:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-16T12:49:14.291Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Jul 16, 2021 12:50:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-16T12:50:09.795Z: Workers have started successfully.
Jul 16, 2021 12:50:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-16T12:50:09.814Z: Workers have started successfully.
Jul 16, 2021 4:00:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-16T16:00:36.939Z: Cancel request is committed for workflow job: 2021-07-16_05_48_22-13047063686675425281.
Jul 16, 2021 4:00:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-16T16:00:37.881Z: Cleaning up.
Jul 16, 2021 4:00:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-16T16:00:38.009Z: Stopping **** pool...
Jul 16, 2021 4:00:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-16T16:00:38.054Z: Stopping **** pool...
Jul 16, 2021 4:02:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-16T16:02:56.886Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Jul 16, 2021 4:02:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-16T16:02:56.924Z: Worker pool stopped.
Jul 16, 2021 4:03:03 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-07-16_05_48_22-13047063686675425281 finished with status CANCELLED.
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:7403cb0ac3852b01b4da69b88fd272e1d2ed59de799f7d9a1994b4c40cf44dff
  Associated tags:
 - 20210716124331
Tags:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210716124331
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210716124331].
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:7403cb0ac3852b01b4da69b88fd272e1d2ed59de799f7d9a1994b4c40cf44dff].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 48s
107 actionable tasks: 77 executed, 28 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/g3he7jmr3rdrm

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #28

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/28/display/redirect?page=changes>

Changes:

[kawaigin] Misc Fixes

[noreply] [BEAM-12611] Add Instruction ID to LogEntry, by introducing

[kawaigin] Updated screendiff golden screenshots for Linux platforms.


------------------------------------------
[...truncated 66.42 KB...]
c4bf6f1574bf: Waiting
951c21529689: Waiting
7095af798ace: Waiting
72003877c3a3: Waiting
fe6a4fdbedc0: Waiting
e4d0e810d54a: Waiting
6ea995e9b7d3: Waiting
c3fe4c6bb6ce: Waiting
c0848348e2f7: Waiting
4e006334a6fd: Waiting
79c550eb7bd2: Waiting
2b78455c9c72: Waiting
6d5ca3b8bbb3: Waiting
53a2dd4985e2: Pushed
a9f9367bc021: Pushed
ccff875018a2: Pushed
2b78455c9c72: Pushed
3e725991a72c: Pushed
29d862d95c65: Pushed
951c21529689: Pushed
69295495e1c9: Pushed
6ea995e9b7d3: Layer already exists
c0848348e2f7: Layer already exists
79c550eb7bd2: Layer already exists
c3fe4c6bb6ce: Pushed
c4bf6f1574bf: Pushed
7095af798ace: Layer already exists
fe6a4fdbedc0: Layer already exists
e4d0e810d54a: Layer already exists
4e006334a6fd: Layer already exists
6d5ca3b8bbb3: Pushed
72003877c3a3: Pushed
20210715124335: digest: sha256:94342c9bc3002d205d475301742f8f9f2469cbf1b24912fd818d91641d8a9ca1 size: 4310

> Task :sdks:java:testing:load-tests:run
Jul 15, 2021 12:48:43 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Jul 15, 2021 12:48:44 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 191 files. Enable logging at DEBUG level to see which files will be staged.
Jul 15, 2021 12:48:45 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Jul 15, 2021 12:48:45 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Jul 15, 2021 12:48:48 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Jul 15, 2021 12:48:48 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <112608 bytes, hash 5522a8330f2829db2b643af5af817fecb9149bdfa03924cf5ec3f8d183a59661> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-VSKoMw8oKdsrZDr1r4F_7LkUm9-gOSTPXsP40YOllmE.pb
Jul 15, 2021 12:48:49 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 191 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Jul 15, 2021 12:48:50 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 191 files cached, 0 files newly uploaded in 0 seconds
Jul 15, 2021 12:48:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Jul 15, 2021 12:48:50 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4f9a6c2d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2b6fcb9f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@75de6341, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@74170687, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@68f0f72c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3d96fa9e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3b545206, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77bb48d5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@181d8899, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12d5c30e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@b887730, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26586b74, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@52f57666, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6e041285, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@267dc982, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@439b15f2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3aa41da1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@74fab04a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c83468e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@272ce069]
Jul 15, 2021 12:48:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Jul 15, 2021 12:48:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Jul 15, 2021 12:48:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Jul 15, 2021 12:48:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Jul 15, 2021 12:48:50 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@574b7f4a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@44ed0a8f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@32177fa5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@a96d56c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6ab4a5b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2abe9173, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@235d29d6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1fdca564, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43f9dd56, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1d12e953, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@57cb70be, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2d4608a6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@20d87335, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2a8a4e0c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26c89563, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3bd6ba24, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58f437b0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@20f6f88c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4277127c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4c7e978c]
Jul 15, 2021 12:48:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Jul 15, 2021 12:48:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Jul 15, 2021 12:48:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Jul 15, 2021 12:48:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Jul 15, 2021 12:48:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Jul 15, 2021 12:48:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Jul 15, 2021 12:48:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Jul 15, 2021 12:48:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Jul 15, 2021 12:48:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Jul 15, 2021 12:48:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Jul 15, 2021 12:48:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Jul 15, 2021 12:48:50 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.32.0-SNAPSHOT
Jul 15, 2021 12:48:51 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-07-15_05_48_50-5255283704164082595?project=apache-beam-testing
Jul 15, 2021 12:48:51 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-07-15_05_48_50-5255283704164082595
Jul 15, 2021 12:48:51 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-07-15_05_48_50-5255283704164082595
Jul 15, 2021 12:48:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-07-15T12:48:55.679Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-07-ohdy. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Jul 15, 2021 12:49:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-15T12:48:59.445Z: Worker configuration: n1-standard-2 in us-central1-a.
Jul 15, 2021 12:49:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-15T12:49:00.088Z: Expanding SplittableParDo operations into optimizable parts.
Jul 15, 2021 12:49:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-15T12:49:00.130Z: Expanding CollectionToSingleton operations into optimizable parts.
Jul 15, 2021 12:49:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-15T12:49:00.219Z: Expanding CoGroupByKey operations into optimizable parts.
Jul 15, 2021 12:49:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-15T12:49:00.293Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Jul 15, 2021 12:49:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-15T12:49:00.331Z: Expanding GroupByKey operations into streaming Read/Write steps
Jul 15, 2021 12:49:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-15T12:49:00.400Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jul 15, 2021 12:49:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-15T12:49:00.505Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jul 15, 2021 12:49:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-15T12:49:00.541Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Jul 15, 2021 12:49:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-15T12:49:00.577Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Jul 15, 2021 12:49:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-15T12:49:00.619Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Jul 15, 2021 12:49:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-15T12:49:00.657Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Jul 15, 2021 12:49:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-15T12:49:00.695Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jul 15, 2021 12:49:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-15T12:49:00.742Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jul 15, 2021 12:49:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-15T12:49:00.776Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jul 15, 2021 12:49:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-15T12:49:00.820Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Jul 15, 2021 12:49:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-15T12:49:00.867Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Jul 15, 2021 12:49:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-15T12:49:00.901Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Jul 15, 2021 12:49:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-15T12:49:00.922Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Jul 15, 2021 12:49:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-15T12:49:00.949Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jul 15, 2021 12:49:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-15T12:49:00.981Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jul 15, 2021 12:49:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-15T12:49:01.054Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jul 15, 2021 12:49:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-15T12:49:01.088Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Jul 15, 2021 12:49:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-15T12:49:01.129Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Jul 15, 2021 12:49:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-15T12:49:01.162Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Jul 15, 2021 12:49:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-15T12:49:01.196Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Jul 15, 2021 12:49:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-15T12:49:01.231Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Jul 15, 2021 12:49:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-15T12:49:01.263Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Jul 15, 2021 12:49:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-15T12:49:01.291Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Jul 15, 2021 12:49:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-15T12:49:01.334Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Jul 15, 2021 12:49:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-15T12:49:01.756Z: Starting 5 ****s in us-central1-a...
Jul 15, 2021 12:49:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-15T12:49:14.134Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jul 15, 2021 12:50:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-15T12:50:44.189Z: Workers have started successfully.
Jul 15, 2021 12:50:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-15T12:50:44.237Z: Workers have started successfully.
Jul 15, 2021 3:49:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-15T15:49:02.429Z: Staged package beam-sdks-java-load-tests-2.32.0-SNAPSHOT-GJdw0l-sAknvcEzphQd69BxHY_91fiWIFAOeLKEe1g4.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-load-tests-2.32.0-SNAPSHOT-GJdw0l-sAknvcEzphQd69BxHY_91fiWIFAOeLKEe1g4.jar' is inaccessible.
Jul 15, 2021 3:49:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-15T15:49:04.017Z: Staged package jackson-annotations-2.12.1-IDzv36bIHmqoThHykvKcqXNEo8O8ApOr6gZc2DdZKHM.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/jackson-annotations-2.12.1-IDzv36bIHmqoThHykvKcqXNEo8O8ApOr6gZc2DdZKHM.jar' is inaccessible.
Jul 15, 2021 3:49:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-15T15:49:04.057Z: Staged package jackson-core-2.12.1-zImcturgyAuH1ZDuqGUoeXNpzE_re3lGMgfWuxjwwlc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/jackson-core-2.12.1-zImcturgyAuH1ZDuqGUoeXNpzE_re3lGMgfWuxjwwlc.jar' is inaccessible.
Jul 15, 2021 3:49:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-15T15:49:04.108Z: Staged package jackson-databind-2.12.1-8so8KOve1ZyYRH1Rr-lFMj35YVQK9moGPAFVl6-TaqA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/jackson-databind-2.12.1-8so8KOve1ZyYRH1Rr-lFMj35YVQK9moGPAFVl6-TaqA.jar' is inaccessible.
Jul 15, 2021 3:49:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-07-15T15:49:05.066Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jul 15, 2021 3:52:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-07-15T15:52:04.280Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jul 15, 2021 3:55:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-15T15:55:02.261Z: Staged package beam-sdks-java-load-tests-2.32.0-SNAPSHOT-GJdw0l-sAknvcEzphQd69BxHY_91fiWIFAOeLKEe1g4.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-load-tests-2.32.0-SNAPSHOT-GJdw0l-sAknvcEzphQd69BxHY_91fiWIFAOeLKEe1g4.jar' is inaccessible.
Jul 15, 2021 3:55:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-15T15:55:03.376Z: Staged package jackson-annotations-2.12.1-IDzv36bIHmqoThHykvKcqXNEo8O8ApOr6gZc2DdZKHM.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/jackson-annotations-2.12.1-IDzv36bIHmqoThHykvKcqXNEo8O8ApOr6gZc2DdZKHM.jar' is inaccessible.
Jul 15, 2021 3:55:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-15T15:55:03.419Z: Staged package jackson-core-2.12.1-zImcturgyAuH1ZDuqGUoeXNpzE_re3lGMgfWuxjwwlc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/jackson-core-2.12.1-zImcturgyAuH1ZDuqGUoeXNpzE_re3lGMgfWuxjwwlc.jar' is inaccessible.
Jul 15, 2021 3:55:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-15T15:55:03.481Z: Staged package jackson-databind-2.12.1-8so8KOve1ZyYRH1Rr-lFMj35YVQK9moGPAFVl6-TaqA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/jackson-databind-2.12.1-8so8KOve1ZyYRH1Rr-lFMj35YVQK9moGPAFVl6-TaqA.jar' is inaccessible.
Jul 15, 2021 3:55:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-07-15T15:55:04.180Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jul 15, 2021 3:58:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-07-15T15:58:04.583Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jul 15, 2021 4:00:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-15T16:00:34.800Z: Cancel request is committed for workflow job: 2021-07-15_05_48_50-5255283704164082595.
Jul 15, 2021 4:00:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-15T16:00:34.826Z: Cleaning up.
Jul 15, 2021 4:00:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-15T16:00:34.901Z: Stopping **** pool...
Jul 15, 2021 4:00:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-15T16:00:34.949Z: Stopping **** pool...
Jul 15, 2021 4:01:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-15T16:01:23.378Z: Worker pool stopped.
Jul 15, 2021 4:01:29 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-07-15_05_48_50-5255283704164082595 finished with status CANCELLED.
Load test results for test (ID): e513e3b9-f7c2-41f9-ad31-c0de7046441b and timestamp: 2021-07-15T12:48:44.793000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11367.522
dataflow_v2_java11_total_bytes_count               4.0231248E9
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210715124335
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:94342c9bc3002d205d475301742f8f9f2469cbf1b24912fd818d91641d8a9ca1
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:94342c9bc3002d205d475301742f8f9f2469cbf1b24912fd818d91641d8a9ca1
  Associated tags:
 - 20210715124335
Tags:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210715124335
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210715124335].
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:94342c9bc3002d205d475301742f8f9f2469cbf1b24912fd818d91641d8a9ca1].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 18m 13s
107 actionable tasks: 77 executed, 28 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/ahfusdgctzpxe

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #27

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/27/display/redirect?page=changes>

Changes:

[noreply] Fix broken package substitution in starter archetype

[je.ik] [BEAM-12597] Add AppendingTransformer for reference.conf in shade

[heejong] [BEAM-12604] Do not expose Zookeeper client port in Kafka k8s config

[noreply] [BEAM-11434]Make SpannerAccessor public (#13641)

[noreply] Update Beam Go row coder to more completely handle schema interfaces

[noreply] [BEAM-12556] Enable Go Build Tests in Samza Runner (#15167)

[noreply] [BEAM-8376] Google Cloud Firestore Connector - Add Firestore v1 Read


------------------------------------------
[...truncated 72.02 KB...]
INFO: Adding Read co-input/StripIds as step s6
Jul 14, 2021 12:48:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Jul 14, 2021 12:48:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Jul 14, 2021 12:48:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Jul 14, 2021 12:48:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Jul 14, 2021 12:48:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Jul 14, 2021 12:48:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Jul 14, 2021 12:48:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Jul 14, 2021 12:48:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Jul 14, 2021 12:48:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Jul 14, 2021 12:48:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Jul 14, 2021 12:48:27 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.32.0-SNAPSHOT
Jul 14, 2021 12:48:28 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-07-14_05_48_27-4547633016899704469?project=apache-beam-testing
Jul 14, 2021 12:48:28 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-07-14_05_48_27-4547633016899704469
Jul 14, 2021 12:48:28 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-07-14_05_48_27-4547633016899704469
Jul 14, 2021 12:48:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-07-14T12:48:32.593Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-07-vjqo. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Jul 14, 2021 12:48:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-14T12:48:36.512Z: Worker configuration: n1-standard-2 in us-central1-a.
Jul 14, 2021 12:48:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-14T12:48:37.177Z: Expanding SplittableParDo operations into optimizable parts.
Jul 14, 2021 12:48:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-14T12:48:37.205Z: Expanding CollectionToSingleton operations into optimizable parts.
Jul 14, 2021 12:48:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-14T12:48:37.276Z: Expanding CoGroupByKey operations into optimizable parts.
Jul 14, 2021 12:48:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-14T12:48:37.368Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Jul 14, 2021 12:48:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-14T12:48:37.407Z: Expanding GroupByKey operations into streaming Read/Write steps
Jul 14, 2021 12:48:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-14T12:48:37.467Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jul 14, 2021 12:48:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-14T12:48:37.578Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jul 14, 2021 12:48:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-14T12:48:37.608Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Jul 14, 2021 12:48:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-14T12:48:37.643Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Jul 14, 2021 12:48:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-14T12:48:37.665Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Jul 14, 2021 12:48:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-14T12:48:37.690Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Jul 14, 2021 12:48:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-14T12:48:37.714Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jul 14, 2021 12:48:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-14T12:48:37.744Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jul 14, 2021 12:48:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-14T12:48:37.771Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jul 14, 2021 12:48:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-14T12:48:37.798Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Jul 14, 2021 12:48:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-14T12:48:37.829Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Jul 14, 2021 12:48:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-14T12:48:37.856Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Jul 14, 2021 12:48:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-14T12:48:37.877Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Jul 14, 2021 12:48:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-14T12:48:37.904Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jul 14, 2021 12:48:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-14T12:48:37.937Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jul 14, 2021 12:48:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-14T12:48:37.980Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jul 14, 2021 12:48:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-14T12:48:38.009Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Jul 14, 2021 12:48:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-14T12:48:38.042Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Jul 14, 2021 12:48:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-14T12:48:38.076Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Jul 14, 2021 12:48:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-14T12:48:38.107Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Jul 14, 2021 12:48:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-14T12:48:38.134Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Jul 14, 2021 12:48:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-14T12:48:38.165Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Jul 14, 2021 12:48:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-14T12:48:38.192Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Jul 14, 2021 12:48:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-14T12:48:38.224Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Jul 14, 2021 12:48:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-14T12:48:38.607Z: Starting 5 ****s in us-central1-a...
Jul 14, 2021 12:48:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-14T12:48:43.587Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jul 14, 2021 12:50:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-14T12:50:20.363Z: Workers have started successfully.
Jul 14, 2021 12:50:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-14T12:50:20.389Z: Workers have started successfully.
Jul 14, 2021 3:15:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-14T15:15:41.157Z: Staged package kafka-clients-2.4.1-or4Vrm01S3aXE64MuzGE8iWqFvyYIFseskr9n8KjYNk.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/kafka-clients-2.4.1-or4Vrm01S3aXE64MuzGE8iWqFvyYIFseskr9n8KjYNk.jar' is inaccessible.
Jul 14, 2021 3:15:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-14T15:15:41.950Z: Staged package lz4-java-1.6.0-0ilUWqKx1SA8h2YUvbz_yswwNpf0-PJvdk4dbB7S5BY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/lz4-java-1.6.0-0ilUWqKx1SA8h2YUvbz_yswwNpf0-PJvdk4dbB7S5BY.jar' is inaccessible.
Jul 14, 2021 3:15:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-07-14T15:15:42.657Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jul 14, 2021 3:18:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-07-14T15:18:42.299Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jul 14, 2021 3:21:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-14T15:21:41.376Z: Staged package kafka-clients-2.4.1-or4Vrm01S3aXE64MuzGE8iWqFvyYIFseskr9n8KjYNk.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/kafka-clients-2.4.1-or4Vrm01S3aXE64MuzGE8iWqFvyYIFseskr9n8KjYNk.jar' is inaccessible.
Jul 14, 2021 3:21:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-14T15:21:41.475Z: Staged package lz4-java-1.6.0-0ilUWqKx1SA8h2YUvbz_yswwNpf0-PJvdk4dbB7S5BY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/lz4-java-1.6.0-0ilUWqKx1SA8h2YUvbz_yswwNpf0-PJvdk4dbB7S5BY.jar' is inaccessible.
Jul 14, 2021 3:21:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-07-14T15:21:42.349Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jul 14, 2021 3:24:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-07-14T15:24:42.363Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jul 14, 2021 3:27:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-14T15:27:41.121Z: Staged package kafka-clients-2.4.1-or4Vrm01S3aXE64MuzGE8iWqFvyYIFseskr9n8KjYNk.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/kafka-clients-2.4.1-or4Vrm01S3aXE64MuzGE8iWqFvyYIFseskr9n8KjYNk.jar' is inaccessible.
Jul 14, 2021 3:27:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-14T15:27:41.199Z: Staged package lz4-java-1.6.0-0ilUWqKx1SA8h2YUvbz_yswwNpf0-PJvdk4dbB7S5BY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/lz4-java-1.6.0-0ilUWqKx1SA8h2YUvbz_yswwNpf0-PJvdk4dbB7S5BY.jar' is inaccessible.
Jul 14, 2021 3:27:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-07-14T15:27:41.886Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jul 14, 2021 3:30:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-07-14T15:30:42.446Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jul 14, 2021 3:33:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-14T15:33:41.103Z: Staged package kafka-clients-2.4.1-or4Vrm01S3aXE64MuzGE8iWqFvyYIFseskr9n8KjYNk.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/kafka-clients-2.4.1-or4Vrm01S3aXE64MuzGE8iWqFvyYIFseskr9n8KjYNk.jar' is inaccessible.
Jul 14, 2021 3:33:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-14T15:33:41.193Z: Staged package lz4-java-1.6.0-0ilUWqKx1SA8h2YUvbz_yswwNpf0-PJvdk4dbB7S5BY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/lz4-java-1.6.0-0ilUWqKx1SA8h2YUvbz_yswwNpf0-PJvdk4dbB7S5BY.jar' is inaccessible.
Jul 14, 2021 3:33:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-07-14T15:33:42.009Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jul 14, 2021 3:36:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-07-14T15:36:42.431Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jul 14, 2021 3:39:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-14T15:39:41.281Z: Staged package kafka-clients-2.4.1-or4Vrm01S3aXE64MuzGE8iWqFvyYIFseskr9n8KjYNk.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/kafka-clients-2.4.1-or4Vrm01S3aXE64MuzGE8iWqFvyYIFseskr9n8KjYNk.jar' is inaccessible.
Jul 14, 2021 3:39:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-14T15:39:41.367Z: Staged package lz4-java-1.6.0-0ilUWqKx1SA8h2YUvbz_yswwNpf0-PJvdk4dbB7S5BY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/lz4-java-1.6.0-0ilUWqKx1SA8h2YUvbz_yswwNpf0-PJvdk4dbB7S5BY.jar' is inaccessible.
Jul 14, 2021 3:39:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-07-14T15:39:42.078Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jul 14, 2021 3:42:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-07-14T15:42:44.869Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jul 14, 2021 3:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-14T15:45:41.223Z: Staged package kafka-clients-2.4.1-or4Vrm01S3aXE64MuzGE8iWqFvyYIFseskr9n8KjYNk.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/kafka-clients-2.4.1-or4Vrm01S3aXE64MuzGE8iWqFvyYIFseskr9n8KjYNk.jar' is inaccessible.
Jul 14, 2021 3:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-14T15:45:41.333Z: Staged package lz4-java-1.6.0-0ilUWqKx1SA8h2YUvbz_yswwNpf0-PJvdk4dbB7S5BY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/lz4-java-1.6.0-0ilUWqKx1SA8h2YUvbz_yswwNpf0-PJvdk4dbB7S5BY.jar' is inaccessible.
Jul 14, 2021 3:45:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-07-14T15:45:42.137Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jul 14, 2021 3:48:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-07-14T15:48:42.323Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jul 14, 2021 3:51:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-14T15:51:41.067Z: Staged package kafka-clients-2.4.1-or4Vrm01S3aXE64MuzGE8iWqFvyYIFseskr9n8KjYNk.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/kafka-clients-2.4.1-or4Vrm01S3aXE64MuzGE8iWqFvyYIFseskr9n8KjYNk.jar' is inaccessible.
Jul 14, 2021 3:51:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-14T15:51:41.160Z: Staged package lz4-java-1.6.0-0ilUWqKx1SA8h2YUvbz_yswwNpf0-PJvdk4dbB7S5BY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/lz4-java-1.6.0-0ilUWqKx1SA8h2YUvbz_yswwNpf0-PJvdk4dbB7S5BY.jar' is inaccessible.
Jul 14, 2021 3:51:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-07-14T15:51:41.874Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jul 14, 2021 3:54:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-07-14T15:54:41.914Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jul 14, 2021 3:57:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-14T15:57:41.214Z: Staged package kafka-clients-2.4.1-or4Vrm01S3aXE64MuzGE8iWqFvyYIFseskr9n8KjYNk.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/kafka-clients-2.4.1-or4Vrm01S3aXE64MuzGE8iWqFvyYIFseskr9n8KjYNk.jar' is inaccessible.
Jul 14, 2021 3:57:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-14T15:57:41.329Z: Staged package lz4-java-1.6.0-0ilUWqKx1SA8h2YUvbz_yswwNpf0-PJvdk4dbB7S5BY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/lz4-java-1.6.0-0ilUWqKx1SA8h2YUvbz_yswwNpf0-PJvdk4dbB7S5BY.jar' is inaccessible.
Jul 14, 2021 3:57:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-07-14T15:57:42.178Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jul 14, 2021 4:00:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-14T16:00:33.165Z: Cancel request is committed for workflow job: 2021-07-14_05_48_27-4547633016899704469.
Jul 14, 2021 4:00:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-14T16:00:33.200Z: Cleaning up.
Jul 14, 2021 4:00:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-14T16:00:33.281Z: Stopping **** pool...
Jul 14, 2021 4:00:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-14T16:00:33.332Z: Stopping **** pool...
Jul 14, 2021 4:01:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-14T16:01:35.713Z: Worker pool stopped.
Jul 14, 2021 4:01:43 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-07-14_05_48_27-4547633016899704469 finished with status CANCELLED.
Load test results for test (ID): 8a3e8d8b-a3cd-4317-9cbb-afd3d583861e and timestamp: 2021-07-14T12:48:22.685000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11385.074
dataflow_v2_java11_total_bytes_count               4.1915646E9
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210714124331
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:75cf54bda2c1e736988ab1d1684ad9175a2c28aa4835c0f85304efcc7f0dfa49
Deleted: sha256:5a4d33f02fe5acf358f17dae369b4d73de5f1714f2fbb46ea2d0949428bd318e
Deleted: sha256:f7adcd1364394c7f90e25fb4d2e4ed20b050436bc07a962e75fa8b4d21bd9ef5
Deleted: sha256:c40dfb2b45cbb77e84126c2b41aa5ad1b74c5c96db2d4584b107ddeb9c61e1dd
Deleted: sha256:ef7b1504cee2caafa8ce1de3e3ddc326cf285e9bd00286de98b40b8a65400525
Deleted: sha256:e28fe78eb6268b6b5e62555f428a2bb58c93545569599c410fca4b52bf13473f
Deleted: sha256:b280634a7aa910d99c2dab1934d0a74b6a0c16a5ed7bc2ebff79505f1b850c46
Deleted: sha256:d569cffbaf392b406a88079fd61c094fab876ee095674142dcb893fdef1f24c5
Deleted: sha256:ec547199f2720f2926e5cb7833fcdb508d0c5f8579e0eb2df6b7782c54ab5ef9
Deleted: sha256:51a7a460e0632c6298b45fde4d7439e9639c148f3afa3ff9151b88ea7cdd2109
Deleted: sha256:8e770333ec7816266739ea8355b64ee354e0b505ac5c9c9a253824555e91de42
Deleted: sha256:505c46071bdc0d4d7542c07c21bad061c8c91b02d64f5660d1199a87f34c4c26
Deleted: sha256:0080c7b6ed670002e50dcdd848314130f8ede1c5ff5ebcf8e68abe41e837911a
Deleted: sha256:6840c9bb82edafccceb8ed6cfdc3238d7a7ae9f8d01d75364ff05d32f26ff72a
Deleted: sha256:dd020760c8e85c5a271de78fe557bc5f327da5d2ed8f667e19cfdb8f785df734
Deleted: sha256:0b98e5252d59fc904ece9470d09dc28bed1d7614e481102df66b8b7d5d5bf7a8
Deleted: sha256:f279ae49233005dfcb398382785de851089b1b7ba771f20526382b56d2f52a18
Deleted: sha256:47e9d32de9ac0496618433b975e16caf7f756c23c6c82b7a61e069abfd3ab030
Deleted: sha256:fcb1815b64fa3a0d1856a436c5c4a278a02699f7fd33b64e821aa5e37be4ecd3
Deleted: sha256:121769318e377c28b8023db7e1ca85d4bd39e74d9dae7a7a0715eec3a7afedba
Deleted: sha256:71004ed62a3ea8dad32430d27eef4f66fcb70f06aa149decc50ebc4895374a62
Deleted: sha256:4d5de452bcf8055fbbb632d6a0f627debf45a74c0a8baecf470ad3431d965ee6
Deleted: sha256:1dc47ef53d16516141a97d0414066800b3f441cc3ded94746c385ead193525b9
Deleted: sha256:888deeae9a8308c876fe3751a825583e429274d57c5109df48df814ca215bb99
Deleted: sha256:4e654ba3f4b00f2f565a134161d3b46380d50740b9034c86c0a181c6b2a55d65
Deleted: sha256:5eb82933e573d68984ec60d0a9231ee8faa36b018d8e1ca21dbd529bd9bc5c08
Deleted: sha256:ffe9d7b3fbfe2b78fc10bab64ef90ecdcd8da951b6d2c488076a6977e2d1b85a
Deleted: sha256:b07055351bf64654fe5b82642e2549b0070278a8014e0aa8bdb1024795c7cd1d
Deleted: sha256:8c2345b4e7132a5bb5af42358307cb8154ba4bc6a203a87eae9a50ac0221ed8a
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:75cf54bda2c1e736988ab1d1684ad9175a2c28aa4835c0f85304efcc7f0dfa49
  Associated tags:
 - 20210714124331
Tags:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210714124331
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210714124331].
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:75cf54bda2c1e736988ab1d1684ad9175a2c28aa4835c0f85304efcc7f0dfa49].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 18m 26s
107 actionable tasks: 76 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/5u7jricygr3pe

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #26

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/26/display/redirect?page=changes>

Changes:

[rohde.samuel] [BEAM-12531] Compat changes for deferred dataframes with ib.show

[rohde.samuel] address linter

[rohde.samuel] address linter

[relax] don't hold watermark for expiry timer

[rohde.samuel] add license to dataframes.ipynb

[Luke Cwik] [BEAM-12596] Validate that @GetSize always returns a value greater then

[yoshiki.obata] [BEAM-7372] restore docstring removed accidentally

[alexkoay88] Cache readers after getting progress.

[mrudary] Generalize S3FileSystem to support multiple URI schemes.

[noreply] [BEAM-12515] Skip flaky PipelineOptions.test_display_data test

[noreply] Revert "[BEAM-12515] Revert "[BEAM-12119] [BEAM-12122] Add integer and

[noreply] [BEAM-12538] Allow PipelineOptions to be specified on command line of

[noreply] [BEAM-12590] Automatically upgrading Dataflow Python pipelines that use

[noreply] Merge pull request #14869 from [BEAM-12357] improve WithKeys transform


------------------------------------------
[...truncated 65.96 KB...]
b42917603194: Preparing
076abcc491a6: Preparing
56a1b02fdf59: Preparing
917c03d11fcc: Preparing
0e2096e73df4: Preparing
f4ea1dafd414: Preparing
085bdde45f1a: Preparing
17a01ad2f4fc: Preparing
405911e4cc90: Preparing
de631976d4e7: Preparing
20d5a1afcef6: Preparing
6ea995e9b7d3: Preparing
c0848348e2f7: Preparing
79c550eb7bd2: Preparing
7095af798ace: Preparing
fe6a4fdbedc0: Preparing
e4d0e810d54a: Preparing
085bdde45f1a: Waiting
4e006334a6fd: Preparing
17a01ad2f4fc: Waiting
de631976d4e7: Waiting
405911e4cc90: Waiting
7095af798ace: Waiting
6ea995e9b7d3: Waiting
fe6a4fdbedc0: Waiting
79c550eb7bd2: Waiting
20d5a1afcef6: Waiting
c0848348e2f7: Waiting
e4d0e810d54a: Waiting
0e2096e73df4: Waiting
f4ea1dafd414: Waiting
4e006334a6fd: Waiting
917c03d11fcc: Pushed
076abcc491a6: Pushed
b42917603194: Pushed
0e2096e73df4: Pushed
a4db41366f5e: Pushed
56a1b02fdf59: Pushed
085bdde45f1a: Pushed
17a01ad2f4fc: Pushed
de631976d4e7: Pushed
6ea995e9b7d3: Layer already exists
c0848348e2f7: Layer already exists
79c550eb7bd2: Layer already exists
7095af798ace: Layer already exists
fe6a4fdbedc0: Layer already exists
e4d0e810d54a: Layer already exists
4e006334a6fd: Layer already exists
20d5a1afcef6: Pushed
f4ea1dafd414: Pushed
405911e4cc90: Pushed
20210713124934: digest: sha256:041368d53e3b5ded96e1772af8fd3b48f938366ea13d40481e9140dbd1897b47 size: 4310

> Task :sdks:java:testing:load-tests:run
Jul 13, 2021 12:55:40 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Jul 13, 2021 12:55:41 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 191 files. Enable logging at DEBUG level to see which files will be staged.
Jul 13, 2021 12:55:41 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Jul 13, 2021 12:55:41 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Jul 13, 2021 12:55:44 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Jul 13, 2021 12:55:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <112670 bytes, hash d19ee1f317b5ca63dd5e4ee77af47a960bc8178cafe8b8ddb4b551633dd99bb5> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-0Z7h8xe1ymPdXk7nevR6lgvIF4yv6LjdtLVRYz3Zm7U.pb
Jul 13, 2021 12:55:45 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 191 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Jul 13, 2021 12:55:46 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 191 files cached, 0 files newly uploaded in 0 seconds
Jul 13, 2021 12:55:46 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Jul 13, 2021 12:55:46 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@68f0f72c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3d96fa9e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3b545206, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77bb48d5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@181d8899, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12d5c30e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@b887730, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26586b74, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@52f57666, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6e041285, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@267dc982, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@439b15f2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3aa41da1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@74fab04a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c83468e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@272ce069, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3d7fb838, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1c26273d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3a37a501, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7a583586]
Jul 13, 2021 12:55:46 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Jul 13, 2021 12:55:46 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Jul 13, 2021 12:55:46 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Jul 13, 2021 12:55:46 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Jul 13, 2021 12:55:46 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2abe9173, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@235d29d6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1fdca564, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43f9dd56, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1d12e953, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@57cb70be, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2d4608a6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@20d87335, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2a8a4e0c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26c89563, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3bd6ba24, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58f437b0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@20f6f88c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4277127c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4c7e978c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@354e7004, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@22ebccb9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5875de6a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@608fe01f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2687725a]
Jul 13, 2021 12:55:46 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Jul 13, 2021 12:55:46 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Jul 13, 2021 12:55:46 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Jul 13, 2021 12:55:46 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Jul 13, 2021 12:55:46 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Jul 13, 2021 12:55:46 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Jul 13, 2021 12:55:46 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Jul 13, 2021 12:55:46 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Jul 13, 2021 12:55:46 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Jul 13, 2021 12:55:46 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Jul 13, 2021 12:55:46 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Jul 13, 2021 12:55:46 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.32.0-SNAPSHOT
Jul 13, 2021 12:55:47 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-07-13_05_55_46-1917470826405443946?project=apache-beam-testing
Jul 13, 2021 12:55:47 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-07-13_05_55_46-1917470826405443946
Jul 13, 2021 12:55:47 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-07-13_05_55_46-1917470826405443946
Jul 13, 2021 12:55:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-07-13T12:55:51.968Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-07-p14x. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Jul 13, 2021 12:55:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-13T12:55:55.985Z: Worker configuration: n1-standard-2 in us-central1-a.
Jul 13, 2021 12:55:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-13T12:55:56.565Z: Expanding SplittableParDo operations into optimizable parts.
Jul 13, 2021 12:55:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-13T12:55:56.597Z: Expanding CollectionToSingleton operations into optimizable parts.
Jul 13, 2021 12:55:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-13T12:55:56.674Z: Expanding CoGroupByKey operations into optimizable parts.
Jul 13, 2021 12:55:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-13T12:55:56.748Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Jul 13, 2021 12:55:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-13T12:55:56.779Z: Expanding GroupByKey operations into streaming Read/Write steps
Jul 13, 2021 12:55:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-13T12:55:56.855Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jul 13, 2021 12:55:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-13T12:55:56.983Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jul 13, 2021 12:55:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-13T12:55:57.024Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Jul 13, 2021 12:55:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-13T12:55:57.044Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Jul 13, 2021 12:55:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-13T12:55:57.078Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Jul 13, 2021 12:55:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-13T12:55:57.112Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Jul 13, 2021 12:55:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-13T12:55:57.146Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jul 13, 2021 12:55:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-13T12:55:57.180Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jul 13, 2021 12:55:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-13T12:55:57.220Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jul 13, 2021 12:55:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-13T12:55:57.252Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Jul 13, 2021 12:55:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-13T12:55:57.284Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Jul 13, 2021 12:55:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-13T12:55:57.322Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Jul 13, 2021 12:55:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-13T12:55:57.358Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Jul 13, 2021 12:55:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-13T12:55:57.393Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jul 13, 2021 12:55:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-13T12:55:57.416Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jul 13, 2021 12:55:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-13T12:55:57.450Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jul 13, 2021 12:55:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-13T12:55:57.480Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Jul 13, 2021 12:55:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-13T12:55:57.501Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Jul 13, 2021 12:55:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-13T12:55:57.535Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Jul 13, 2021 12:55:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-13T12:55:57.571Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Jul 13, 2021 12:55:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-13T12:55:57.603Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Jul 13, 2021 12:55:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-13T12:55:57.638Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Jul 13, 2021 12:55:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-13T12:55:57.675Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Jul 13, 2021 12:55:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-13T12:55:57.701Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Jul 13, 2021 12:55:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-13T12:55:58.077Z: Starting 5 ****s in us-central1-a...
Jul 13, 2021 12:56:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-13T12:56:19.463Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jul 13, 2021 12:57:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-13T12:57:25.046Z: Workers have started successfully.
Jul 13, 2021 12:57:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-13T12:57:25.084Z: Workers have started successfully.
Jul 13, 2021 2:17:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-13T14:17:09.746Z: generic::internal: Current work item is no longer valid on backend. The work has already completed or will be retried. This is expected during autoscaling events.
generic::internal: Current work item is no longer valid on backend. The work has already completed or will be retried. This is expected during autoscaling events.
Jul 13, 2021 2:17:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-13T14:17:50.074Z: generic::internal: Current work item is no longer valid on backend. The work has already completed or will be retried. This is expected during autoscaling events.
Jul 13, 2021 4:00:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-13T16:00:33.733Z: Cancel request is committed for workflow job: 2021-07-13_05_55_46-1917470826405443946.
Jul 13, 2021 4:00:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-13T16:00:33.766Z: Cleaning up.
Jul 13, 2021 4:00:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-13T16:00:33.830Z: Stopping **** pool...
Jul 13, 2021 4:00:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-13T16:00:33.873Z: Stopping **** pool...
Jul 13, 2021 4:01:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-13T16:01:47.295Z: Worker pool stopped.
Jul 13, 2021 4:01:53 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-07-13_05_55_46-1917470826405443946 finished with status CANCELLED.
Load test results for test (ID): 6812bd02-e867-4b7c-bc31-fdb94ca1ac1c and timestamp: 2021-07-13T12:55:41.531000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 10957.261
dataflow_v2_java11_total_bytes_count               3.4731269E9
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210713124934
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:041368d53e3b5ded96e1772af8fd3b48f938366ea13d40481e9140dbd1897b47
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:041368d53e3b5ded96e1772af8fd3b48f938366ea13d40481e9140dbd1897b47
  Associated tags:
 - 20210713124934
Tags:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210713124934
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210713124934].
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:041368d53e3b5ded96e1772af8fd3b48f938366ea13d40481e9140dbd1897b47].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 13m 5s
107 actionable tasks: 76 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/v4km4vfgyzzdo

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #25

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/25/display/redirect?page=changes>

Changes:

[veblush] Integration

[veblush] Pass credentials to gcsio

[veblush] Update by review


------------------------------------------
[...truncated 65.64 KB...]
`--verbosity=error` to silence this warning: `gcloud docker
--verbosity=error -- pull gcr.io/project-id/my-image`.

See: https://cloud.google.com/container-registry/docs/support/deprecation-notices#gcloud-docker

The push refers to repository [us.gcr.io/apache-beam-testing/java-postcommit-it/java]
7e4cb5cbcfcf: Preparing
f60b3dfcf901: Preparing
1ac1eaa10b25: Preparing
a3acfb936b93: Preparing
677c8a62dac6: Preparing
6648df149d8c: Preparing
93467b52d81c: Preparing
e56dc941867f: Preparing
f321abf80fcc: Preparing
5295de51c2f0: Preparing
a512a97feb49: Preparing
c9dc0a435304: Preparing
6ea995e9b7d3: Preparing
c0848348e2f7: Preparing
79c550eb7bd2: Preparing
7095af798ace: Preparing
fe6a4fdbedc0: Preparing
e4d0e810d54a: Preparing
4e006334a6fd: Preparing
f321abf80fcc: Waiting
5295de51c2f0: Waiting
6648df149d8c: Waiting
e56dc941867f: Waiting
c9dc0a435304: Waiting
79c550eb7bd2: Waiting
c0848348e2f7: Waiting
a512a97feb49: Waiting
93467b52d81c: Waiting
e4d0e810d54a: Waiting
4e006334a6fd: Waiting
7095af798ace: Waiting
677c8a62dac6: Pushed
1ac1eaa10b25: Pushed
f60b3dfcf901: Pushed
7e4cb5cbcfcf: Pushed
6648df149d8c: Pushed
a3acfb936b93: Pushed
e56dc941867f: Pushed
f321abf80fcc: Pushed
6ea995e9b7d3: Layer already exists
c0848348e2f7: Layer already exists
a512a97feb49: Pushed
79c550eb7bd2: Layer already exists
7095af798ace: Layer already exists
e4d0e810d54a: Layer already exists
fe6a4fdbedc0: Layer already exists
4e006334a6fd: Layer already exists
c9dc0a435304: Pushed
93467b52d81c: Pushed
5295de51c2f0: Pushed
20210712124335: digest: sha256:1546f6aac3c8abfaaf43b7d0d3dcc3e6a64d2eed2d1d66b97d0fe608bb5c5ec3 size: 4310

> Task :sdks:java:testing:load-tests:run
Jul 12, 2021 12:48:47 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Jul 12, 2021 12:48:47 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 191 files. Enable logging at DEBUG level to see which files will be staged.
Jul 12, 2021 12:48:48 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Jul 12, 2021 12:48:48 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Jul 12, 2021 12:48:51 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Jul 12, 2021 12:48:51 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <112608 bytes, hash 4feb24574bd77107004b753ac582dfbe38592ba1e020a72c66afc88e5c977dcf> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-T-skV0vXcQcAS3U6xYLfvjhZK6HgIKcsZq_IjlyXfc8.pb
Jul 12, 2021 12:48:52 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 191 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Jul 12, 2021 12:48:53 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 191 files cached, 0 files newly uploaded in 0 seconds
Jul 12, 2021 12:48:53 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Jul 12, 2021 12:48:53 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1352434e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4f9a6c2d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2b6fcb9f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@75de6341, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@74170687, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@68f0f72c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3d96fa9e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3b545206, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77bb48d5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@181d8899, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12d5c30e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@b887730, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26586b74, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@52f57666, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6e041285, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@267dc982, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@439b15f2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3aa41da1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@74fab04a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c83468e]
Jul 12, 2021 12:48:53 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Jul 12, 2021 12:48:53 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Jul 12, 2021 12:48:53 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Jul 12, 2021 12:48:53 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Jul 12, 2021 12:48:53 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@574b7f4a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@44ed0a8f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@32177fa5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@a96d56c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6ab4a5b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2abe9173, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@235d29d6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1fdca564, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43f9dd56, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1d12e953, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@57cb70be, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2d4608a6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@20d87335, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2a8a4e0c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26c89563, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3bd6ba24, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58f437b0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@20f6f88c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4277127c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4c7e978c]
Jul 12, 2021 12:48:53 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Jul 12, 2021 12:48:53 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Jul 12, 2021 12:48:53 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Jul 12, 2021 12:48:53 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Jul 12, 2021 12:48:53 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Jul 12, 2021 12:48:53 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Jul 12, 2021 12:48:53 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Jul 12, 2021 12:48:53 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Jul 12, 2021 12:48:53 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Jul 12, 2021 12:48:53 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Jul 12, 2021 12:48:53 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Jul 12, 2021 12:48:53 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.32.0-SNAPSHOT
Jul 12, 2021 12:48:54 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-07-12_05_48_53-8227298622909193110?project=apache-beam-testing
Jul 12, 2021 12:48:54 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-07-12_05_48_53-8227298622909193110
Jul 12, 2021 12:48:54 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-07-12_05_48_53-8227298622909193110
Jul 12, 2021 12:49:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-07-12T12:48:58.403Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-07-5kjj. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Jul 12, 2021 12:49:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-12T12:49:02.249Z: Worker configuration: n1-standard-2 in us-central1-a.
Jul 12, 2021 12:49:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-12T12:49:02.800Z: Expanding SplittableParDo operations into optimizable parts.
Jul 12, 2021 12:49:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-12T12:49:02.832Z: Expanding CollectionToSingleton operations into optimizable parts.
Jul 12, 2021 12:49:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-12T12:49:02.898Z: Expanding CoGroupByKey operations into optimizable parts.
Jul 12, 2021 12:49:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-12T12:49:02.957Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Jul 12, 2021 12:49:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-12T12:49:02.992Z: Expanding GroupByKey operations into streaming Read/Write steps
Jul 12, 2021 12:49:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-12T12:49:03.060Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jul 12, 2021 12:49:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-12T12:49:03.158Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jul 12, 2021 12:49:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-12T12:49:03.183Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Jul 12, 2021 12:49:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-12T12:49:03.217Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Jul 12, 2021 12:49:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-12T12:49:03.253Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Jul 12, 2021 12:49:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-12T12:49:03.285Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Jul 12, 2021 12:49:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-12T12:49:03.318Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jul 12, 2021 12:49:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-12T12:49:03.357Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jul 12, 2021 12:49:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-12T12:49:03.386Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jul 12, 2021 12:49:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-12T12:49:03.412Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Jul 12, 2021 12:49:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-12T12:49:03.438Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Jul 12, 2021 12:49:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-12T12:49:03.464Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Jul 12, 2021 12:49:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-12T12:49:03.488Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Jul 12, 2021 12:49:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-12T12:49:03.528Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jul 12, 2021 12:49:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-12T12:49:03.560Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jul 12, 2021 12:49:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-12T12:49:03.613Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jul 12, 2021 12:49:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-12T12:49:03.639Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Jul 12, 2021 12:49:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-12T12:49:03.673Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Jul 12, 2021 12:49:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-12T12:49:03.699Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Jul 12, 2021 12:49:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-12T12:49:03.728Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Jul 12, 2021 12:49:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-12T12:49:03.754Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Jul 12, 2021 12:49:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-12T12:49:03.775Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Jul 12, 2021 12:49:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-12T12:49:03.803Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Jul 12, 2021 12:49:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-12T12:49:03.832Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Jul 12, 2021 12:49:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-12T12:49:04.147Z: Starting 5 ****s in us-central1-a...
Jul 12, 2021 12:49:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-12T12:49:12.850Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jul 12, 2021 12:50:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-12T12:50:31.452Z: Workers have started successfully.
Jul 12, 2021 12:50:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-12T12:50:31.476Z: Workers have started successfully.
Jul 12, 2021 4:00:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-12T16:00:34.981Z: Cancel request is committed for workflow job: 2021-07-12_05_48_53-8227298622909193110.
Jul 12, 2021 4:00:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-12T16:00:35.041Z: Cleaning up.
Jul 12, 2021 4:00:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-12T16:00:35.110Z: Stopping **** pool...
Jul 12, 2021 4:00:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-12T16:00:35.154Z: Stopping **** pool...
Jul 12, 2021 4:01:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-12T16:01:29.078Z: Worker pool stopped.
Jul 12, 2021 4:01:34 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-07-12_05_48_53-8227298622909193110 finished with status CANCELLED.
Load test results for test (ID): ccb56746-8cdf-473c-a434-c3c62d884c46 and timestamp: 2021-07-12T12:48:47.989000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11364.925
dataflow_v2_java11_total_bytes_count               5.9572159E9
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210712124335
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:1546f6aac3c8abfaaf43b7d0d3dcc3e6a64d2eed2d1d66b97d0fe608bb5c5ec3
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:1546f6aac3c8abfaaf43b7d0d3dcc3e6a64d2eed2d1d66b97d0fe608bb5c5ec3
  Associated tags:
 - 20210712124335
Tags:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210712124335
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210712124335].
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:1546f6aac3c8abfaaf43b7d0d3dcc3e6a64d2eed2d1d66b97d0fe608bb5c5ec3].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 18m 16s
107 actionable tasks: 78 executed, 27 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/3ailoq2whmzg6

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #24

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/24/display/redirect>

Changes:


------------------------------------------
[...truncated 65.39 KB...]
--verbosity=error -- pull gcr.io/project-id/my-image`.

See: https://cloud.google.com/container-registry/docs/support/deprecation-notices#gcloud-docker

The push refers to repository [us.gcr.io/apache-beam-testing/java-postcommit-it/java]
6ea79729e0ef: Preparing
2a1c220ff6c5: Preparing
56e8ad49fa14: Preparing
07b7f2425bae: Preparing
787e8e52c3b0: Preparing
c624e7e4c157: Preparing
224d4c9a11c2: Preparing
6ddfb06c746d: Preparing
9ac7e219a11a: Preparing
3daa999b2c1d: Preparing
e722e0f84309: Preparing
a691d412388d: Preparing
6ea995e9b7d3: Preparing
c0848348e2f7: Preparing
79c550eb7bd2: Preparing
7095af798ace: Preparing
fe6a4fdbedc0: Preparing
e4d0e810d54a: Preparing
4e006334a6fd: Preparing
a691d412388d: Waiting
c624e7e4c157: Waiting
6ea995e9b7d3: Waiting
c0848348e2f7: Waiting
224d4c9a11c2: Waiting
e4d0e810d54a: Waiting
79c550eb7bd2: Waiting
6ddfb06c746d: Waiting
7095af798ace: Waiting
9ac7e219a11a: Waiting
4e006334a6fd: Waiting
3daa999b2c1d: Waiting
fe6a4fdbedc0: Waiting
787e8e52c3b0: Pushed
56e8ad49fa14: Pushed
2a1c220ff6c5: Pushed
c624e7e4c157: Pushed
6ea79729e0ef: Pushed
07b7f2425bae: Pushed
6ddfb06c746d: Pushed
9ac7e219a11a: Pushed
6ea995e9b7d3: Layer already exists
c0848348e2f7: Layer already exists
79c550eb7bd2: Layer already exists
7095af798ace: Layer already exists
fe6a4fdbedc0: Layer already exists
e4d0e810d54a: Layer already exists
a691d412388d: Pushed
4e006334a6fd: Layer already exists
e722e0f84309: Pushed
224d4c9a11c2: Pushed
3daa999b2c1d: Pushed
20210711124335: digest: sha256:24b8d658f02d2fb4f5773ff0d74331512825f616db5a854fd1327097d0b670e4 size: 4310

> Task :sdks:java:testing:load-tests:run
Jul 11, 2021 12:48:25 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Jul 11, 2021 12:48:26 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 194 files. Enable logging at DEBUG level to see which files will be staged.
Jul 11, 2021 12:48:26 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Jul 11, 2021 12:48:26 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Jul 11, 2021 12:48:29 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Jul 11, 2021 12:48:29 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <113773 bytes, hash 3039ffd41a2ab693a9eb7b32d42bc126d0dbc71834d9c83e08e58464c23a3ff1> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-MDn_1BoqtpOp63sy1CvBJtDbxxg02cg-COWEZMI6P_E.pb
Jul 11, 2021 12:48:31 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 194 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Jul 11, 2021 12:48:31 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 194 files cached, 0 files newly uploaded in 0 seconds
Jul 11, 2021 12:48:31 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Jul 11, 2021 12:48:31 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c3c4a71, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1352434e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4f9a6c2d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2b6fcb9f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@75de6341, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@74170687, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@68f0f72c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3d96fa9e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3b545206, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77bb48d5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@181d8899, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12d5c30e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@b887730, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26586b74, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@52f57666, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6e041285, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@267dc982, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@439b15f2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3aa41da1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@74fab04a]
Jul 11, 2021 12:48:31 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Jul 11, 2021 12:48:31 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Jul 11, 2021 12:48:31 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Jul 11, 2021 12:48:31 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Jul 11, 2021 12:48:31 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3791f50e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@574b7f4a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@44ed0a8f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@32177fa5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@a96d56c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6ab4a5b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2abe9173, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@235d29d6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1fdca564, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43f9dd56, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1d12e953, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@57cb70be, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2d4608a6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@20d87335, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2a8a4e0c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26c89563, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3bd6ba24, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58f437b0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@20f6f88c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4277127c]
Jul 11, 2021 12:48:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Jul 11, 2021 12:48:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Jul 11, 2021 12:48:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Jul 11, 2021 12:48:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Jul 11, 2021 12:48:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Jul 11, 2021 12:48:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Jul 11, 2021 12:48:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Jul 11, 2021 12:48:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Jul 11, 2021 12:48:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Jul 11, 2021 12:48:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Jul 11, 2021 12:48:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Jul 11, 2021 12:48:32 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.32.0-SNAPSHOT
Jul 11, 2021 12:48:33 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-07-11_05_48_32-7063727085335206007?project=apache-beam-testing
Jul 11, 2021 12:48:33 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-07-11_05_48_32-7063727085335206007
Jul 11, 2021 12:48:33 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-07-11_05_48_32-7063727085335206007
Jul 11, 2021 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-07-11T12:48:37.424Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-07-tlv0. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Jul 11, 2021 12:48:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-11T12:48:42.451Z: Worker configuration: n1-standard-2 in us-central1-a.
Jul 11, 2021 12:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-11T12:48:43.042Z: Expanding SplittableParDo operations into optimizable parts.
Jul 11, 2021 12:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-11T12:48:43.063Z: Expanding CollectionToSingleton operations into optimizable parts.
Jul 11, 2021 12:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-11T12:48:43.130Z: Expanding CoGroupByKey operations into optimizable parts.
Jul 11, 2021 12:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-11T12:48:43.203Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Jul 11, 2021 12:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-11T12:48:43.222Z: Expanding GroupByKey operations into streaming Read/Write steps
Jul 11, 2021 12:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-11T12:48:43.281Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jul 11, 2021 12:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-11T12:48:43.396Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jul 11, 2021 12:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-11T12:48:43.431Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Jul 11, 2021 12:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-11T12:48:43.460Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Jul 11, 2021 12:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-11T12:48:43.491Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Jul 11, 2021 12:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-11T12:48:43.528Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Jul 11, 2021 12:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-11T12:48:43.549Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jul 11, 2021 12:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-11T12:48:43.571Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jul 11, 2021 12:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-11T12:48:43.605Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jul 11, 2021 12:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-11T12:48:43.640Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Jul 11, 2021 12:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-11T12:48:43.669Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Jul 11, 2021 12:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-11T12:48:43.701Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Jul 11, 2021 12:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-11T12:48:43.733Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Jul 11, 2021 12:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-11T12:48:43.760Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jul 11, 2021 12:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-11T12:48:43.782Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jul 11, 2021 12:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-11T12:48:43.823Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jul 11, 2021 12:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-11T12:48:43.856Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Jul 11, 2021 12:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-11T12:48:43.893Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Jul 11, 2021 12:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-11T12:48:43.924Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Jul 11, 2021 12:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-11T12:48:43.959Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Jul 11, 2021 12:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-11T12:48:43.993Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Jul 11, 2021 12:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-11T12:48:44.026Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Jul 11, 2021 12:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-11T12:48:44.061Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Jul 11, 2021 12:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-11T12:48:44.094Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Jul 11, 2021 12:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-11T12:48:44.472Z: Starting 5 ****s in us-central1-a...
Jul 11, 2021 12:48:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-11T12:48:50.278Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jul 11, 2021 12:50:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-11T12:50:03.146Z: Workers have started successfully.
Jul 11, 2021 12:50:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-11T12:50:03.190Z: Workers have started successfully.
Jul 11, 2021 4:00:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-11T16:00:40.177Z: Cancel request is committed for workflow job: 2021-07-11_05_48_32-7063727085335206007.
Jul 11, 2021 4:00:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-11T16:00:40.285Z: Cleaning up.
Jul 11, 2021 4:00:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-11T16:00:40.359Z: Stopping **** pool...
Jul 11, 2021 4:00:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-11T16:00:40.405Z: Stopping **** pool...
Jul 11, 2021 4:01:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-11T16:01:22.926Z: Worker pool stopped.
Jul 11, 2021 4:01:28 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-07-11_05_48_32-7063727085335206007 finished with status CANCELLED.
Load test results for test (ID): 9923c4d3-6afa-49aa-9470-cb190b42177a and timestamp: 2021-07-11T12:48:26.467000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11385.938
dataflow_v2_java11_total_bytes_count               4.6689778E9
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210711124335
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:24b8d658f02d2fb4f5773ff0d74331512825f616db5a854fd1327097d0b670e4
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:24b8d658f02d2fb4f5773ff0d74331512825f616db5a854fd1327097d0b670e4
  Associated tags:
 - 20210711124335
Tags:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210711124335
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210711124335].
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:24b8d658f02d2fb4f5773ff0d74331512825f616db5a854fd1327097d0b670e4].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 18m 10s
107 actionable tasks: 76 executed, 29 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/wvokloy7lyuf6

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #23

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/23/display/redirect?page=changes>

Changes:

[samuelw] [BEAM-12516] - Fix exception from ShardedKey.toString if key is less

[Andrew Pilloud] Cleanup cruft from #14239

[mouyang] [BEAM-12532] correctly handle LocalDateTime and LocalTime with 0 seconds

[mouyang] [BEAM-12532] ./gradlew spotlessApply

[Andrew Pilloud] Fix typo in 2.31.0 on website

[noreply] Swap length() for size()

[zyichi] [BEAM-12594] Remove google-cloud-profiler package from apache-beam

[noreply] [BEAM-8933] Add support for reading Arrow over the BigQuery Storage API

[noreply] [BEAM-12548] Add AllWithinBounds() check to PAssert (#15136)

[noreply] docs: remove extra backtick (#15149)

[Andrew Pilloud] Fix broken download links


------------------------------------------
[...truncated 65.55 KB...]
registries, e.g. `docker pull gcr.io/project-id/my-image`. Add
`--verbosity=error` to silence this warning: `gcloud docker
--verbosity=error -- pull gcr.io/project-id/my-image`.

See: https://cloud.google.com/container-registry/docs/support/deprecation-notices#gcloud-docker

The push refers to repository [us.gcr.io/apache-beam-testing/java-postcommit-it/java]
db3bd8b8663d: Preparing
8e0973bb0966: Preparing
b301fcfaf9e9: Preparing
ac0ad0d7f052: Preparing
de74961b1510: Preparing
a7a17a73ee1a: Preparing
b0870fca71ca: Preparing
3542249770cc: Preparing
9a3a4acb7ac7: Preparing
09ebcf1a0d26: Preparing
f267ff60afb8: Preparing
7d06e90af879: Preparing
6ea995e9b7d3: Preparing
c0848348e2f7: Preparing
79c550eb7bd2: Preparing
7095af798ace: Preparing
fe6a4fdbedc0: Preparing
e4d0e810d54a: Preparing
4e006334a6fd: Preparing
3542249770cc: Waiting
9a3a4acb7ac7: Waiting
f267ff60afb8: Waiting
79c550eb7bd2: Waiting
09ebcf1a0d26: Waiting
b0870fca71ca: Waiting
e4d0e810d54a: Waiting
7d06e90af879: Waiting
c0848348e2f7: Waiting
fe6a4fdbedc0: Waiting
a7a17a73ee1a: Waiting
de74961b1510: Pushed
8e0973bb0966: Pushed
b301fcfaf9e9: Pushed
a7a17a73ee1a: Pushed
ac0ad0d7f052: Pushed
3542249770cc: Pushed
db3bd8b8663d: Pushed
9a3a4acb7ac7: Pushed
6ea995e9b7d3: Layer already exists
7d06e90af879: Pushed
f267ff60afb8: Pushed
c0848348e2f7: Layer already exists
79c550eb7bd2: Layer already exists
7095af798ace: Layer already exists
fe6a4fdbedc0: Layer already exists
e4d0e810d54a: Layer already exists
4e006334a6fd: Layer already exists
b0870fca71ca: Pushed
09ebcf1a0d26: Pushed
20210710124334: digest: sha256:54f30a40d4fe0899b64a9ed895888aee11a048642665c3ac97712d933047a1b8 size: 4310

> Task :sdks:java:testing:load-tests:run
Jul 10, 2021 12:48:26 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Jul 10, 2021 12:48:26 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 194 files. Enable logging at DEBUG level to see which files will be staged.
Jul 10, 2021 12:48:27 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Jul 10, 2021 12:48:27 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Jul 10, 2021 12:48:30 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Jul 10, 2021 12:48:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <113773 bytes, hash 66e0b18a1d64955a877e7e6489bcd8e3ffb17bf7aab26aab4616334ce096ad10> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-ZuCxih1klVqHfn5kibzY4_-xe_eqsmqrRhYzTOCWrRA.pb
Jul 10, 2021 12:48:32 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 194 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Jul 10, 2021 12:48:32 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 194 files cached, 0 files newly uploaded in 0 seconds
Jul 10, 2021 12:48:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Jul 10, 2021 12:48:32 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2b6fcb9f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@75de6341, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@74170687, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@68f0f72c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3d96fa9e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3b545206, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77bb48d5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@181d8899, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12d5c30e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@b887730, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26586b74, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@52f57666, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6e041285, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@267dc982, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@439b15f2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3aa41da1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@74fab04a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c83468e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@272ce069, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3d7fb838]
Jul 10, 2021 12:48:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Jul 10, 2021 12:48:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Jul 10, 2021 12:48:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Jul 10, 2021 12:48:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Jul 10, 2021 12:48:33 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@32177fa5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@a96d56c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6ab4a5b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2abe9173, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@235d29d6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1fdca564, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43f9dd56, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1d12e953, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@57cb70be, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2d4608a6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@20d87335, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2a8a4e0c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26c89563, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3bd6ba24, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58f437b0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@20f6f88c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4277127c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4c7e978c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@354e7004, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@22ebccb9]
Jul 10, 2021 12:48:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Jul 10, 2021 12:48:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Jul 10, 2021 12:48:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Jul 10, 2021 12:48:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Jul 10, 2021 12:48:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Jul 10, 2021 12:48:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Jul 10, 2021 12:48:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Jul 10, 2021 12:48:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Jul 10, 2021 12:48:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Jul 10, 2021 12:48:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Jul 10, 2021 12:48:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Jul 10, 2021 12:48:33 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.32.0-SNAPSHOT
Jul 10, 2021 12:48:34 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-07-10_05_48_33-12289940991114653624?project=apache-beam-testing
Jul 10, 2021 12:48:34 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-07-10_05_48_33-12289940991114653624
Jul 10, 2021 12:48:34 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-07-10_05_48_33-12289940991114653624
Jul 10, 2021 12:48:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-07-10T12:48:38.716Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-07-3pai. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Jul 10, 2021 12:48:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-10T12:48:43.122Z: Worker configuration: n1-standard-2 in us-central1-a.
Jul 10, 2021 12:48:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-10T12:48:43.731Z: Expanding SplittableParDo operations into optimizable parts.
Jul 10, 2021 12:48:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-10T12:48:43.759Z: Expanding CollectionToSingleton operations into optimizable parts.
Jul 10, 2021 12:48:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-10T12:48:43.835Z: Expanding CoGroupByKey operations into optimizable parts.
Jul 10, 2021 12:48:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-10T12:48:43.905Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Jul 10, 2021 12:48:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-10T12:48:43.933Z: Expanding GroupByKey operations into streaming Read/Write steps
Jul 10, 2021 12:48:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-10T12:48:43.992Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jul 10, 2021 12:48:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-10T12:48:44.106Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jul 10, 2021 12:48:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-10T12:48:44.141Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Jul 10, 2021 12:48:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-10T12:48:44.174Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Jul 10, 2021 12:48:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-10T12:48:44.196Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Jul 10, 2021 12:48:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-10T12:48:44.230Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Jul 10, 2021 12:48:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-10T12:48:44.263Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jul 10, 2021 12:48:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-10T12:48:44.301Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jul 10, 2021 12:48:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-10T12:48:44.338Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jul 10, 2021 12:48:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-10T12:48:44.374Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Jul 10, 2021 12:48:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-10T12:48:44.406Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Jul 10, 2021 12:48:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-10T12:48:44.441Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Jul 10, 2021 12:48:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-10T12:48:44.474Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Jul 10, 2021 12:48:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-10T12:48:44.512Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jul 10, 2021 12:48:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-10T12:48:44.569Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jul 10, 2021 12:48:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-10T12:48:44.607Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jul 10, 2021 12:48:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-10T12:48:44.639Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Jul 10, 2021 12:48:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-10T12:48:44.683Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Jul 10, 2021 12:48:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-10T12:48:44.724Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Jul 10, 2021 12:48:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-10T12:48:44.764Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Jul 10, 2021 12:48:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-10T12:48:44.797Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Jul 10, 2021 12:48:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-10T12:48:44.833Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Jul 10, 2021 12:48:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-10T12:48:44.864Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Jul 10, 2021 12:48:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-10T12:48:44.898Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Jul 10, 2021 12:48:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-10T12:48:45.304Z: Starting 5 ****s in us-central1-a...
Jul 10, 2021 12:49:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-10T12:48:59.254Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jul 10, 2021 12:50:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-10T12:50:30.093Z: Workers have started successfully.
Jul 10, 2021 12:50:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-10T12:50:30.124Z: Workers have started successfully.
Jul 10, 2021 4:00:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-10T16:00:34.690Z: Cancel request is committed for workflow job: 2021-07-10_05_48_33-12289940991114653624.
Jul 10, 2021 4:00:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-10T16:00:34.774Z: Cleaning up.
Jul 10, 2021 4:00:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-10T16:00:34.850Z: Stopping **** pool...
Jul 10, 2021 4:00:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-10T16:00:34.903Z: Stopping **** pool...
Jul 10, 2021 4:01:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-10T16:01:17.459Z: Worker pool stopped.
Jul 10, 2021 4:01:23 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-07-10_05_48_33-12289940991114653624 finished with status CANCELLED.
Load test results for test (ID): a60d9add-e197-48e1-9add-5de4b34bcde8 and timestamp: 2021-07-10T12:48:27.278000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11374.884
dataflow_v2_java11_total_bytes_count               5.1530709E9
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210710124334
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:54f30a40d4fe0899b64a9ed895888aee11a048642665c3ac97712d933047a1b8
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:54f30a40d4fe0899b64a9ed895888aee11a048642665c3ac97712d933047a1b8
  Associated tags:
 - 20210710124334
Tags:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210710124334
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210710124334].
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:54f30a40d4fe0899b64a9ed895888aee11a048642665c3ac97712d933047a1b8].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 18m 6s
107 actionable tasks: 78 executed, 27 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/sixprvdujzx5s

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #22

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/22/display/redirect?page=changes>

Changes:

[raphael.sanamyan] [BEAM-12511] Add ability to use JdbcIO.Write.withResults without

[Luke Cwik] [BEAM-12547] Consume vendored gRPC 1.36.0 0.2 and fix test to ensure

[noreply] Merge pull request #15135: [BEAM-10887] Timer clear Reapply timer clear

[Andrew Pilloud] Update Beam website to release 2.31.0.

[noreply] [BEAM-9547] Add implementations for some lesser-used pandas operations

[noreply] Merge pull request #15054: Fix broken build.gradle release tag

[noreply] [BEAM-12513] Finish adding Go to BPG 4. Transforms (#15142)

[noreply] [BEAM-12548] Add unit tests for passert.go (#15141)

[noreply] [BEAM-9547] Identify remaining WontImplement operations (#15095)


------------------------------------------
[...truncated 76.73 KB...]
Jul 09, 2021 12:48:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-09T12:48:47.924Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Jul 09, 2021 12:48:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-09T12:48:47.966Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Jul 09, 2021 12:48:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-09T12:48:47.990Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jul 09, 2021 12:48:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-09T12:48:48.013Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jul 09, 2021 12:48:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-09T12:48:48.049Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jul 09, 2021 12:48:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-09T12:48:48.086Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Jul 09, 2021 12:48:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-09T12:48:48.142Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Jul 09, 2021 12:48:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-09T12:48:48.172Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Jul 09, 2021 12:48:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-09T12:48:48.195Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Jul 09, 2021 12:48:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-09T12:48:48.235Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jul 09, 2021 12:48:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-09T12:48:48.283Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jul 09, 2021 12:48:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-09T12:48:48.343Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jul 09, 2021 12:48:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-09T12:48:48.374Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Jul 09, 2021 12:48:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-09T12:48:48.408Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Jul 09, 2021 12:48:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-09T12:48:48.444Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Jul 09, 2021 12:48:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-09T12:48:48.480Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Jul 09, 2021 12:48:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-09T12:48:48.515Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Jul 09, 2021 12:48:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-09T12:48:48.577Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Jul 09, 2021 12:48:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-09T12:48:48.611Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Jul 09, 2021 12:48:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-09T12:48:48.643Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Jul 09, 2021 12:48:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-09T12:48:48.988Z: Starting 5 ****s in us-central1-a...
Jul 09, 2021 12:49:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-09T12:49:19.272Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jul 09, 2021 12:50:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-09T12:50:06.006Z: Workers have started successfully.
Jul 09, 2021 12:50:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-09T12:50:06.025Z: Workers have started successfully.
Jul 09, 2021 3:15:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-09T15:15:51.397Z: Staged package cldrdata-k07I6K9W3X5KTQbcDIEsqM0LXyM18f0eR6IaJw-P_xk.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/cldrdata-k07I6K9W3X5KTQbcDIEsqM0LXyM18f0eR6IaJw-P_xk.jar' is inaccessible.
Jul 09, 2021 3:15:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-09T15:15:51.701Z: Staged package dnsns-RGhCDg3GVOQVC2r6ka2N0hmI4eqQH6VobuoAnQ74MnE.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/dnsns-RGhCDg3GVOQVC2r6ka2N0hmI4eqQH6VobuoAnQ74MnE.jar' is inaccessible.
Jul 09, 2021 3:15:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-09T15:15:53.103Z: Staged package icedtea-sound-Bdoi2wYa757-fzq5vconCy4SSQ22ZaOq7yuC98fKPs8.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/icedtea-sound-Bdoi2wYa757-fzq5vconCy4SSQ22ZaOq7yuC98fKPs8.jar' is inaccessible.
Jul 09, 2021 3:15:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-09T15:15:53.191Z: Staged package jaccess-CMbK-IOdQPLKHEqCuDnE-yBk-VpbtVT-hgjbHRUGO78.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/jaccess-CMbK-IOdQPLKHEqCuDnE-yBk-VpbtVT-hgjbHRUGO78.jar' is inaccessible.
Jul 09, 2021 3:15:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-09T15:15:53.431Z: Staged package joda-time-2.10.10-3Y58khhaZ40be5M_MSCbYgPI_6kemIBHWhvgNGuWF-M.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/joda-time-2.10.10-3Y58khhaZ40be5M_MSCbYgPI_6kemIBHWhvgNGuWF-M.jar' is inaccessible.
Jul 09, 2021 3:15:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-09T15:15:53.622Z: Staged package localedata-ae5Z0L6ak4922fztWeWy7ajiWXdG3ubNrwerJRFoFj0.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/localedata-ae5Z0L6ak4922fztWeWy7ajiWXdG3ubNrwerJRFoFj0.jar' is inaccessible.
Jul 09, 2021 3:15:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-09T15:15:53.699Z: Staged package nashorn-XHtz_UehGpYcLTOrATrTnMNVUgEVa_ttoWkPxnVfqTo.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/nashorn-XHtz_UehGpYcLTOrATrTnMNVUgEVa_ttoWkPxnVfqTo.jar' is inaccessible.
Jul 09, 2021 3:15:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-07-09T15:15:54.528Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jul 09, 2021 3:18:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-07-09T15:18:52.568Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jul 09, 2021 3:21:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-09T15:21:49.956Z: Staged package cldrdata-k07I6K9W3X5KTQbcDIEsqM0LXyM18f0eR6IaJw-P_xk.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/cldrdata-k07I6K9W3X5KTQbcDIEsqM0LXyM18f0eR6IaJw-P_xk.jar' is inaccessible.
Jul 09, 2021 3:21:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-09T15:21:50.243Z: Staged package dnsns-RGhCDg3GVOQVC2r6ka2N0hmI4eqQH6VobuoAnQ74MnE.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/dnsns-RGhCDg3GVOQVC2r6ka2N0hmI4eqQH6VobuoAnQ74MnE.jar' is inaccessible.
Jul 09, 2021 3:21:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-09T15:21:51.564Z: Staged package icedtea-sound-Bdoi2wYa757-fzq5vconCy4SSQ22ZaOq7yuC98fKPs8.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/icedtea-sound-Bdoi2wYa757-fzq5vconCy4SSQ22ZaOq7yuC98fKPs8.jar' is inaccessible.
Jul 09, 2021 3:21:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-09T15:21:51.705Z: Staged package jaccess-CMbK-IOdQPLKHEqCuDnE-yBk-VpbtVT-hgjbHRUGO78.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/jaccess-CMbK-IOdQPLKHEqCuDnE-yBk-VpbtVT-hgjbHRUGO78.jar' is inaccessible.
Jul 09, 2021 3:21:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-09T15:21:51.933Z: Staged package joda-time-2.10.10-3Y58khhaZ40be5M_MSCbYgPI_6kemIBHWhvgNGuWF-M.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/joda-time-2.10.10-3Y58khhaZ40be5M_MSCbYgPI_6kemIBHWhvgNGuWF-M.jar' is inaccessible.
Jul 09, 2021 3:21:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-09T15:21:52.112Z: Staged package localedata-ae5Z0L6ak4922fztWeWy7ajiWXdG3ubNrwerJRFoFj0.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/localedata-ae5Z0L6ak4922fztWeWy7ajiWXdG3ubNrwerJRFoFj0.jar' is inaccessible.
Jul 09, 2021 3:21:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-09T15:21:52.183Z: Staged package nashorn-XHtz_UehGpYcLTOrATrTnMNVUgEVa_ttoWkPxnVfqTo.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/nashorn-XHtz_UehGpYcLTOrATrTnMNVUgEVa_ttoWkPxnVfqTo.jar' is inaccessible.
Jul 09, 2021 3:21:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-07-09T15:21:53.016Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jul 09, 2021 3:24:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-07-09T15:24:53.021Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jul 09, 2021 3:27:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-09T15:27:50.020Z: Staged package cldrdata-k07I6K9W3X5KTQbcDIEsqM0LXyM18f0eR6IaJw-P_xk.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/cldrdata-k07I6K9W3X5KTQbcDIEsqM0LXyM18f0eR6IaJw-P_xk.jar' is inaccessible.
Jul 09, 2021 3:27:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-09T15:27:50.352Z: Staged package dnsns-RGhCDg3GVOQVC2r6ka2N0hmI4eqQH6VobuoAnQ74MnE.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/dnsns-RGhCDg3GVOQVC2r6ka2N0hmI4eqQH6VobuoAnQ74MnE.jar' is inaccessible.
Jul 09, 2021 3:27:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-09T15:27:51.628Z: Staged package icedtea-sound-Bdoi2wYa757-fzq5vconCy4SSQ22ZaOq7yuC98fKPs8.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/icedtea-sound-Bdoi2wYa757-fzq5vconCy4SSQ22ZaOq7yuC98fKPs8.jar' is inaccessible.
Jul 09, 2021 3:27:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-09T15:27:51.706Z: Staged package jaccess-CMbK-IOdQPLKHEqCuDnE-yBk-VpbtVT-hgjbHRUGO78.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/jaccess-CMbK-IOdQPLKHEqCuDnE-yBk-VpbtVT-hgjbHRUGO78.jar' is inaccessible.
Jul 09, 2021 3:27:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-09T15:27:51.900Z: Staged package joda-time-2.10.10-3Y58khhaZ40be5M_MSCbYgPI_6kemIBHWhvgNGuWF-M.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/joda-time-2.10.10-3Y58khhaZ40be5M_MSCbYgPI_6kemIBHWhvgNGuWF-M.jar' is inaccessible.
Jul 09, 2021 3:27:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-09T15:27:52.076Z: Staged package localedata-ae5Z0L6ak4922fztWeWy7ajiWXdG3ubNrwerJRFoFj0.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/localedata-ae5Z0L6ak4922fztWeWy7ajiWXdG3ubNrwerJRFoFj0.jar' is inaccessible.
Jul 09, 2021 3:27:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-09T15:27:52.176Z: Staged package nashorn-XHtz_UehGpYcLTOrATrTnMNVUgEVa_ttoWkPxnVfqTo.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/nashorn-XHtz_UehGpYcLTOrATrTnMNVUgEVa_ttoWkPxnVfqTo.jar' is inaccessible.
Jul 09, 2021 3:27:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-07-09T15:27:53.068Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jul 09, 2021 3:30:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-07-09T15:30:52.325Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jul 09, 2021 3:33:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-09T15:33:49.994Z: Staged package cldrdata-k07I6K9W3X5KTQbcDIEsqM0LXyM18f0eR6IaJw-P_xk.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/cldrdata-k07I6K9W3X5KTQbcDIEsqM0LXyM18f0eR6IaJw-P_xk.jar' is inaccessible.
Jul 09, 2021 3:33:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-09T15:33:50.264Z: Staged package dnsns-RGhCDg3GVOQVC2r6ka2N0hmI4eqQH6VobuoAnQ74MnE.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/dnsns-RGhCDg3GVOQVC2r6ka2N0hmI4eqQH6VobuoAnQ74MnE.jar' is inaccessible.
Jul 09, 2021 3:33:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-09T15:33:51.638Z: Staged package icedtea-sound-Bdoi2wYa757-fzq5vconCy4SSQ22ZaOq7yuC98fKPs8.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/icedtea-sound-Bdoi2wYa757-fzq5vconCy4SSQ22ZaOq7yuC98fKPs8.jar' is inaccessible.
Jul 09, 2021 3:33:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-09T15:33:51.788Z: Staged package jaccess-CMbK-IOdQPLKHEqCuDnE-yBk-VpbtVT-hgjbHRUGO78.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/jaccess-CMbK-IOdQPLKHEqCuDnE-yBk-VpbtVT-hgjbHRUGO78.jar' is inaccessible.
Jul 09, 2021 3:33:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-09T15:33:52.043Z: Staged package joda-time-2.10.10-3Y58khhaZ40be5M_MSCbYgPI_6kemIBHWhvgNGuWF-M.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/joda-time-2.10.10-3Y58khhaZ40be5M_MSCbYgPI_6kemIBHWhvgNGuWF-M.jar' is inaccessible.
Jul 09, 2021 3:33:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-09T15:33:52.287Z: Staged package localedata-ae5Z0L6ak4922fztWeWy7ajiWXdG3ubNrwerJRFoFj0.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/localedata-ae5Z0L6ak4922fztWeWy7ajiWXdG3ubNrwerJRFoFj0.jar' is inaccessible.
Jul 09, 2021 3:33:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-09T15:33:52.363Z: Staged package nashorn-XHtz_UehGpYcLTOrATrTnMNVUgEVa_ttoWkPxnVfqTo.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/nashorn-XHtz_UehGpYcLTOrATrTnMNVUgEVa_ttoWkPxnVfqTo.jar' is inaccessible.
Jul 09, 2021 3:33:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-07-09T15:33:53.200Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jul 09, 2021 3:36:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-07-09T15:36:53.205Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jul 09, 2021 3:39:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-09T15:39:49.814Z: Staged package cldrdata-k07I6K9W3X5KTQbcDIEsqM0LXyM18f0eR6IaJw-P_xk.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/cldrdata-k07I6K9W3X5KTQbcDIEsqM0LXyM18f0eR6IaJw-P_xk.jar' is inaccessible.
Jul 09, 2021 3:39:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-09T15:39:50.057Z: Staged package dnsns-RGhCDg3GVOQVC2r6ka2N0hmI4eqQH6VobuoAnQ74MnE.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/dnsns-RGhCDg3GVOQVC2r6ka2N0hmI4eqQH6VobuoAnQ74MnE.jar' is inaccessible.
Jul 09, 2021 3:39:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-09T15:39:51.242Z: Staged package icedtea-sound-Bdoi2wYa757-fzq5vconCy4SSQ22ZaOq7yuC98fKPs8.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/icedtea-sound-Bdoi2wYa757-fzq5vconCy4SSQ22ZaOq7yuC98fKPs8.jar' is inaccessible.
Jul 09, 2021 3:39:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-09T15:39:51.316Z: Staged package jaccess-CMbK-IOdQPLKHEqCuDnE-yBk-VpbtVT-hgjbHRUGO78.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/jaccess-CMbK-IOdQPLKHEqCuDnE-yBk-VpbtVT-hgjbHRUGO78.jar' is inaccessible.
Jul 09, 2021 3:39:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-09T15:39:51.520Z: Staged package joda-time-2.10.10-3Y58khhaZ40be5M_MSCbYgPI_6kemIBHWhvgNGuWF-M.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/joda-time-2.10.10-3Y58khhaZ40be5M_MSCbYgPI_6kemIBHWhvgNGuWF-M.jar' is inaccessible.
Jul 09, 2021 3:39:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-09T15:39:51.668Z: Staged package localedata-ae5Z0L6ak4922fztWeWy7ajiWXdG3ubNrwerJRFoFj0.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/localedata-ae5Z0L6ak4922fztWeWy7ajiWXdG3ubNrwerJRFoFj0.jar' is inaccessible.
Jul 09, 2021 3:39:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-09T15:39:51.763Z: Staged package nashorn-XHtz_UehGpYcLTOrATrTnMNVUgEVa_ttoWkPxnVfqTo.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/nashorn-XHtz_UehGpYcLTOrATrTnMNVUgEVa_ttoWkPxnVfqTo.jar' is inaccessible.
Jul 09, 2021 3:39:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-07-09T15:39:52.444Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jul 09, 2021 3:42:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-07-09T15:42:52.643Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jul 09, 2021 3:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-09T15:45:50.056Z: Staged package cldrdata-k07I6K9W3X5KTQbcDIEsqM0LXyM18f0eR6IaJw-P_xk.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/cldrdata-k07I6K9W3X5KTQbcDIEsqM0LXyM18f0eR6IaJw-P_xk.jar' is inaccessible.
Jul 09, 2021 3:45:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-09T15:45:50.332Z: Staged package dnsns-RGhCDg3GVOQVC2r6ka2N0hmI4eqQH6VobuoAnQ74MnE.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/dnsns-RGhCDg3GVOQVC2r6ka2N0hmI4eqQH6VobuoAnQ74MnE.jar' is inaccessible.
Jul 09, 2021 3:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-09T15:45:51.630Z: Staged package icedtea-sound-Bdoi2wYa757-fzq5vconCy4SSQ22ZaOq7yuC98fKPs8.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/icedtea-sound-Bdoi2wYa757-fzq5vconCy4SSQ22ZaOq7yuC98fKPs8.jar' is inaccessible.
Jul 09, 2021 3:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-09T15:45:51.710Z: Staged package jaccess-CMbK-IOdQPLKHEqCuDnE-yBk-VpbtVT-hgjbHRUGO78.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/jaccess-CMbK-IOdQPLKHEqCuDnE-yBk-VpbtVT-hgjbHRUGO78.jar' is inaccessible.
Jul 09, 2021 3:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-09T15:45:51.915Z: Staged package joda-time-2.10.10-3Y58khhaZ40be5M_MSCbYgPI_6kemIBHWhvgNGuWF-M.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/joda-time-2.10.10-3Y58khhaZ40be5M_MSCbYgPI_6kemIBHWhvgNGuWF-M.jar' is inaccessible.
Jul 09, 2021 3:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-09T15:45:52.048Z: Staged package localedata-ae5Z0L6ak4922fztWeWy7ajiWXdG3ubNrwerJRFoFj0.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/localedata-ae5Z0L6ak4922fztWeWy7ajiWXdG3ubNrwerJRFoFj0.jar' is inaccessible.
Jul 09, 2021 3:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-09T15:45:52.125Z: Staged package nashorn-XHtz_UehGpYcLTOrATrTnMNVUgEVa_ttoWkPxnVfqTo.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/nashorn-XHtz_UehGpYcLTOrATrTnMNVUgEVa_ttoWkPxnVfqTo.jar' is inaccessible.
Jul 09, 2021 3:45:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-07-09T15:45:52.898Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jul 09, 2021 3:48:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-07-09T15:48:53.036Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jul 09, 2021 3:51:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-09T15:51:49.849Z: Staged package cldrdata-k07I6K9W3X5KTQbcDIEsqM0LXyM18f0eR6IaJw-P_xk.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/cldrdata-k07I6K9W3X5KTQbcDIEsqM0LXyM18f0eR6IaJw-P_xk.jar' is inaccessible.
Jul 09, 2021 3:51:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-09T15:51:50.115Z: Staged package dnsns-RGhCDg3GVOQVC2r6ka2N0hmI4eqQH6VobuoAnQ74MnE.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/dnsns-RGhCDg3GVOQVC2r6ka2N0hmI4eqQH6VobuoAnQ74MnE.jar' is inaccessible.
Jul 09, 2021 3:51:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-09T15:51:51.655Z: Staged package icedtea-sound-Bdoi2wYa757-fzq5vconCy4SSQ22ZaOq7yuC98fKPs8.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/icedtea-sound-Bdoi2wYa757-fzq5vconCy4SSQ22ZaOq7yuC98fKPs8.jar' is inaccessible.
Jul 09, 2021 3:51:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-09T15:51:51.744Z: Staged package jaccess-CMbK-IOdQPLKHEqCuDnE-yBk-VpbtVT-hgjbHRUGO78.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/jaccess-CMbK-IOdQPLKHEqCuDnE-yBk-VpbtVT-hgjbHRUGO78.jar' is inaccessible.
Jul 09, 2021 3:51:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-09T15:51:51.937Z: Staged package joda-time-2.10.10-3Y58khhaZ40be5M_MSCbYgPI_6kemIBHWhvgNGuWF-M.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/joda-time-2.10.10-3Y58khhaZ40be5M_MSCbYgPI_6kemIBHWhvgNGuWF-M.jar' is inaccessible.
Jul 09, 2021 3:51:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-09T15:51:52.082Z: Staged package localedata-ae5Z0L6ak4922fztWeWy7ajiWXdG3ubNrwerJRFoFj0.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/localedata-ae5Z0L6ak4922fztWeWy7ajiWXdG3ubNrwerJRFoFj0.jar' is inaccessible.
Jul 09, 2021 3:51:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-09T15:51:52.159Z: Staged package nashorn-XHtz_UehGpYcLTOrATrTnMNVUgEVa_ttoWkPxnVfqTo.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/nashorn-XHtz_UehGpYcLTOrATrTnMNVUgEVa_ttoWkPxnVfqTo.jar' is inaccessible.
Jul 09, 2021 3:51:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-07-09T15:51:52.864Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jul 09, 2021 3:54:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-07-09T15:54:52.397Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jul 09, 2021 3:57:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-09T15:57:49.772Z: Staged package cldrdata-k07I6K9W3X5KTQbcDIEsqM0LXyM18f0eR6IaJw-P_xk.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/cldrdata-k07I6K9W3X5KTQbcDIEsqM0LXyM18f0eR6IaJw-P_xk.jar' is inaccessible.
Jul 09, 2021 3:57:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-09T15:57:50.041Z: Staged package dnsns-RGhCDg3GVOQVC2r6ka2N0hmI4eqQH6VobuoAnQ74MnE.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/dnsns-RGhCDg3GVOQVC2r6ka2N0hmI4eqQH6VobuoAnQ74MnE.jar' is inaccessible.
Jul 09, 2021 3:57:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-09T15:57:51.322Z: Staged package icedtea-sound-Bdoi2wYa757-fzq5vconCy4SSQ22ZaOq7yuC98fKPs8.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/icedtea-sound-Bdoi2wYa757-fzq5vconCy4SSQ22ZaOq7yuC98fKPs8.jar' is inaccessible.
Jul 09, 2021 3:57:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-09T15:57:51.397Z: Staged package jaccess-CMbK-IOdQPLKHEqCuDnE-yBk-VpbtVT-hgjbHRUGO78.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/jaccess-CMbK-IOdQPLKHEqCuDnE-yBk-VpbtVT-hgjbHRUGO78.jar' is inaccessible.
Jul 09, 2021 3:57:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-09T15:57:51.600Z: Staged package joda-time-2.10.10-3Y58khhaZ40be5M_MSCbYgPI_6kemIBHWhvgNGuWF-M.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/joda-time-2.10.10-3Y58khhaZ40be5M_MSCbYgPI_6kemIBHWhvgNGuWF-M.jar' is inaccessible.
Jul 09, 2021 3:57:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-09T15:57:51.753Z: Staged package localedata-ae5Z0L6ak4922fztWeWy7ajiWXdG3ubNrwerJRFoFj0.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/localedata-ae5Z0L6ak4922fztWeWy7ajiWXdG3ubNrwerJRFoFj0.jar' is inaccessible.
Jul 09, 2021 3:57:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-09T15:57:51.848Z: Staged package nashorn-XHtz_UehGpYcLTOrATrTnMNVUgEVa_ttoWkPxnVfqTo.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/nashorn-XHtz_UehGpYcLTOrATrTnMNVUgEVa_ttoWkPxnVfqTo.jar' is inaccessible.
Jul 09, 2021 3:57:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-07-09T15:57:52.499Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jul 09, 2021 4:00:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-09T16:00:32.765Z: Cancel request is committed for workflow job: 2021-07-09_05_48_37-4416048604304487553.
Jul 09, 2021 4:00:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-09T16:00:32.795Z: Cleaning up.
Jul 09, 2021 4:00:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-09T16:00:32.846Z: Stopping **** pool...
Jul 09, 2021 4:00:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-09T16:00:32.891Z: Stopping **** pool...
Jul 09, 2021 4:01:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-09T16:01:27.984Z: Worker pool stopped.
Jul 09, 2021 4:01:33 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-07-09_05_48_37-4416048604304487553 finished with status CANCELLED.
Load test results for test (ID): 26eb2829-f701-4dd0-b67e-a0c82161f0ef and timestamp: 2021-07-09T12:48:32.131000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11378.425
dataflow_v2_java11_total_bytes_count               4.9226029E9
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210709124335
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:c61c56e3e50a2b39b4998f3cdfa4b6d8c9ebf5ad068c9f0d303e7e655f4bc03e
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:c61c56e3e50a2b39b4998f3cdfa4b6d8c9ebf5ad068c9f0d303e7e655f4bc03e
  Associated tags:
 - 20210709124335
Tags:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210709124335
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210709124335].
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:c61c56e3e50a2b39b4998f3cdfa4b6d8c9ebf5ad068c9f0d303e7e655f4bc03e].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 18m 15s
104 actionable tasks: 76 executed, 26 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/zft7rdsasaq7y

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #21

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/21/display/redirect?page=changes>

Changes:

[noreply] Revert "[BEAM-12119] [BEAM-12122] Add integer and string `_id` keys

[Valentyn Tymofieiev] Fix a Py3 error in coders example.

[tysonjh] Update Python container image to 20210702.

[noreply] [BEAM-12548] Add unit tests for sum function to passert package (#15131)

[noreply] Merge pull request #15056: [BEAM-10887] Timer clear

[noreply] Revert "Merge pull request #15056: [BEAM-10887] Timer clear"

[noreply] [BEAM-12584] Fix sample and make tag in documentation point to correct

[noreply] Skip BQ tests if no BQ client is installed (#15124)

[ajamato] [BEAM-12583] Fix HarnessMonitoringInfoInstructionHandler to us the short


------------------------------------------
[...truncated 64.23 KB...]
--verbosity=error -- pull gcr.io/project-id/my-image`.

See: https://cloud.google.com/container-registry/docs/support/deprecation-notices#gcloud-docker

The push refers to repository [us.gcr.io/apache-beam-testing/java-postcommit-it/java]
8507dcaac242: Preparing
bd8f1e59d178: Preparing
403d039f87be: Preparing
3a621f7aa899: Preparing
19b280cac34e: Preparing
604efa1fd869: Preparing
36c5d9d4e76a: Preparing
6d947e16b7cd: Preparing
15041a753962: Preparing
1b2fa45b9431: Preparing
916467c25001: Preparing
24f06cadf59c: Preparing
6ea995e9b7d3: Preparing
c0848348e2f7: Preparing
79c550eb7bd2: Preparing
7095af798ace: Preparing
fe6a4fdbedc0: Preparing
e4d0e810d54a: Preparing
4e006334a6fd: Preparing
79c550eb7bd2: Waiting
15041a753962: Waiting
7095af798ace: Waiting
fe6a4fdbedc0: Waiting
6ea995e9b7d3: Waiting
e4d0e810d54a: Waiting
c0848348e2f7: Waiting
4e006334a6fd: Waiting
1b2fa45b9431: Waiting
24f06cadf59c: Waiting
916467c25001: Waiting
604efa1fd869: Waiting
6d947e16b7cd: Waiting
403d039f87be: Pushed
bd8f1e59d178: Pushed
19b280cac34e: Pushed
604efa1fd869: Pushed
8507dcaac242: Pushed
3a621f7aa899: Pushed
6d947e16b7cd: Pushed
15041a753962: Pushed
6ea995e9b7d3: Layer already exists
c0848348e2f7: Layer already exists
916467c25001: Pushed
79c550eb7bd2: Layer already exists
7095af798ace: Layer already exists
fe6a4fdbedc0: Layer already exists
e4d0e810d54a: Layer already exists
4e006334a6fd: Layer already exists
24f06cadf59c: Pushed
36c5d9d4e76a: Pushed
1b2fa45b9431: Pushed
20210708124332: digest: sha256:f004eed7894fd49fb22a1ac222ea42c383728edbd09fe1b6ab3a667b92928f35 size: 4310

> Task :sdks:java:testing:load-tests:run
Jul 08, 2021 12:48:06 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Jul 08, 2021 12:48:07 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 190 files. Enable logging at DEBUG level to see which files will be staged.
Jul 08, 2021 12:48:08 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Jul 08, 2021 12:48:08 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Jul 08, 2021 12:48:10 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Jul 08, 2021 12:48:10 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <112311 bytes, hash 798a7216ce4961d036ac44a8f3ea3f81af74b0df58a78cd0b737620182faedc9> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-eYpyFs5JYdA2rESo8-o_ga90sN9Yp4zQtzdiAYL67ck.pb
Jul 08, 2021 12:48:11 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 190 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Jul 08, 2021 12:48:12 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 190 files cached, 0 files newly uploaded in 0 seconds
Jul 08, 2021 12:48:12 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Jul 08, 2021 12:48:12 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@21505815, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7e53339, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58fef7f7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@31ddb930, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12eedfee, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c3c4a71, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1352434e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4f9a6c2d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2b6fcb9f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@75de6341, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@74170687, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@68f0f72c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3d96fa9e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3b545206, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77bb48d5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@181d8899, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12d5c30e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@b887730, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26586b74, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@52f57666]
Jul 08, 2021 12:48:12 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Jul 08, 2021 12:48:12 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Jul 08, 2021 12:48:12 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Jul 08, 2021 12:48:12 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Jul 08, 2021 12:48:12 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@9679750, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@9b9a327, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@75a0c890, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@358ab600, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@e26af6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@44065156, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3791f50e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@574b7f4a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@44ed0a8f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@32177fa5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@a96d56c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6ab4a5b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2abe9173, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@235d29d6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1fdca564, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43f9dd56, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1d12e953, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@57cb70be, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2d4608a6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@20d87335]
Jul 08, 2021 12:48:12 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Jul 08, 2021 12:48:12 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Jul 08, 2021 12:48:12 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Jul 08, 2021 12:48:12 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Jul 08, 2021 12:48:12 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Jul 08, 2021 12:48:12 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Jul 08, 2021 12:48:12 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Jul 08, 2021 12:48:12 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Jul 08, 2021 12:48:12 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Jul 08, 2021 12:48:12 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Jul 08, 2021 12:48:12 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Jul 08, 2021 12:48:12 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.32.0-SNAPSHOT
Jul 08, 2021 12:48:13 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-07-08_05_48_12-15508621589466096840?project=apache-beam-testing
Jul 08, 2021 12:48:13 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-07-08_05_48_12-15508621589466096840
Jul 08, 2021 12:48:13 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-07-08_05_48_12-15508621589466096840
Jul 08, 2021 12:48:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-07-08T12:48:18.509Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-07-z77e. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Jul 08, 2021 12:48:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-08T12:48:25.811Z: Worker configuration: n1-standard-2 in us-central1-f.
Jul 08, 2021 12:48:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-08T12:48:26.430Z: Expanding SplittableParDo operations into optimizable parts.
Jul 08, 2021 12:48:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-08T12:48:26.466Z: Expanding CollectionToSingleton operations into optimizable parts.
Jul 08, 2021 12:48:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-08T12:48:26.523Z: Expanding CoGroupByKey operations into optimizable parts.
Jul 08, 2021 12:48:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-08T12:48:26.603Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Jul 08, 2021 12:48:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-08T12:48:26.644Z: Expanding GroupByKey operations into streaming Read/Write steps
Jul 08, 2021 12:48:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-08T12:48:26.703Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jul 08, 2021 12:48:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-08T12:48:26.816Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jul 08, 2021 12:48:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-08T12:48:26.845Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Jul 08, 2021 12:48:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-08T12:48:26.889Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Jul 08, 2021 12:48:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-08T12:48:26.925Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Jul 08, 2021 12:48:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-08T12:48:26.980Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Jul 08, 2021 12:48:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-08T12:48:27.011Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jul 08, 2021 12:48:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-08T12:48:27.045Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jul 08, 2021 12:48:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-08T12:48:27.066Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jul 08, 2021 12:48:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-08T12:48:27.089Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Jul 08, 2021 12:48:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-08T12:48:27.122Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Jul 08, 2021 12:48:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-08T12:48:27.143Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Jul 08, 2021 12:48:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-08T12:48:27.169Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Jul 08, 2021 12:48:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-08T12:48:27.216Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jul 08, 2021 12:48:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-08T12:48:27.248Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jul 08, 2021 12:48:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-08T12:48:27.297Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jul 08, 2021 12:48:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-08T12:48:27.331Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Jul 08, 2021 12:48:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-08T12:48:27.353Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Jul 08, 2021 12:48:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-08T12:48:27.375Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Jul 08, 2021 12:48:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-08T12:48:27.397Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Jul 08, 2021 12:48:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-08T12:48:27.421Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Jul 08, 2021 12:48:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-08T12:48:27.457Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Jul 08, 2021 12:48:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-08T12:48:27.490Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Jul 08, 2021 12:48:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-08T12:48:27.511Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Jul 08, 2021 12:48:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-08T12:48:27.893Z: Starting 5 ****s in us-central1-f...
Jul 08, 2021 12:48:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-08T12:48:34.591Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jul 08, 2021 12:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-08T12:49:49.030Z: Workers have started successfully.
Jul 08, 2021 12:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-08T12:49:49.065Z: Workers have started successfully.
Jul 08, 2021 4:00:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-08T16:00:33.936Z: Cancel request is committed for workflow job: 2021-07-08_05_48_12-15508621589466096840.
Jul 08, 2021 4:00:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-08T16:00:34.009Z: Cleaning up.
Jul 08, 2021 4:00:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-08T16:00:34.109Z: Stopping **** pool...
Jul 08, 2021 4:00:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-08T16:00:34.164Z: Stopping **** pool...
Jul 08, 2021 4:01:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-08T16:01:27.347Z: Worker pool stopped.
Jul 08, 2021 4:01:34 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-07-08_05_48_12-15508621589466096840 finished with status CANCELLED.
Load test results for test (ID): c751f678-cc5f-4689-8fce-64b804176953 and timestamp: 2021-07-08T12:48:07.677000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                  11404.72
dataflow_v2_java11_total_bytes_count               6.5488269E9
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210708124332
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:f004eed7894fd49fb22a1ac222ea42c383728edbd09fe1b6ab3a667b92928f35
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:f004eed7894fd49fb22a1ac222ea42c383728edbd09fe1b6ab3a667b92928f35
  Associated tags:
 - 20210708124332
Tags:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210708124332
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210708124332].
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:f004eed7894fd49fb22a1ac222ea42c383728edbd09fe1b6ab3a667b92928f35].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 18m 17s
104 actionable tasks: 74 executed, 28 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/hmjcamy5lq6i4

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #20

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/20/display/redirect?page=changes>

Changes:

[Robert Bradshaw] Fix typo in dataframes docs.

[aromanenko.dev] [BEAM-12583] Ignore testReturnsProcessWideMonitoringInfos() until it


------------------------------------------
[...truncated 63.91 KB...]
6ea995e9b7d3: Waiting
f4d99b6fbeb3: Waiting
c0848348e2f7: Waiting
7095af798ace: Waiting
2f08997b94de: Waiting
e4d0e810d54a: Waiting
79c550eb7bd2: Waiting
ec2e07daecf1: Waiting
4acaec977e3f: Waiting
4f7f80050055: Waiting
fe6a4fdbedc0: Waiting
bd6762227996: Waiting
4e006334a6fd: Waiting
d7e9d5256fa6: Pushed
7374a18c84be: Pushed
0297ef69f906: Pushed
0f77989f3ab8: Pushed
2f08997b94de: Pushed
a779f01567a2: Pushed
3219a985e39b: Pushed
ec2e07daecf1: Pushed
6ea995e9b7d3: Layer already exists
c0848348e2f7: Layer already exists
79c550eb7bd2: Layer already exists
7095af798ace: Layer already exists
4acaec977e3f: Pushed
fe6a4fdbedc0: Layer already exists
e4d0e810d54a: Layer already exists
4e006334a6fd: Layer already exists
4f7f80050055: Pushed
f4d99b6fbeb3: Pushed
bd6762227996: Pushed
20210707124333: digest: sha256:ea161a744acead978a167de780450fc1088be44bdbe77f03128024743a8cc286 size: 4310

> Task :sdks:java:testing:load-tests:run
Jul 07, 2021 12:48:13 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Jul 07, 2021 12:48:13 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 190 files. Enable logging at DEBUG level to see which files will be staged.
Jul 07, 2021 12:48:14 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Jul 07, 2021 12:48:14 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Jul 07, 2021 12:48:16 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Jul 07, 2021 12:48:17 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <112311 bytes, hash 0b0e189c950b1a56451fb5e83ccc2639f9583f676e979236ea90635de90c2ba4> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-Cw4YnJULGlZFH7XoPMwmOflYP2dul5I26pBjXekMK6Q.pb
Jul 07, 2021 12:48:18 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 190 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Jul 07, 2021 12:48:19 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 190 files cached, 0 files newly uploaded in 0 seconds
Jul 07, 2021 12:48:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Jul 07, 2021 12:48:19 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@21505815, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7e53339, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58fef7f7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@31ddb930, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12eedfee, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c3c4a71, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1352434e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4f9a6c2d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2b6fcb9f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@75de6341, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@74170687, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@68f0f72c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3d96fa9e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3b545206, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77bb48d5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@181d8899, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12d5c30e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@b887730, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26586b74, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@52f57666]
Jul 07, 2021 12:48:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Jul 07, 2021 12:48:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Jul 07, 2021 12:48:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Jul 07, 2021 12:48:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Jul 07, 2021 12:48:19 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@9b9a327, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@75a0c890, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@358ab600, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@e26af6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@44065156, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3791f50e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@574b7f4a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@44ed0a8f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@32177fa5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@a96d56c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6ab4a5b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2abe9173, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@235d29d6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1fdca564, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43f9dd56, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1d12e953, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@57cb70be, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2d4608a6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@20d87335, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2a8a4e0c]
Jul 07, 2021 12:48:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Jul 07, 2021 12:48:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Jul 07, 2021 12:48:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Jul 07, 2021 12:48:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Jul 07, 2021 12:48:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Jul 07, 2021 12:48:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Jul 07, 2021 12:48:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Jul 07, 2021 12:48:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Jul 07, 2021 12:48:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Jul 07, 2021 12:48:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Jul 07, 2021 12:48:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Jul 07, 2021 12:48:19 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.32.0-SNAPSHOT
Jul 07, 2021 12:48:20 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-07-07_05_48_19-5692894373051920154?project=apache-beam-testing
Jul 07, 2021 12:48:20 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-07-07_05_48_19-5692894373051920154
Jul 07, 2021 12:48:20 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-07-07_05_48_19-5692894373051920154
Jul 07, 2021 12:48:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-07-07T12:48:25.743Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-07-gbwm. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Jul 07, 2021 12:48:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-07T12:48:29.643Z: Worker configuration: n1-standard-2 in us-central1-f.
Jul 07, 2021 12:48:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-07T12:48:30.401Z: Expanding SplittableParDo operations into optimizable parts.
Jul 07, 2021 12:48:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-07T12:48:30.436Z: Expanding CollectionToSingleton operations into optimizable parts.
Jul 07, 2021 12:48:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-07T12:48:30.508Z: Expanding CoGroupByKey operations into optimizable parts.
Jul 07, 2021 12:48:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-07T12:48:30.582Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Jul 07, 2021 12:48:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-07T12:48:30.611Z: Expanding GroupByKey operations into streaming Read/Write steps
Jul 07, 2021 12:48:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-07T12:48:30.687Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jul 07, 2021 12:48:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-07T12:48:30.795Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jul 07, 2021 12:48:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-07T12:48:30.822Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Jul 07, 2021 12:48:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-07T12:48:30.852Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Jul 07, 2021 12:48:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-07T12:48:30.915Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Jul 07, 2021 12:48:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-07T12:48:30.948Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Jul 07, 2021 12:48:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-07T12:48:30.975Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jul 07, 2021 12:48:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-07T12:48:31.006Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jul 07, 2021 12:48:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-07T12:48:31.033Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jul 07, 2021 12:48:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-07T12:48:31.065Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Jul 07, 2021 12:48:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-07T12:48:31.103Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Jul 07, 2021 12:48:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-07T12:48:31.122Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Jul 07, 2021 12:48:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-07T12:48:31.166Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Jul 07, 2021 12:48:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-07T12:48:31.204Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jul 07, 2021 12:48:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-07T12:48:31.241Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jul 07, 2021 12:48:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-07T12:48:31.265Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jul 07, 2021 12:48:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-07T12:48:31.303Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Jul 07, 2021 12:48:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-07T12:48:31.325Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Jul 07, 2021 12:48:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-07T12:48:31.353Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Jul 07, 2021 12:48:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-07T12:48:31.377Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Jul 07, 2021 12:48:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-07T12:48:31.416Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Jul 07, 2021 12:48:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-07T12:48:31.444Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Jul 07, 2021 12:48:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-07T12:48:31.465Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Jul 07, 2021 12:48:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-07T12:48:31.491Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Jul 07, 2021 12:48:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-07T12:48:31.842Z: Starting 5 ****s in us-central1-f...
Jul 07, 2021 12:48:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-07T12:48:50.542Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jul 07, 2021 12:49:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-07T12:49:58.157Z: Workers have started successfully.
Jul 07, 2021 12:49:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-07T12:49:58.189Z: Workers have started successfully.
Jul 07, 2021 3:39:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-07T15:39:32.391Z: Staged package beam-sdks-java-io-kinesis-2.32.0-SNAPSHOT-o6SOFHYXQ-jGvY2B1i53ShUTQWh-3eejpXywmFFOCC4.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-kinesis-2.32.0-SNAPSHOT-o6SOFHYXQ-jGvY2B1i53ShUTQWh-3eejpXywmFFOCC4.jar' is inaccessible.
Jul 07, 2021 3:39:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-07-07T15:39:35.979Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jul 07, 2021 3:42:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-07-07T15:42:34.972Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jul 07, 2021 3:45:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-07T15:45:32.515Z: Staged package beam-sdks-java-io-kinesis-2.32.0-SNAPSHOT-o6SOFHYXQ-jGvY2B1i53ShUTQWh-3eejpXywmFFOCC4.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-kinesis-2.32.0-SNAPSHOT-o6SOFHYXQ-jGvY2B1i53ShUTQWh-3eejpXywmFFOCC4.jar' is inaccessible.
Jul 07, 2021 3:45:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-07-07T15:45:35.406Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jul 07, 2021 3:48:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-07-07T15:48:35.038Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jul 07, 2021 3:51:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-07T15:51:32.416Z: Staged package beam-sdks-java-io-kinesis-2.32.0-SNAPSHOT-o6SOFHYXQ-jGvY2B1i53ShUTQWh-3eejpXywmFFOCC4.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-kinesis-2.32.0-SNAPSHOT-o6SOFHYXQ-jGvY2B1i53ShUTQWh-3eejpXywmFFOCC4.jar' is inaccessible.
Jul 07, 2021 3:51:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-07-07T15:51:35.092Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jul 07, 2021 3:54:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-07-07T15:54:35.351Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jul 07, 2021 3:57:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-07T15:57:32.455Z: Staged package beam-sdks-java-io-kinesis-2.32.0-SNAPSHOT-o6SOFHYXQ-jGvY2B1i53ShUTQWh-3eejpXywmFFOCC4.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-kinesis-2.32.0-SNAPSHOT-o6SOFHYXQ-jGvY2B1i53ShUTQWh-3eejpXywmFFOCC4.jar' is inaccessible.
Jul 07, 2021 3:57:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-07-07T15:57:35.156Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jul 07, 2021 4:00:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-07T16:00:34.190Z: Cancel request is committed for workflow job: 2021-07-07_05_48_19-5692894373051920154.
Jul 07, 2021 4:00:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-07-07T16:00:35.102Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jul 07, 2021 4:00:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-07T16:00:35.132Z: Cleaning up.
Jul 07, 2021 4:00:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-07T16:00:35.207Z: Stopping **** pool...
Jul 07, 2021 4:00:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-07T16:00:35.260Z: Stopping **** pool...
Jul 07, 2021 4:01:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-07T16:01:33.726Z: Worker pool stopped.
Jul 07, 2021 4:01:39 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-07-07_05_48_19-5692894373051920154 finished with status CANCELLED.
Load test results for test (ID): acb31d36-c3f2-4147-854f-d66dc4da0827 and timestamp: 2021-07-07T12:48:13.713000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11381.795
dataflow_v2_java11_total_bytes_count               5.3170986E9
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210707124333
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ea161a744acead978a167de780450fc1088be44bdbe77f03128024743a8cc286
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ea161a744acead978a167de780450fc1088be44bdbe77f03128024743a8cc286
  Associated tags:
 - 20210707124333
Tags:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210707124333
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210707124333].
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ea161a744acead978a167de780450fc1088be44bdbe77f03128024743a8cc286].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 18m 22s
104 actionable tasks: 74 executed, 28 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/yt5rr7a5ij6ue

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #19

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/19/display/redirect?page=changes>

Changes:

[dennisylyung] [BEAM-11907] Full function SqsIO Consumer''

[dennisylyung] Add "test" prefix to tests

[dennisylyung] Separate tests to SqsUnboundedReaderTest and SqsUnboundedSourceTest


------------------------------------------
[...truncated 65.53 KB...]
a4990c201b9a: Pushed
6ea995e9b7d3: Layer already exists
c0848348e2f7: Layer already exists
4c56fd404703: Pushed
79c550eb7bd2: Layer already exists
7095af798ace: Layer already exists
fe6a4fdbedc0: Layer already exists
620c99afa47f: Pushed
e4d0e810d54a: Layer already exists
4e006334a6fd: Layer already exists
0294278906de: Pushed
2635e1766e0c: Pushed
20210706124332: digest: sha256:1e470c9944a147f1e413cef7ce939b281a5e4be5c987604f5da9ab4b0c9022cf size: 4310

> Task :sdks:java:testing:load-tests:run
Jul 06, 2021 12:48:06 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Jul 06, 2021 12:48:06 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 190 files. Enable logging at DEBUG level to see which files will be staged.
Jul 06, 2021 12:48:07 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Jul 06, 2021 12:48:07 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Jul 06, 2021 12:48:09 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Jul 06, 2021 12:48:09 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <112311 bytes, hash 194afb3453e3796f0f67bf2a905ebc2029585a08de6ebaf2f36bb61512b7f40c> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-GUr7NFPjeW8PZ78qkF68IClYWgjebrry82u2FRK39Aw.pb
Jul 06, 2021 12:48:11 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 190 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Jul 06, 2021 12:48:11 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 190 files cached, 0 files newly uploaded in 0 seconds
Jul 06, 2021 12:48:11 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Jul 06, 2021 12:48:11 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58fef7f7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@31ddb930, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12eedfee, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c3c4a71, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1352434e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4f9a6c2d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2b6fcb9f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@75de6341, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@74170687, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@68f0f72c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3d96fa9e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3b545206, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77bb48d5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@181d8899, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12d5c30e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@b887730, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26586b74, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@52f57666, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6e041285, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@267dc982]
Jul 06, 2021 12:48:11 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Jul 06, 2021 12:48:11 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Jul 06, 2021 12:48:11 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Jul 06, 2021 12:48:11 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Jul 06, 2021 12:48:11 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@358ab600, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@e26af6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@44065156, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3791f50e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@574b7f4a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@44ed0a8f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@32177fa5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@a96d56c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6ab4a5b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2abe9173, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@235d29d6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1fdca564, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43f9dd56, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1d12e953, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@57cb70be, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2d4608a6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@20d87335, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2a8a4e0c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26c89563, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3bd6ba24]
Jul 06, 2021 12:48:11 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Jul 06, 2021 12:48:11 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Jul 06, 2021 12:48:11 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Jul 06, 2021 12:48:11 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Jul 06, 2021 12:48:11 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Jul 06, 2021 12:48:11 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Jul 06, 2021 12:48:11 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Jul 06, 2021 12:48:11 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Jul 06, 2021 12:48:11 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Jul 06, 2021 12:48:11 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Jul 06, 2021 12:48:11 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Jul 06, 2021 12:48:11 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.32.0-SNAPSHOT
Jul 06, 2021 12:48:12 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-07-06_05_48_12-3191544166704252678?project=apache-beam-testing
Jul 06, 2021 12:48:12 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-07-06_05_48_12-3191544166704252678
Jul 06, 2021 12:48:12 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-07-06_05_48_12-3191544166704252678
Jul 06, 2021 12:48:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-07-06T12:48:17.139Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-07-lv1f. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Jul 06, 2021 12:48:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-06T12:48:21.787Z: Worker configuration: n1-standard-2 in us-central1-f.
Jul 06, 2021 12:48:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-06T12:48:22.511Z: Expanding SplittableParDo operations into optimizable parts.
Jul 06, 2021 12:48:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-06T12:48:22.551Z: Expanding CollectionToSingleton operations into optimizable parts.
Jul 06, 2021 12:48:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-06T12:48:22.666Z: Expanding CoGroupByKey operations into optimizable parts.
Jul 06, 2021 12:48:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-06T12:48:22.740Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Jul 06, 2021 12:48:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-06T12:48:22.773Z: Expanding GroupByKey operations into streaming Read/Write steps
Jul 06, 2021 12:48:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-06T12:48:22.843Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jul 06, 2021 12:48:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-06T12:48:22.964Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jul 06, 2021 12:48:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-06T12:48:23.002Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Jul 06, 2021 12:48:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-06T12:48:23.032Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Jul 06, 2021 12:48:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-06T12:48:23.068Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Jul 06, 2021 12:48:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-06T12:48:23.101Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Jul 06, 2021 12:48:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-06T12:48:23.134Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jul 06, 2021 12:48:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-06T12:48:23.169Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jul 06, 2021 12:48:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-06T12:48:23.204Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jul 06, 2021 12:48:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-06T12:48:23.237Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Jul 06, 2021 12:48:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-06T12:48:23.272Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Jul 06, 2021 12:48:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-06T12:48:23.303Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Jul 06, 2021 12:48:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-06T12:48:23.341Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Jul 06, 2021 12:48:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-06T12:48:23.366Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jul 06, 2021 12:48:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-06T12:48:23.399Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jul 06, 2021 12:48:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-06T12:48:23.447Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jul 06, 2021 12:48:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-06T12:48:23.475Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Jul 06, 2021 12:48:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-06T12:48:23.510Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Jul 06, 2021 12:48:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-06T12:48:23.533Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Jul 06, 2021 12:48:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-06T12:48:23.567Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Jul 06, 2021 12:48:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-06T12:48:23.602Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Jul 06, 2021 12:48:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-06T12:48:23.640Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Jul 06, 2021 12:48:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-06T12:48:23.672Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Jul 06, 2021 12:48:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-06T12:48:23.700Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Jul 06, 2021 12:48:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-06T12:48:24.129Z: Starting 5 ****s in us-central1-f...
Jul 06, 2021 12:48:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-06T12:48:49.897Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jul 06, 2021 12:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-06T12:49:48.770Z: Workers have started successfully.
Jul 06, 2021 12:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-06T12:49:48.806Z: Workers have started successfully.
Jul 06, 2021 3:48:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-06T15:48:27.203Z: Staged package jackson-dataformat-cbor-2.12.1-52d5rqlCfKc9f0B-X6gIoEBVeOwFZlOoZcJuTG8B1Cg.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/jackson-dataformat-cbor-2.12.1-52d5rqlCfKc9f0B-X6gIoEBVeOwFZlOoZcJuTG8B1Cg.jar' is inaccessible.
Jul 06, 2021 3:48:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-07-06T15:48:28.629Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jul 06, 2021 3:51:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-07-06T15:51:28.242Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jul 06, 2021 3:54:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-06T15:54:27.399Z: Staged package jackson-dataformat-cbor-2.12.1-52d5rqlCfKc9f0B-X6gIoEBVeOwFZlOoZcJuTG8B1Cg.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/jackson-dataformat-cbor-2.12.1-52d5rqlCfKc9f0B-X6gIoEBVeOwFZlOoZcJuTG8B1Cg.jar' is inaccessible.
Jul 06, 2021 3:54:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-07-06T15:54:28.596Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jul 06, 2021 3:57:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-07-06T15:57:28.262Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jul 06, 2021 4:00:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-07-06T16:00:27.461Z: Staged package jackson-dataformat-cbor-2.12.1-52d5rqlCfKc9f0B-X6gIoEBVeOwFZlOoZcJuTG8B1Cg.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/jackson-dataformat-cbor-2.12.1-52d5rqlCfKc9f0B-X6gIoEBVeOwFZlOoZcJuTG8B1Cg.jar' is inaccessible.
Jul 06, 2021 4:00:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-07-06T16:00:28.793Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jul 06, 2021 4:00:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-06T16:00:35.338Z: Cancel request is committed for workflow job: 2021-07-06_05_48_12-3191544166704252678.
Jul 06, 2021 4:00:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-06T16:00:35.381Z: Cleaning up.
Jul 06, 2021 4:00:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-06T16:00:35.444Z: Stopping **** pool...
Jul 06, 2021 4:00:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-06T16:00:35.489Z: Stopping **** pool...
Jul 06, 2021 4:01:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-06T16:01:34.943Z: Worker pool stopped.
Jul 06, 2021 4:01:40 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-07-06_05_48_12-3191544166704252678 finished with status CANCELLED.
Load test results for test (ID): 1bc22d02-7dc9-4a4c-854a-a67f111a1563 and timestamp: 2021-07-06T12:48:07.129000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11401.582
dataflow_v2_java11_total_bytes_count                3.366261E9
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210706124332
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:1e470c9944a147f1e413cef7ce939b281a5e4be5c987604f5da9ab4b0c9022cf
Deleted: sha256:f8e68f5807e3cc377fbc5ae94b194ec4675432ae03f14f6ba472725954614f0a
Deleted: sha256:ef2a3a7b457d9a7e6ff2a0c20623d25ea7008b125c5afd08e92f14a7607acc86
Deleted: sha256:07ae8a7e72530aa68b0461a1733ac2447bf22ace79d92a416bf96e7b3c981fc9
Deleted: sha256:532575bb28eef467faefc6bb2d367c42d55776ce2a4637de4238ae313fc99e07
Deleted: sha256:fd4f108d88d27411664f7b2c2e9d0109abeab69c8eedd4bfce5be8c2c3e3ec8a
Deleted: sha256:1fd3c3c19d7149bab807011b7a33fcd07f7e819d520c435c9594aaa1bccf229e
Deleted: sha256:4146568d612eb3e878105b9e4209fa203425d1bc3549766fe8419a7a8c592922
Deleted: sha256:f4460c35159eb7e27cd8397630dabca4e88fc20d84f84cb1875e8683c670d1d9
Deleted: sha256:fae5cdb1b21b3a97ac38075cef5258b3c1e92bc708b1ba632209ef1f935d56b5
Deleted: sha256:d8e468d2d8934323472b5526d7f1fd9d8bf610d045f3d175433d270539c9bf1e
Deleted: sha256:c8c18c3934f53629148f544d20a8881d3b285fa69141fa441a60268607b9b67d
Deleted: sha256:39c3c368dc2936e73ec89dfea2b1f0c8ad8300b552be6109a7794fe6a41fead3
Deleted: sha256:b58e6a0f2718db74397a5d7c4734b686c04e3a6be349be6af4157ee82624f6fe
Deleted: sha256:d9b8d2cbf304f8f067fd7059040219261a14e439976d5d013c92622aaf8f6062
Deleted: sha256:9ba440bef9c22c3db6657fd0f524c5b1d43738448bb2499cc1fef7fbac1039ea
Deleted: sha256:cb6167bb487a4adb27d060ab19b3e82b21d395e3eaecd4b5bf5f265fd8ccb540
Deleted: sha256:a5554188c4189de28ef4a0fb1e6226a6e8d15e6b64eb9dd52834897c90285dcd
Deleted: sha256:6e61a7224fc5d369f498190ec58fd792dbeb0336addc27f1c9cc8d36ca8815e0
Deleted: sha256:535c9120eb18235d5294387e7d676b5d4dfbd24381c6d14604af6b1a981420ff
Deleted: sha256:ff2dc9781ac3e5b4b0907188d97d5016d06de0553685e8d7ece927d0c58a66ed
Deleted: sha256:0b3cd9f6b503a1841e8a920c1b1ffe8e75b10dabb1fce36a9165cecf03913ad7
Deleted: sha256:3f0a19a041897cee63dcd83ec76e71717f33d7bf1cf2f89dac22729c58d59939
Deleted: sha256:4b2618d7a74bbd2d21ca9a0fd9aab82d29f9202eda2028eea926b36cb08a8cd7
Deleted: sha256:6d5a1e5dd1d5fe20d96f8e86cbdb0c4a9391885d1038587a10589b4e836e698d
Deleted: sha256:efeef3d32446adce918495872238907a92f9d336d391df799fa486f6dbfb89f4
Deleted: sha256:1d8b7af6b48f12be16bd9ceb2a450e7c1a4695c94df9d39da7fc0528fb4e8a90
Deleted: sha256:99c2c5a97de14abd811cd345420449a44edd9df19ae559a694adf1ef0f7d44fe
Deleted: sha256:9de766a1d30e25558b8452fbd7528f8ea0bf6b1b31a680e2a02e1a74747d6f09
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:1e470c9944a147f1e413cef7ce939b281a5e4be5c987604f5da9ab4b0c9022cf
  Associated tags:
 - 20210706124332
Tags:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210706124332
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210706124332].
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:1e470c9944a147f1e413cef7ce939b281a5e4be5c987604f5da9ab4b0c9022cf].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 18m 23s
104 actionable tasks: 74 executed, 28 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/tqmlzl5re6gfy

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #18

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/18/display/redirect>

Changes:


------------------------------------------
[...truncated 63.70 KB...]
--verbosity=error -- pull gcr.io/project-id/my-image`.

See: https://cloud.google.com/container-registry/docs/support/deprecation-notices#gcloud-docker

The push refers to repository [us.gcr.io/apache-beam-testing/java-postcommit-it/java]
36f6ff980496: Preparing
c2bad424afa6: Preparing
dcbf68797da7: Preparing
cd759ab680e0: Preparing
1d51ea948747: Preparing
c46b5475b8cb: Preparing
5600c0afec40: Preparing
209cbea4f6dc: Preparing
a5895c78a516: Preparing
aecae6c79dd7: Preparing
1dcbcd76a8fd: Preparing
16b7f86f2b9a: Preparing
6ea995e9b7d3: Preparing
c0848348e2f7: Preparing
79c550eb7bd2: Preparing
7095af798ace: Preparing
fe6a4fdbedc0: Preparing
e4d0e810d54a: Preparing
4e006334a6fd: Preparing
a5895c78a516: Waiting
aecae6c79dd7: Waiting
79c550eb7bd2: Waiting
fe6a4fdbedc0: Waiting
c46b5475b8cb: Waiting
7095af798ace: Waiting
e4d0e810d54a: Waiting
4e006334a6fd: Waiting
5600c0afec40: Waiting
1dcbcd76a8fd: Waiting
16b7f86f2b9a: Waiting
6ea995e9b7d3: Waiting
c0848348e2f7: Waiting
1d51ea948747: Pushed
c2bad424afa6: Pushed
dcbf68797da7: Pushed
cd759ab680e0: Pushed
36f6ff980496: Pushed
c46b5475b8cb: Pushed
209cbea4f6dc: Pushed
a5895c78a516: Pushed
6ea995e9b7d3: Layer already exists
1dcbcd76a8fd: Pushed
c0848348e2f7: Layer already exists
79c550eb7bd2: Layer already exists
7095af798ace: Layer already exists
fe6a4fdbedc0: Layer already exists
e4d0e810d54a: Layer already exists
4e006334a6fd: Layer already exists
16b7f86f2b9a: Pushed
5600c0afec40: Pushed
aecae6c79dd7: Pushed
20210705124332: digest: sha256:f0661026516f3e462b7e830819aa1029477c4098dcdc88cc6e874958ca3c40e2 size: 4310

> Task :sdks:java:testing:load-tests:run
Jul 05, 2021 12:48:14 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Jul 05, 2021 12:48:14 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 190 files. Enable logging at DEBUG level to see which files will be staged.
Jul 05, 2021 12:48:15 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Jul 05, 2021 12:48:15 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Jul 05, 2021 12:48:18 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Jul 05, 2021 12:48:18 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <112311 bytes, hash 089ab64177f257db573807ab9b3d61d18e915b87a53b02eb94701ebd2d88772b> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-CJq2QXfyV9tXOAermz1h0Y6RW4elOwLrlHAevS2Idys.pb
Jul 05, 2021 12:48:19 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 190 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Jul 05, 2021 12:48:20 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 190 files cached, 0 files newly uploaded in 0 seconds
Jul 05, 2021 12:48:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Jul 05, 2021 12:48:20 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@21505815, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7e53339, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58fef7f7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@31ddb930, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12eedfee, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c3c4a71, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1352434e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4f9a6c2d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2b6fcb9f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@75de6341, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@74170687, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@68f0f72c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3d96fa9e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3b545206, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77bb48d5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@181d8899, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12d5c30e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@b887730, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26586b74, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@52f57666]
Jul 05, 2021 12:48:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Jul 05, 2021 12:48:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Jul 05, 2021 12:48:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Jul 05, 2021 12:48:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Jul 05, 2021 12:48:20 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@9b9a327, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@75a0c890, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@358ab600, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@e26af6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@44065156, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3791f50e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@574b7f4a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@44ed0a8f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@32177fa5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@a96d56c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6ab4a5b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2abe9173, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@235d29d6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1fdca564, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43f9dd56, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1d12e953, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@57cb70be, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2d4608a6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@20d87335, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2a8a4e0c]
Jul 05, 2021 12:48:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Jul 05, 2021 12:48:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Jul 05, 2021 12:48:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Jul 05, 2021 12:48:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Jul 05, 2021 12:48:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Jul 05, 2021 12:48:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Jul 05, 2021 12:48:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Jul 05, 2021 12:48:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Jul 05, 2021 12:48:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Jul 05, 2021 12:48:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Jul 05, 2021 12:48:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Jul 05, 2021 12:48:20 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.32.0-SNAPSHOT
Jul 05, 2021 12:48:21 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-07-05_05_48_20-18197408750596366103?project=apache-beam-testing
Jul 05, 2021 12:48:21 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-07-05_05_48_20-18197408750596366103
Jul 05, 2021 12:48:21 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-07-05_05_48_20-18197408750596366103
Jul 05, 2021 12:48:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-07-05T12:48:26.043Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-07-802i. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Jul 05, 2021 12:48:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-05T12:48:30.529Z: Worker configuration: n1-standard-2 in us-central1-f.
Jul 05, 2021 12:48:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-05T12:48:31.111Z: Expanding SplittableParDo operations into optimizable parts.
Jul 05, 2021 12:48:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-05T12:48:31.162Z: Expanding CollectionToSingleton operations into optimizable parts.
Jul 05, 2021 12:48:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-05T12:48:31.233Z: Expanding CoGroupByKey operations into optimizable parts.
Jul 05, 2021 12:48:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-05T12:48:31.308Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Jul 05, 2021 12:48:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-05T12:48:31.348Z: Expanding GroupByKey operations into streaming Read/Write steps
Jul 05, 2021 12:48:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-05T12:48:31.420Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jul 05, 2021 12:48:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-05T12:48:31.528Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jul 05, 2021 12:48:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-05T12:48:31.556Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Jul 05, 2021 12:48:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-05T12:48:31.590Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Jul 05, 2021 12:48:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-05T12:48:31.613Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Jul 05, 2021 12:48:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-05T12:48:31.646Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Jul 05, 2021 12:48:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-05T12:48:31.682Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jul 05, 2021 12:48:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-05T12:48:31.723Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jul 05, 2021 12:48:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-05T12:48:31.757Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jul 05, 2021 12:48:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-05T12:48:31.798Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Jul 05, 2021 12:48:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-05T12:48:31.830Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Jul 05, 2021 12:48:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-05T12:48:31.861Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Jul 05, 2021 12:48:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-05T12:48:31.892Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Jul 05, 2021 12:48:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-05T12:48:31.931Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jul 05, 2021 12:48:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-05T12:48:31.967Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jul 05, 2021 12:48:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-05T12:48:32.009Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jul 05, 2021 12:48:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-05T12:48:32.052Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Jul 05, 2021 12:48:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-05T12:48:32.092Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Jul 05, 2021 12:48:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-05T12:48:32.128Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Jul 05, 2021 12:48:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-05T12:48:32.164Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Jul 05, 2021 12:48:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-05T12:48:32.197Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Jul 05, 2021 12:48:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-05T12:48:32.240Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Jul 05, 2021 12:48:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-05T12:48:32.281Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Jul 05, 2021 12:48:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-05T12:48:32.313Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Jul 05, 2021 12:48:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-05T12:48:32.684Z: Starting 5 ****s in us-central1-f...
Jul 05, 2021 12:48:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-05T12:48:49.369Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jul 05, 2021 12:49:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-05T12:49:57.451Z: Workers have started successfully.
Jul 05, 2021 12:49:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-05T12:49:57.477Z: Workers have started successfully.
Jul 05, 2021 4:00:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-05T16:00:35.044Z: Cancel request is committed for workflow job: 2021-07-05_05_48_20-18197408750596366103.
Jul 05, 2021 4:00:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-05T16:00:36.996Z: Cleaning up.
Jul 05, 2021 4:00:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-05T16:00:37.184Z: Stopping **** pool...
Jul 05, 2021 4:00:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-05T16:00:37.227Z: Stopping **** pool...
Jul 05, 2021 4:02:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-05T16:02:05.358Z: Worker pool stopped.
Jul 05, 2021 4:02:11 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-07-05_05_48_20-18197408750596366103 finished with status CANCELLED.
Load test results for test (ID): 4663d851-51b1-419a-ba70-96064dab4181 and timestamp: 2021-07-05T12:48:14.769000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11390.255
dataflow_v2_java11_total_bytes_count               6.9726127E9
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210705124332
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:f0661026516f3e462b7e830819aa1029477c4098dcdc88cc6e874958ca3c40e2
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:f0661026516f3e462b7e830819aa1029477c4098dcdc88cc6e874958ca3c40e2
  Associated tags:
 - 20210705124332
Tags:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210705124332
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210705124332].
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:f0661026516f3e462b7e830819aa1029477c4098dcdc88cc6e874958ca3c40e2].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 18m 54s
104 actionable tasks: 74 executed, 28 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/6qunl2uzzxkbm

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #17

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/17/display/redirect>

Changes:


------------------------------------------
[...truncated 65.03 KB...]
5ec31583d77f: Waiting
fe6a4fdbedc0: Waiting
6ea995e9b7d3: Waiting
f97c1bb1a062: Waiting
e4d0e810d54a: Waiting
c0848348e2f7: Waiting
79c550eb7bd2: Waiting
1a887ee30d03: Waiting
7095af798ace: Waiting
0a47ac5f0220: Pushed
3ba466965a16: Pushed
f5fe05415f5f: Pushed
d831dbe609c9: Pushed
5e6ab3d9caa0: Pushed
485ace73cc86: Pushed
5ec31583d77f: Pushed
2ee9c6bf9fed: Pushed
6ea995e9b7d3: Layer already exists
c0848348e2f7: Layer already exists
f97c1bb1a062: Pushed
79c550eb7bd2: Layer already exists
7095af798ace: Layer already exists
fe6a4fdbedc0: Layer already exists
e4d0e810d54a: Layer already exists
4e006334a6fd: Layer already exists
69352ff67439: Pushed
1692c8ca89fd: Pushed
1a887ee30d03: Pushed
20210704124331: digest: sha256:0abc71e521c7f21fdb9c1268f51a6be0934b2e93184e5650e95ec591ba3987c4 size: 4310

> Task :sdks:java:testing:load-tests:run
Jul 04, 2021 12:48:22 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Jul 04, 2021 12:48:22 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 190 files. Enable logging at DEBUG level to see which files will be staged.
Jul 04, 2021 12:48:23 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Jul 04, 2021 12:48:23 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Jul 04, 2021 12:48:25 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Jul 04, 2021 12:48:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <112311 bytes, hash 20804bae054a87e89d616b323a66049fa8ee5f42613faf8d4db39bb0fa182f96> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-IIBLrgVKh-idYWsyOmYEn6juX0JhP6-NTbObsPoYL5Y.pb
Jul 04, 2021 12:48:26 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 190 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Jul 04, 2021 12:48:27 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 190 files cached, 0 files newly uploaded in 0 seconds
Jul 04, 2021 12:48:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Jul 04, 2021 12:48:27 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@21505815, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7e53339, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58fef7f7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@31ddb930, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12eedfee, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c3c4a71, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1352434e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4f9a6c2d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2b6fcb9f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@75de6341, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@74170687, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@68f0f72c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3d96fa9e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3b545206, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77bb48d5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@181d8899, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12d5c30e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@b887730, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26586b74, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@52f57666]
Jul 04, 2021 12:48:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Jul 04, 2021 12:48:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Jul 04, 2021 12:48:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Jul 04, 2021 12:48:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Jul 04, 2021 12:48:27 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@9b9a327, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@75a0c890, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@358ab600, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@e26af6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@44065156, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3791f50e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@574b7f4a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@44ed0a8f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@32177fa5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@a96d56c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6ab4a5b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2abe9173, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@235d29d6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1fdca564, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43f9dd56, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1d12e953, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@57cb70be, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2d4608a6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@20d87335, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2a8a4e0c]
Jul 04, 2021 12:48:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Jul 04, 2021 12:48:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Jul 04, 2021 12:48:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Jul 04, 2021 12:48:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Jul 04, 2021 12:48:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Jul 04, 2021 12:48:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Jul 04, 2021 12:48:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Jul 04, 2021 12:48:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Jul 04, 2021 12:48:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Jul 04, 2021 12:48:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Jul 04, 2021 12:48:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Jul 04, 2021 12:48:27 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.32.0-SNAPSHOT
Jul 04, 2021 12:48:28 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-07-04_05_48_27-6033680735721203850?project=apache-beam-testing
Jul 04, 2021 12:48:28 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-07-04_05_48_27-6033680735721203850
Jul 04, 2021 12:48:28 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-07-04_05_48_27-6033680735721203850
Jul 04, 2021 12:48:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-07-04T12:48:33.363Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-07-rpmi. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Jul 04, 2021 12:48:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-04T12:48:36.471Z: Worker configuration: n1-standard-2 in us-central1-f.
Jul 04, 2021 12:48:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-04T12:48:37.097Z: Expanding SplittableParDo operations into optimizable parts.
Jul 04, 2021 12:48:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-04T12:48:37.120Z: Expanding CollectionToSingleton operations into optimizable parts.
Jul 04, 2021 12:48:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-04T12:48:37.191Z: Expanding CoGroupByKey operations into optimizable parts.
Jul 04, 2021 12:48:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-04T12:48:37.270Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Jul 04, 2021 12:48:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-04T12:48:37.298Z: Expanding GroupByKey operations into streaming Read/Write steps
Jul 04, 2021 12:48:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-04T12:48:37.366Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jul 04, 2021 12:48:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-04T12:48:37.457Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jul 04, 2021 12:48:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-04T12:48:37.496Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Jul 04, 2021 12:48:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-04T12:48:37.530Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Jul 04, 2021 12:48:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-04T12:48:37.554Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Jul 04, 2021 12:48:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-04T12:48:37.576Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Jul 04, 2021 12:48:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-04T12:48:37.600Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jul 04, 2021 12:48:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-04T12:48:37.622Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jul 04, 2021 12:48:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-04T12:48:37.647Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jul 04, 2021 12:48:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-04T12:48:37.671Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Jul 04, 2021 12:48:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-04T12:48:37.695Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Jul 04, 2021 12:48:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-04T12:48:37.728Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Jul 04, 2021 12:48:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-04T12:48:37.763Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Jul 04, 2021 12:48:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-04T12:48:37.816Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jul 04, 2021 12:48:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-04T12:48:37.862Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jul 04, 2021 12:48:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-04T12:48:37.896Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jul 04, 2021 12:48:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-04T12:48:37.922Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Jul 04, 2021 12:48:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-04T12:48:37.957Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Jul 04, 2021 12:48:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-04T12:48:37.988Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Jul 04, 2021 12:48:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-04T12:48:38.022Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Jul 04, 2021 12:48:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-04T12:48:38.047Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Jul 04, 2021 12:48:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-04T12:48:38.069Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Jul 04, 2021 12:48:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-04T12:48:38.094Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Jul 04, 2021 12:48:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-04T12:48:38.128Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Jul 04, 2021 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-04T12:48:38.419Z: Starting 5 ****s in us-central1-f...
Jul 04, 2021 12:49:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-04T12:48:59.267Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jul 04, 2021 12:49:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-04T12:49:57.017Z: Workers have started successfully.
Jul 04, 2021 12:49:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-04T12:49:57.045Z: Workers have started successfully.
Jul 04, 2021 4:00:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-04T16:00:33.293Z: Cancel request is committed for workflow job: 2021-07-04_05_48_27-6033680735721203850.
Jul 04, 2021 4:00:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-04T16:00:33.471Z: Cleaning up.
Jul 04, 2021 4:00:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-04T16:00:33.547Z: Stopping **** pool...
Jul 04, 2021 4:00:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-04T16:00:33.596Z: Stopping **** pool...
Jul 04, 2021 4:01:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-04T16:01:23.705Z: Worker pool stopped.
Jul 04, 2021 4:01:28 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-07-04_05_48_27-6033680735721203850 finished with status CANCELLED.
Load test results for test (ID): 854cd007-5b42-479c-97ac-641b6a3e87bd and timestamp: 2021-07-04T12:48:22.739000000Z:
Exception in thread "main"                  Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11392.567
dataflow_v2_java11_total_bytes_count                5.985853E9
java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210704124331
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:0abc71e521c7f21fdb9c1268f51a6be0934b2e93184e5650e95ec591ba3987c4
Deleted: sha256:1db872fbba09601eeaa1da86f5a01e61514ecd27b7ed19da0362636170421667
Deleted: sha256:f3a0b94197fa464a6ad5c9d7b0339ba354c96e8e9c971cea163ae511923d62ad
Deleted: sha256:80f505d594e62ef9e18e35d1dcfb3d1840e2c926f36262156194951c5017d8e7
Deleted: sha256:077ed68bda629fb66c366e165cd8c263f28fe39edcbb140668126c14da538550
Deleted: sha256:80a4d9d042a4c9dc70a0a5e7cd35a058efc96707e1f7908abf56f4c65b24954c
Deleted: sha256:b22e07bdc83ce5962e4b8246bdd445a9c3bf618630284706f30ef3bdca7df4a7
Deleted: sha256:3e360f561fd37b295ea73df48697a7431e7103ec2eff35fb11e343b0f5da7273
Deleted: sha256:aafbd150bf69f3889da7df860c3349087122b9bef1c45303ab802691df6ec860
Deleted: sha256:ae7c274068c20355ab2cc2f95502b63ae965444f3892fd4322b67d42d40745fb
Deleted: sha256:06094890d874521942b3893de1ff00d094f825cca94e6ea8a3685b8a89ca74f8
Deleted: sha256:af6fa0f58ebad166b7bfb7ac767544e4822d212a86d1a36eca40f702467d2713
Deleted: sha256:be86378226c8988855ccdc520b28c51b121ce213abedefd1674b206aa8816e43
Deleted: sha256:eaa2ec33162dde0c7cd9ee06dd835d67ad5f69c8b0077df7c9937d9aeca9fab7
Deleted: sha256:e683af61c5442ab1b268a57ae18a98e78e511ff2f4686f1a47f1b8150c15f011
Deleted: sha256:a85848eb871e4c7bc912ae9fdefafce91c19a3b0270ed723a4e9ca85fad79917
Deleted: sha256:338ff3672653ba5701dfa7095550dc766a77a3e210523c40d0b8a6f6b7637b1d
Deleted: sha256:ab5f5aaa468cb7af6a966cdd093fa9ba2df8fa1dd57c6558f3b519e7ffd09aee
Deleted: sha256:f8848b431198db36b69b182de68400904825bb6ec44b6f9fd2a94e8457448e17
Deleted: sha256:0f761690c0d0406d07580172881af89e1b5e9cf8f0caa0ea38a78464813ed628
Deleted: sha256:7102164676c1177ed78fa895ab9f63c23cddb471514c1d118ebb3b7810718e2d
Deleted: sha256:7a0ed91aec743ab98f8fe5245819f07a22cd09ed6913881fc683949ecece9a3b
Deleted: sha256:23a0531b5e4bbed15ded9220ce65ea17f14dc7f462cfa4191564d3f37e9b5a33
Deleted: sha256:3ffbd07d1feae32ead750edf0f76dc516274c4798d7585aa7f9c34433ea04b4e
Deleted: sha256:54000846f540e19fda4453ee0c4c4f542b80239baa9babe337d3e26514dffb4a
Deleted: sha256:cd2b878b3d9a5975a2ed802b5ada2427e176ced2d517f3a938cc01960c329ecb
Deleted: sha256:f4b68e98bd96fa337674336708af36eb62f02f241db976b839156788a84f0078
Deleted: sha256:57fcddf217b911688a4087e68364657e8e6a63652edb4448d7b5019f3d46b87e
Deleted: sha256:aad49775139abdef045c2b3f88434727a08b4530f20206ec388b62833576971c
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:0abc71e521c7f21fdb9c1268f51a6be0934b2e93184e5650e95ec591ba3987c4
  Associated tags:
 - 20210704124331
Tags:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210704124331
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210704124331].
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:0abc71e521c7f21fdb9c1268f51a6be0934b2e93184e5650e95ec591ba3987c4].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 18m 13s
104 actionable tasks: 74 executed, 28 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/rhp6iygmq3njo

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #16

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/16/display/redirect?page=changes>

Changes:

[alexkoay88] Fix generator send_type / return_type warning.

[ajamato] [BEAM-11994] Add HarnessMonitoringInfosRequest/Response

[noreply] [BEAM-12513] Update initial sections of BPG for Go (#15057)

[noreply] Merge pull request #14980 from [BEAM-12468] added bigquery specific


------------------------------------------
[...truncated 65.13 KB...]

See: https://cloud.google.com/container-registry/docs/support/deprecation-notices#gcloud-docker

The push refers to repository [us.gcr.io/apache-beam-testing/java-postcommit-it/java]
a3f60b0b79af: Preparing
2b6e8a9bd9d1: Preparing
082ae32da61b: Preparing
7cb0d76ea0f7: Preparing
68f3a65c4057: Preparing
0996fb42e692: Preparing
b72180cc284b: Preparing
f78650ba44f3: Preparing
66653688080c: Preparing
d133c90c06e8: Preparing
3ccd0ddd94a1: Preparing
2c6886f14c5e: Preparing
6ea995e9b7d3: Preparing
c0848348e2f7: Preparing
79c550eb7bd2: Preparing
7095af798ace: Preparing
fe6a4fdbedc0: Preparing
e4d0e810d54a: Preparing
4e006334a6fd: Preparing
0996fb42e692: Waiting
b72180cc284b: Waiting
f78650ba44f3: Waiting
66653688080c: Waiting
79c550eb7bd2: Waiting
6ea995e9b7d3: Waiting
7095af798ace: Waiting
c0848348e2f7: Waiting
fe6a4fdbedc0: Waiting
d133c90c06e8: Waiting
e4d0e810d54a: Waiting
2c6886f14c5e: Waiting
3ccd0ddd94a1: Waiting
4e006334a6fd: Waiting
2b6e8a9bd9d1: Pushed
082ae32da61b: Pushed
68f3a65c4057: Pushed
0996fb42e692: Pushed
a3f60b0b79af: Pushed
f78650ba44f3: Pushed
7cb0d76ea0f7: Pushed
66653688080c: Pushed
6ea995e9b7d3: Layer already exists
3ccd0ddd94a1: Pushed
c0848348e2f7: Layer already exists
79c550eb7bd2: Layer already exists
7095af798ace: Layer already exists
fe6a4fdbedc0: Layer already exists
e4d0e810d54a: Layer already exists
4e006334a6fd: Layer already exists
2c6886f14c5e: Pushed
b72180cc284b: Pushed
d133c90c06e8: Pushed
20210703124332: digest: sha256:650fe1a677b2f156c29c409666b91f596a0b9eb08811c1d77efe23fdac938800 size: 4310

> Task :sdks:java:testing:load-tests:run
Jul 03, 2021 12:48:20 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Jul 03, 2021 12:48:21 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 190 files. Enable logging at DEBUG level to see which files will be staged.
Jul 03, 2021 12:48:22 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Jul 03, 2021 12:48:22 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Jul 03, 2021 12:48:25 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Jul 03, 2021 12:48:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <112311 bytes, hash 0b2a6d974f283c058549fb21bad9f318d8056fcbd9c0c26d48a345d08f927077> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-Cyptl08oPAWFSfshutnzGNgFb8vZwMJtSKNF0I-ScHc.pb
Jul 03, 2021 12:48:27 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 190 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Jul 03, 2021 12:48:27 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 190 files cached, 0 files newly uploaded in 0 seconds
Jul 03, 2021 12:48:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Jul 03, 2021 12:48:27 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@51d387d3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2370ac7a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@534180a2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6cf0a747, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@21505815, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7e53339, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58fef7f7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@31ddb930, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12eedfee, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c3c4a71, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1352434e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4f9a6c2d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2b6fcb9f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@75de6341, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@74170687, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@68f0f72c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3d96fa9e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3b545206, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77bb48d5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@181d8899]
Jul 03, 2021 12:48:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Jul 03, 2021 12:48:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Jul 03, 2021 12:48:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Jul 03, 2021 12:48:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Jul 03, 2021 12:48:27 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1f03fba0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@17dd671f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3bbf6abe, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@9679750, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@9b9a327, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@75a0c890, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@358ab600, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@e26af6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@44065156, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3791f50e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@574b7f4a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@44ed0a8f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@32177fa5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@a96d56c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6ab4a5b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2abe9173, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@235d29d6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1fdca564, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43f9dd56, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1d12e953]
Jul 03, 2021 12:48:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Jul 03, 2021 12:48:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Jul 03, 2021 12:48:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Jul 03, 2021 12:48:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Jul 03, 2021 12:48:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Jul 03, 2021 12:48:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Jul 03, 2021 12:48:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Jul 03, 2021 12:48:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Jul 03, 2021 12:48:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Jul 03, 2021 12:48:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Jul 03, 2021 12:48:28 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Jul 03, 2021 12:48:28 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.32.0-SNAPSHOT
Jul 03, 2021 12:48:29 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-07-03_05_48_28-14830668211932846817?project=apache-beam-testing
Jul 03, 2021 12:48:29 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-07-03_05_48_28-14830668211932846817
Jul 03, 2021 12:48:29 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-07-03_05_48_28-14830668211932846817
Jul 03, 2021 12:48:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-07-03T12:48:33.928Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-07-ng4o. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Jul 03, 2021 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-03T12:48:38.814Z: Worker configuration: n1-standard-2 in us-central1-f.
Jul 03, 2021 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-03T12:48:39.367Z: Expanding SplittableParDo operations into optimizable parts.
Jul 03, 2021 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-03T12:48:39.399Z: Expanding CollectionToSingleton operations into optimizable parts.
Jul 03, 2021 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-03T12:48:39.465Z: Expanding CoGroupByKey operations into optimizable parts.
Jul 03, 2021 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-03T12:48:39.539Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Jul 03, 2021 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-03T12:48:39.576Z: Expanding GroupByKey operations into streaming Read/Write steps
Jul 03, 2021 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-03T12:48:39.640Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jul 03, 2021 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-03T12:48:39.745Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jul 03, 2021 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-03T12:48:39.778Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Jul 03, 2021 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-03T12:48:39.812Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Jul 03, 2021 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-03T12:48:39.843Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Jul 03, 2021 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-03T12:48:39.869Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Jul 03, 2021 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-03T12:48:39.904Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jul 03, 2021 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-03T12:48:39.938Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jul 03, 2021 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-03T12:48:39.974Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jul 03, 2021 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-03T12:48:40.004Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Jul 03, 2021 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-03T12:48:40.047Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Jul 03, 2021 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-03T12:48:40.085Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Jul 03, 2021 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-03T12:48:40.122Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Jul 03, 2021 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-03T12:48:40.157Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jul 03, 2021 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-03T12:48:40.187Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jul 03, 2021 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-03T12:48:40.220Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jul 03, 2021 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-03T12:48:40.247Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Jul 03, 2021 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-03T12:48:40.274Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Jul 03, 2021 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-03T12:48:40.311Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Jul 03, 2021 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-03T12:48:40.340Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Jul 03, 2021 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-03T12:48:40.365Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Jul 03, 2021 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-03T12:48:40.401Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Jul 03, 2021 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-03T12:48:40.455Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Jul 03, 2021 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-03T12:48:40.499Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Jul 03, 2021 12:48:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-03T12:48:40.892Z: Starting 5 ****s in us-central1-f...
Jul 03, 2021 12:49:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-03T12:49:09.352Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jul 03, 2021 12:50:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-03T12:50:01.133Z: Workers have started successfully.
Jul 03, 2021 12:50:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-03T12:50:01.157Z: Workers have started successfully.
Jul 03, 2021 4:00:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-03T16:00:32.780Z: Cancel request is committed for workflow job: 2021-07-03_05_48_28-14830668211932846817.
Jul 03, 2021 4:00:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-03T16:00:32.891Z: Cleaning up.
Jul 03, 2021 4:00:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-03T16:00:32.999Z: Stopping **** pool...
Jul 03, 2021 4:00:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-03T16:00:33.144Z: Stopping **** pool...
Jul 03, 2021 4:01:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-03T16:01:35.471Z: Worker pool stopped.
Jul 03, 2021 4:01:40 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-07-03_05_48_28-14830668211932846817 finished with status CANCELLED.
Load test results for test (ID): 082c0e2f-58a7-4937-829f-cfa9a530e2ca and timestamp: 2021-07-03T12:48:21.763000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11391.986
dataflow_v2_java11_total_bytes_count               6.1173878E9
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210703124332
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:650fe1a677b2f156c29c409666b91f596a0b9eb08811c1d77efe23fdac938800
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:650fe1a677b2f156c29c409666b91f596a0b9eb08811c1d77efe23fdac938800
  Associated tags:
 - 20210703124332
Tags:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210703124332
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210703124332].
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:650fe1a677b2f156c29c409666b91f596a0b9eb08811c1d77efe23fdac938800].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 18m 24s
104 actionable tasks: 77 executed, 25 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/55th44xefsoay

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #15

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/15/display/redirect?page=changes>

Changes:

[ajamato] [BEAM-11994] Update ShortIdMap's maps to only use valid keyable fields

[noreply] Update _index.md

[editgym] Add support for BigQuery data types aliases to BigQueryAvroUtils

[noreply] Add MatchContinuously PTransform to Python SDK (#15106)

[ajthomas] Expose ReadAllFromTFRecord as public in tfrecordio

[noreply] [BEAM-10955] Fix Flink Java Runner test flake: Could not find Flink job

[noreply] [BEAM-12548] Add examples of Equals and EqualsList to equals_test.go

[Luke Cwik] [BEAM-12422] Remove all optional logging libraries as Netty attempts to

[noreply] [BEAM-12076] Adds a Kafka external read transform for reading with

[noreply] [BEAM-10937] Tour of Beam Windowing notebook (#14962)

[daria.malkova] [BEAM-12456] Add reading jdbc by partitions


------------------------------------------
[...truncated 65.50 KB...]

See: https://cloud.google.com/container-registry/docs/support/deprecation-notices#gcloud-docker

The push refers to repository [us.gcr.io/apache-beam-testing/java-postcommit-it/java]
c20d1aede57a: Preparing
626764278133: Preparing
e1fcb356b5d8: Preparing
50711b4c4664: Preparing
1db608638372: Preparing
dda087cdf696: Preparing
7cb94f9f9e28: Preparing
45d940473be4: Preparing
49b91a56dd83: Preparing
520499a87fb8: Preparing
c8f3d2b2b9c7: Preparing
31e5d08811ff: Preparing
6ea995e9b7d3: Preparing
c0848348e2f7: Preparing
79c550eb7bd2: Preparing
7095af798ace: Preparing
fe6a4fdbedc0: Preparing
e4d0e810d54a: Preparing
4e006334a6fd: Preparing
49b91a56dd83: Waiting
dda087cdf696: Waiting
7cb94f9f9e28: Waiting
520499a87fb8: Waiting
79c550eb7bd2: Waiting
45d940473be4: Waiting
e4d0e810d54a: Waiting
c8f3d2b2b9c7: Waiting
4e006334a6fd: Waiting
31e5d08811ff: Waiting
6ea995e9b7d3: Waiting
7095af798ace: Waiting
c0848348e2f7: Waiting
fe6a4fdbedc0: Waiting
e1fcb356b5d8: Pushed
626764278133: Pushed
dda087cdf696: Pushed
c20d1aede57a: Pushed
50711b4c4664: Pushed
45d940473be4: Pushed
49b91a56dd83: Pushed
7cb94f9f9e28: Pushed
6ea995e9b7d3: Layer already exists
c0848348e2f7: Layer already exists
79c550eb7bd2: Layer already exists
7095af798ace: Layer already exists
fe6a4fdbedc0: Layer already exists
e4d0e810d54a: Layer already exists
4e006334a6fd: Layer already exists
c8f3d2b2b9c7: Pushed
31e5d08811ff: Pushed
520499a87fb8: Pushed
1db608638372: Pushed
20210702124335: digest: sha256:971e7bb2f378fd4cb7f85439e2834dfdf28853828fcdf3de6c75416895f2bf18 size: 4310

> Task :sdks:java:testing:load-tests:run
Jul 02, 2021 12:48:48 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Jul 02, 2021 12:48:49 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 190 files. Enable logging at DEBUG level to see which files will be staged.
Jul 02, 2021 12:48:49 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Jul 02, 2021 12:48:49 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Jul 02, 2021 12:48:52 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Jul 02, 2021 12:48:53 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <112268 bytes, hash 5a411e13dd7dce338e4811bfb549d28a2904c469c4d6134bf236ae5553d11b10> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-WkEeE919zjOOSBG_tUnSiikExGnE1hNL8jauVVPRGxA.pb
Jul 02, 2021 12:48:54 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 190 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Jul 02, 2021 12:48:55 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 190 files cached, 0 files newly uploaded in 0 seconds
Jul 02, 2021 12:48:55 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Jul 02, 2021 12:48:55 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@21505815, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7e53339, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58fef7f7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@31ddb930, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12eedfee, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c3c4a71, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1352434e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4f9a6c2d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2b6fcb9f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@75de6341, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@74170687, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@68f0f72c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3d96fa9e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3b545206, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77bb48d5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@181d8899, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12d5c30e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@b887730, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26586b74, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@52f57666]
Jul 02, 2021 12:48:55 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Jul 02, 2021 12:48:55 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Jul 02, 2021 12:48:55 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Jul 02, 2021 12:48:55 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Jul 02, 2021 12:48:55 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@9679750, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@9b9a327, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@75a0c890, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@358ab600, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@e26af6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@44065156, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3791f50e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@574b7f4a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@44ed0a8f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@32177fa5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@a96d56c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6ab4a5b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2abe9173, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@235d29d6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1fdca564, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43f9dd56, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1d12e953, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@57cb70be, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2d4608a6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@20d87335]
Jul 02, 2021 12:48:55 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Jul 02, 2021 12:48:55 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Jul 02, 2021 12:48:55 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Jul 02, 2021 12:48:55 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Jul 02, 2021 12:48:55 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Jul 02, 2021 12:48:55 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Jul 02, 2021 12:48:55 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Jul 02, 2021 12:48:55 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Jul 02, 2021 12:48:55 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Jul 02, 2021 12:48:55 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Jul 02, 2021 12:48:55 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Jul 02, 2021 12:48:55 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.32.0-SNAPSHOT
Jul 02, 2021 12:48:56 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-07-02_05_48_55-12237855788952407789?project=apache-beam-testing
Jul 02, 2021 12:48:56 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-07-02_05_48_55-12237855788952407789
Jul 02, 2021 12:48:56 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-07-02_05_48_55-12237855788952407789
Jul 02, 2021 12:49:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-07-02T12:49:02.836Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-07-7wso. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Jul 02, 2021 12:49:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-02T12:49:06.891Z: Worker configuration: n1-standard-2 in us-central1-f.
Jul 02, 2021 12:49:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-02T12:49:07.481Z: Expanding SplittableParDo operations into optimizable parts.
Jul 02, 2021 12:49:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-02T12:49:07.508Z: Expanding CollectionToSingleton operations into optimizable parts.
Jul 02, 2021 12:49:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-02T12:49:07.574Z: Expanding CoGroupByKey operations into optimizable parts.
Jul 02, 2021 12:49:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-02T12:49:07.647Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Jul 02, 2021 12:49:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-02T12:49:07.686Z: Expanding GroupByKey operations into streaming Read/Write steps
Jul 02, 2021 12:49:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-02T12:49:07.759Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jul 02, 2021 12:49:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-02T12:49:07.879Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jul 02, 2021 12:49:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-02T12:49:07.905Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Jul 02, 2021 12:49:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-02T12:49:07.929Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Jul 02, 2021 12:49:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-02T12:49:07.963Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Jul 02, 2021 12:49:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-02T12:49:08.000Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Jul 02, 2021 12:49:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-02T12:49:08.036Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jul 02, 2021 12:49:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-02T12:49:08.071Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jul 02, 2021 12:49:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-02T12:49:08.111Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jul 02, 2021 12:49:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-02T12:49:08.149Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Jul 02, 2021 12:49:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-02T12:49:08.180Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Jul 02, 2021 12:49:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-02T12:49:08.206Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Jul 02, 2021 12:49:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-02T12:49:08.239Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Jul 02, 2021 12:49:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-02T12:49:08.276Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jul 02, 2021 12:49:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-02T12:49:08.310Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jul 02, 2021 12:49:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-02T12:49:08.343Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jul 02, 2021 12:49:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-02T12:49:08.377Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Jul 02, 2021 12:49:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-02T12:49:08.412Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Jul 02, 2021 12:49:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-02T12:49:08.445Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Jul 02, 2021 12:49:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-02T12:49:08.480Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Jul 02, 2021 12:49:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-02T12:49:08.529Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Jul 02, 2021 12:49:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-02T12:49:08.554Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Jul 02, 2021 12:49:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-02T12:49:08.590Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Jul 02, 2021 12:49:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-02T12:49:08.620Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Jul 02, 2021 12:49:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-02T12:49:09.033Z: Starting 5 ****s in us-central1-f...
Jul 02, 2021 12:49:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-02T12:49:21.914Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jul 02, 2021 12:50:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-02T12:50:30.684Z: Workers have started successfully.
Jul 02, 2021 12:50:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-02T12:50:30.708Z: Workers have started successfully.
Jul 02, 2021 4:00:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-02T16:00:34.999Z: Cancel request is committed for workflow job: 2021-07-02_05_48_55-12237855788952407789.
Jul 02, 2021 4:00:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-02T16:00:35.132Z: Cleaning up.
Jul 02, 2021 4:00:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-02T16:00:35.217Z: Stopping **** pool...
Jul 02, 2021 4:00:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-02T16:00:35.324Z: Stopping **** pool...
Jul 02, 2021 4:01:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-02T16:01:32.041Z: Worker pool stopped.
Jul 02, 2021 4:01:38 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-07-02_05_48_55-12237855788952407789 finished with status CANCELLED.
Load test results for test (ID): 48145732-62e5-4a60-8a98-c38778366dfb and timestamp: 2021-07-02T12:48:49.383000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11364.746
dataflow_v2_java11_total_bytes_count               6.8580185E9
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210702124335
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:971e7bb2f378fd4cb7f85439e2834dfdf28853828fcdf3de6c75416895f2bf18
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:971e7bb2f378fd4cb7f85439e2834dfdf28853828fcdf3de6c75416895f2bf18
  Associated tags:
 - 20210702124335
Tags:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210702124335
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210702124335].
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:971e7bb2f378fd4cb7f85439e2834dfdf28853828fcdf3de6c75416895f2bf18].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 18m 21s
104 actionable tasks: 77 executed, 25 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/caikv3m3gyzni

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #14

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/14/display/redirect?page=changes>

Changes:

[suztomo] [BEAM-12422] Removing unnecessary log4j-api dependency

[noreply] [BEAM-11289] [Python] Integrate Google Cloud Recommendations AI

[noreply] Update README to mention modules and Go versions. (#15109)

[noreply] [BEAM-12548] Add EqualsList functionality to PAssert (#15110)


------------------------------------------
[...truncated 64.32 KB...]
`--verbosity=error` to silence this warning: `gcloud docker
--verbosity=error -- pull gcr.io/project-id/my-image`.

See: https://cloud.google.com/container-registry/docs/support/deprecation-notices#gcloud-docker

The push refers to repository [us.gcr.io/apache-beam-testing/java-postcommit-it/java]
d2df0d194f5e: Preparing
2ad5c3f0e855: Preparing
fe213d63d347: Preparing
2b90ed778eb0: Preparing
b6e293bac083: Preparing
77c152047427: Preparing
7f2bd40f2aac: Preparing
363d406c4f78: Preparing
360346242f5a: Preparing
bb5b9adc7770: Preparing
6ea296c319bd: Preparing
ee876450100b: Preparing
82f8ebb33b27: Preparing
c0848348e2f7: Preparing
79c550eb7bd2: Preparing
7095af798ace: Preparing
fe6a4fdbedc0: Preparing
e4d0e810d54a: Preparing
4e006334a6fd: Preparing
fe6a4fdbedc0: Waiting
82f8ebb33b27: Waiting
7f2bd40f2aac: Waiting
363d406c4f78: Waiting
77c152047427: Waiting
c0848348e2f7: Waiting
7095af798ace: Waiting
360346242f5a: Waiting
79c550eb7bd2: Waiting
bb5b9adc7770: Waiting
4e006334a6fd: Waiting
ee876450100b: Waiting
fe213d63d347: Pushed
b6e293bac083: Pushed
2ad5c3f0e855: Pushed
d2df0d194f5e: Pushed
77c152047427: Pushed
363d406c4f78: Pushed
2b90ed778eb0: Pushed
360346242f5a: Pushed
82f8ebb33b27: Layer already exists
c0848348e2f7: Layer already exists
79c550eb7bd2: Layer already exists
7095af798ace: Layer already exists
fe6a4fdbedc0: Layer already exists
e4d0e810d54a: Layer already exists
4e006334a6fd: Layer already exists
6ea296c319bd: Pushed
7f2bd40f2aac: Pushed
ee876450100b: Pushed
bb5b9adc7770: Pushed
20210701124332: digest: sha256:58a56bfcc86381b190fd635eaa30015ba138cced131599cf8058c76515ccc87f size: 4310

> Task :sdks:java:testing:load-tests:run
Jul 01, 2021 12:47:56 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Jul 01, 2021 12:47:56 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 190 files. Enable logging at DEBUG level to see which files will be staged.
Jul 01, 2021 12:47:57 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Jul 01, 2021 12:47:57 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Jul 01, 2021 12:47:59 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Jul 01, 2021 12:47:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <112268 bytes, hash d69b6cf0e22250d30a4fc48559be98363df9f453fab444542d921415f9d90cc7> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-1pts8OIiUNMKT8SFWb6YNj359FP6tERULZIUFfnZDMc.pb
Jul 01, 2021 12:48:00 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 190 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Jul 01, 2021 12:48:01 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 190 files cached, 0 files newly uploaded in 0 seconds
Jul 01, 2021 12:48:01 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Jul 01, 2021 12:48:01 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@534180a2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6cf0a747, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@21505815, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7e53339, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58fef7f7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@31ddb930, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12eedfee, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c3c4a71, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1352434e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4f9a6c2d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2b6fcb9f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@75de6341, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@74170687, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@68f0f72c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3d96fa9e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3b545206, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77bb48d5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@181d8899, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12d5c30e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@b887730]
Jul 01, 2021 12:48:01 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Jul 01, 2021 12:48:01 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Jul 01, 2021 12:48:01 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Jul 01, 2021 12:48:01 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Jul 01, 2021 12:48:01 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3bbf6abe, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@9679750, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@9b9a327, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@75a0c890, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@358ab600, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@e26af6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@44065156, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3791f50e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@574b7f4a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@44ed0a8f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@32177fa5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@a96d56c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6ab4a5b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2abe9173, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@235d29d6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1fdca564, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43f9dd56, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1d12e953, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@57cb70be, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2d4608a6]
Jul 01, 2021 12:48:01 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Jul 01, 2021 12:48:01 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Jul 01, 2021 12:48:01 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Jul 01, 2021 12:48:01 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Jul 01, 2021 12:48:01 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Jul 01, 2021 12:48:01 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Jul 01, 2021 12:48:01 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Jul 01, 2021 12:48:01 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Jul 01, 2021 12:48:01 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Jul 01, 2021 12:48:01 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Jul 01, 2021 12:48:01 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Jul 01, 2021 12:48:01 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.32.0-SNAPSHOT
Jul 01, 2021 12:48:03 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-07-01_05_48_01-15955217050123870291?project=apache-beam-testing
Jul 01, 2021 12:48:03 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-07-01_05_48_01-15955217050123870291
Jul 01, 2021 12:48:03 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-07-01_05_48_01-15955217050123870291
Jul 01, 2021 12:48:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-07-01T12:48:07.097Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-07-4rz. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Jul 01, 2021 12:48:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-01T12:48:12.685Z: Worker configuration: n1-standard-2 in us-central1-f.
Jul 01, 2021 12:48:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-01T12:48:13.400Z: Expanding SplittableParDo operations into optimizable parts.
Jul 01, 2021 12:48:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-01T12:48:13.438Z: Expanding CollectionToSingleton operations into optimizable parts.
Jul 01, 2021 12:48:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-01T12:48:13.520Z: Expanding CoGroupByKey operations into optimizable parts.
Jul 01, 2021 12:48:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-01T12:48:13.593Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Jul 01, 2021 12:48:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-01T12:48:13.622Z: Expanding GroupByKey operations into streaming Read/Write steps
Jul 01, 2021 12:48:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-01T12:48:13.682Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jul 01, 2021 12:48:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-01T12:48:13.779Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jul 01, 2021 12:48:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-01T12:48:13.807Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Jul 01, 2021 12:48:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-01T12:48:13.840Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Jul 01, 2021 12:48:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-01T12:48:13.867Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Jul 01, 2021 12:48:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-01T12:48:13.897Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Jul 01, 2021 12:48:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-01T12:48:13.922Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jul 01, 2021 12:48:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-01T12:48:13.967Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jul 01, 2021 12:48:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-01T12:48:13.998Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jul 01, 2021 12:48:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-01T12:48:14.033Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Jul 01, 2021 12:48:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-01T12:48:14.066Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Jul 01, 2021 12:48:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-01T12:48:14.100Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Jul 01, 2021 12:48:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-01T12:48:14.123Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Jul 01, 2021 12:48:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-01T12:48:14.157Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jul 01, 2021 12:48:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-01T12:48:14.191Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jul 01, 2021 12:48:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-01T12:48:14.219Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jul 01, 2021 12:48:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-01T12:48:14.246Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Jul 01, 2021 12:48:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-01T12:48:14.280Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Jul 01, 2021 12:48:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-01T12:48:14.313Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Jul 01, 2021 12:48:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-01T12:48:14.340Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Jul 01, 2021 12:48:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-01T12:48:14.360Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Jul 01, 2021 12:48:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-01T12:48:14.392Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Jul 01, 2021 12:48:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-01T12:48:14.414Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Jul 01, 2021 12:48:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-01T12:48:14.444Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Jul 01, 2021 12:48:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-01T12:48:14.801Z: Starting 5 ****s in us-central1-f...
Jul 01, 2021 12:48:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-01T12:48:31.991Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jul 01, 2021 12:49:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-01T12:49:35.790Z: Workers have started successfully.
Jul 01, 2021 12:49:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-01T12:49:35.824Z: Workers have started successfully.
Jul 01, 2021 4:00:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-01T16:00:33.824Z: Cancel request is committed for workflow job: 2021-07-01_05_48_01-15955217050123870291.
Jul 01, 2021 4:00:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-01T16:00:33.890Z: Cleaning up.
Jul 01, 2021 4:00:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-01T16:00:33.950Z: Stopping **** pool...
Jul 01, 2021 4:00:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-01T16:00:33.998Z: Stopping **** pool...
Jul 01, 2021 4:01:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-01T16:01:28.238Z: Worker pool stopped.
Jul 01, 2021 4:01:34 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-07-01_05_48_01-15955217050123870291 finished with status CANCELLED.
Load test results for test (ID): 6a01d046-a823-452a-995a-c4a449f0841f and timestamp: 2021-07-01T12:47:57.063000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11419.014
dataflow_v2_java11_total_bytes_count               6.4937955E9
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210701124332
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:58a56bfcc86381b190fd635eaa30015ba138cced131599cf8058c76515ccc87f
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:58a56bfcc86381b190fd635eaa30015ba138cced131599cf8058c76515ccc87f
  Associated tags:
 - 20210701124332
Tags:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210701124332
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210701124332].
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:58a56bfcc86381b190fd635eaa30015ba138cced131599cf8058c76515ccc87f].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 18m 15s
104 actionable tasks: 74 executed, 28 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/vkn6rhm2zurhw

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #13

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/13/display/redirect?page=changes>

Changes:

[Luke Cwik] [BEAM-12547] Enable epoll again.

[daniel.o.programmer] [BEAM-12383] Go Kafka integration test and framework changes.

[noreply] Update

[noreply] [BEAM-12529] Allow converting arbitrary pandas dtypes back to Python

[noreply] [BEAM-12533] Add simple `__repr__` for `DeferredDataFrame` and


------------------------------------------
[...truncated 64.83 KB...]
285b5eb6a427: Preparing
3306cec88ccf: Preparing
87af4294a359: Preparing
4c84fe9408b1: Preparing
a51b0f596827: Preparing
4db6bea0a2de: Preparing
5532cfef8218: Preparing
32718134f6f2: Preparing
0c081500f09d: Preparing
82f8ebb33b27: Preparing
c0848348e2f7: Preparing
79c550eb7bd2: Preparing
7095af798ace: Preparing
fe6a4fdbedc0: Preparing
e4d0e810d54a: Preparing
4e006334a6fd: Preparing
82f8ebb33b27: Waiting
fe6a4fdbedc0: Waiting
5532cfef8218: Waiting
c0848348e2f7: Waiting
e4d0e810d54a: Waiting
0c081500f09d: Waiting
4e006334a6fd: Waiting
32718134f6f2: Waiting
79c550eb7bd2: Waiting
a51b0f596827: Waiting
4db6bea0a2de: Waiting
87af4294a359: Waiting
4c84fe9408b1: Waiting
5e4525adba9e: Pushed
3306cec88ccf: Pushed
7bf1108dc4fe: Pushed
08f947b9eb22: Pushed
87af4294a359: Pushed
285b5eb6a427: Pushed
a51b0f596827: Pushed
4db6bea0a2de: Pushed
82f8ebb33b27: Layer already exists
c0848348e2f7: Layer already exists
79c550eb7bd2: Layer already exists
32718134f6f2: Pushed
7095af798ace: Layer already exists
fe6a4fdbedc0: Layer already exists
e4d0e810d54a: Layer already exists
4e006334a6fd: Layer already exists
4c84fe9408b1: Pushed
0c081500f09d: Pushed
5532cfef8218: Pushed
20210630124331: digest: sha256:a90d8f5a356efd3ee8e9c99944ef951d41a844743dcac2fbff49a5527b4c1161 size: 4310

> Task :sdks:java:testing:load-tests:run
Jun 30, 2021 12:48:10 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Jun 30, 2021 12:48:10 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 190 files. Enable logging at DEBUG level to see which files will be staged.
Jun 30, 2021 12:48:11 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Jun 30, 2021 12:48:11 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Jun 30, 2021 12:48:14 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Jun 30, 2021 12:48:14 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <112268 bytes, hash c4124ed1ab4c9ed5a85e4eec17f75e31d5403c683b93a6c9d0977a621466636e> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-xBJO0atMntWoXk7sF_deMdVAPGg7k6bJ0Jd6YhRmY24.pb
Jun 30, 2021 12:48:15 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 190 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Jun 30, 2021 12:48:16 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 190 files cached, 0 files newly uploaded in 0 seconds
Jun 30, 2021 12:48:16 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Jun 30, 2021 12:48:16 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@534180a2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6cf0a747, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@21505815, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7e53339, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58fef7f7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@31ddb930, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12eedfee, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c3c4a71, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1352434e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4f9a6c2d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2b6fcb9f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@75de6341, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@74170687, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@68f0f72c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3d96fa9e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3b545206, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77bb48d5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@181d8899, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12d5c30e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@b887730]
Jun 30, 2021 12:48:16 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Jun 30, 2021 12:48:16 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Jun 30, 2021 12:48:16 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Jun 30, 2021 12:48:16 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Jun 30, 2021 12:48:16 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3bbf6abe, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@9679750, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@9b9a327, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@75a0c890, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@358ab600, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@e26af6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@44065156, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3791f50e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@574b7f4a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@44ed0a8f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@32177fa5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@a96d56c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6ab4a5b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2abe9173, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@235d29d6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1fdca564, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43f9dd56, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1d12e953, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@57cb70be, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2d4608a6]
Jun 30, 2021 12:48:16 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Jun 30, 2021 12:48:16 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Jun 30, 2021 12:48:16 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Jun 30, 2021 12:48:16 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Jun 30, 2021 12:48:16 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Jun 30, 2021 12:48:16 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Jun 30, 2021 12:48:16 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Jun 30, 2021 12:48:16 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Jun 30, 2021 12:48:16 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Jun 30, 2021 12:48:16 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Jun 30, 2021 12:48:16 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Jun 30, 2021 12:48:16 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.32.0-SNAPSHOT
Jun 30, 2021 12:48:17 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-06-30_05_48_16-7585738902660620063?project=apache-beam-testing
Jun 30, 2021 12:48:17 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-06-30_05_48_16-7585738902660620063
Jun 30, 2021 12:48:17 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-06-30_05_48_16-7585738902660620063
Jun 30, 2021 12:48:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-06-30T12:48:22.407Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-06-apqw. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Jun 30, 2021 12:48:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-30T12:48:26.824Z: Worker configuration: n1-standard-2 in us-central1-f.
Jun 30, 2021 12:48:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-30T12:48:27.345Z: Expanding SplittableParDo operations into optimizable parts.
Jun 30, 2021 12:48:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-30T12:48:27.380Z: Expanding CollectionToSingleton operations into optimizable parts.
Jun 30, 2021 12:48:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-30T12:48:27.446Z: Expanding CoGroupByKey operations into optimizable parts.
Jun 30, 2021 12:48:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-30T12:48:27.505Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Jun 30, 2021 12:48:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-30T12:48:27.539Z: Expanding GroupByKey operations into streaming Read/Write steps
Jun 30, 2021 12:48:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-30T12:48:27.596Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jun 30, 2021 12:48:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-30T12:48:27.712Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jun 30, 2021 12:48:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-30T12:48:27.742Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Jun 30, 2021 12:48:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-30T12:48:27.784Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Jun 30, 2021 12:48:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-30T12:48:27.819Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Jun 30, 2021 12:48:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-30T12:48:27.854Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Jun 30, 2021 12:48:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-30T12:48:27.889Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jun 30, 2021 12:48:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-30T12:48:27.926Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jun 30, 2021 12:48:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-30T12:48:28.038Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jun 30, 2021 12:48:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-30T12:48:28.203Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Jun 30, 2021 12:48:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-30T12:48:28.280Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Jun 30, 2021 12:48:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-30T12:48:28.326Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Jun 30, 2021 12:48:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-30T12:48:28.366Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Jun 30, 2021 12:48:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-30T12:48:28.417Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jun 30, 2021 12:48:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-30T12:48:28.454Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jun 30, 2021 12:48:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-30T12:48:28.483Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jun 30, 2021 12:48:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-30T12:48:28.526Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Jun 30, 2021 12:48:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-30T12:48:28.566Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Jun 30, 2021 12:48:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-30T12:48:28.600Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Jun 30, 2021 12:48:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-30T12:48:28.624Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Jun 30, 2021 12:48:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-30T12:48:28.656Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Jun 30, 2021 12:48:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-30T12:48:28.694Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Jun 30, 2021 12:48:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-30T12:48:28.727Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Jun 30, 2021 12:48:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-30T12:48:28.760Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Jun 30, 2021 12:48:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-30T12:48:29.209Z: Starting 5 ****s in us-central1-f...
Jun 30, 2021 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-30T12:48:40.826Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jun 30, 2021 12:49:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-30T12:49:06.012Z: Autoscaling: Raised the number of ****s to 2 so that the pipeline can catch up with its backlog and keep up with its input rate.
Jun 30, 2021 12:49:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-30T12:49:06.071Z: Resized **** pool to 2, though goal was 5.  This could be a quota issue.
Jun 30, 2021 12:49:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-30T12:49:16.404Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Jun 30, 2021 12:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-30T12:49:50.068Z: Workers have started successfully.
Jun 30, 2021 12:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-30T12:49:50.098Z: Workers have started successfully.
Jun 30, 2021 4:01:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-30T16:01:35.011Z: Cancel request is committed for workflow job: 2021-06-30_05_48_16-7585738902660620063.
Jun 30, 2021 4:01:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-30T16:01:35.703Z: Cleaning up.
Jun 30, 2021 4:01:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-30T16:01:35.794Z: Stopping **** pool...
Jun 30, 2021 4:01:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-30T16:01:35.890Z: Stopping **** pool...
Jun 30, 2021 4:02:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-30T16:02:23.644Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Jun 30, 2021 4:02:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-30T16:02:23.688Z: Worker pool stopped.
Jun 30, 2021 4:02:31 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-06-30_05_48_16-7585738902660620063 finished with status CANCELLED.
Load test results for test (ID): 37430e86-d1a1-4ac1-a9c3-2e114a40bdb5 and timestamp: 2021-06-30T12:48:10.866000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11456.603
dataflow_v2_java11_total_bytes_count               5.2081896E9
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210630124331
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:a90d8f5a356efd3ee8e9c99944ef951d41a844743dcac2fbff49a5527b4c1161
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:a90d8f5a356efd3ee8e9c99944ef951d41a844743dcac2fbff49a5527b4c1161
  Associated tags:
 - 20210630124331
Tags:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210630124331
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210630124331].
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:a90d8f5a356efd3ee8e9c99944ef951d41a844743dcac2fbff49a5527b4c1161].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 19m 15s
104 actionable tasks: 76 executed, 26 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/clic2j2jqfhgm

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #12

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/12/display/redirect?page=changes>

Changes:

[Boyuan Zhang] [BEAM-12524] Use synchronized function to set instruction_id to null.

[Luke Cwik] [BEAM-12546] Swap to use ArrayBlockingQueue to improve performance of

[zhoufek] [BEAM-10955] Sickbay very flaky Flink test

[noreply] [BEAM-12541] Fix partitioning requirements for filllna with

[noreply] Revert "Update test_error_message_includes_stage test"

[noreply] Sync Beam and Dataflow image dependencies (#15026)

[noreply] More easily substituted CoGroupByKey transform. (#15029)


------------------------------------------
[...truncated 63.83 KB...]

The push refers to repository [us.gcr.io/apache-beam-testing/java-postcommit-it/java]
be1f3d89bada: Preparing
3d44be8be5ef: Preparing
28c42dfd5023: Preparing
2b03e9c02861: Preparing
fce269bb27b9: Preparing
0b71cb9bba94: Preparing
8c828d353600: Preparing
9a4cb2855738: Preparing
b3876ac2fbc5: Preparing
206c4e1b7ce6: Preparing
39c1da89cee3: Preparing
ab13562f0aaf: Preparing
82f8ebb33b27: Preparing
c0848348e2f7: Preparing
79c550eb7bd2: Preparing
7095af798ace: Preparing
fe6a4fdbedc0: Preparing
e4d0e810d54a: Preparing
4e006334a6fd: Preparing
9a4cb2855738: Waiting
8c828d353600: Waiting
7095af798ace: Waiting
0b71cb9bba94: Waiting
c0848348e2f7: Waiting
b3876ac2fbc5: Waiting
79c550eb7bd2: Waiting
fe6a4fdbedc0: Waiting
4e006334a6fd: Waiting
39c1da89cee3: Waiting
206c4e1b7ce6: Waiting
82f8ebb33b27: Waiting
fce269bb27b9: Pushed
28c42dfd5023: Pushed
3d44be8be5ef: Pushed
be1f3d89bada: Pushed
0b71cb9bba94: Pushed
2b03e9c02861: Pushed
9a4cb2855738: Pushed
b3876ac2fbc5: Pushed
39c1da89cee3: Pushed
82f8ebb33b27: Layer already exists
c0848348e2f7: Layer already exists
79c550eb7bd2: Layer already exists
ab13562f0aaf: Pushed
7095af798ace: Layer already exists
fe6a4fdbedc0: Layer already exists
e4d0e810d54a: Layer already exists
4e006334a6fd: Layer already exists
8c828d353600: Pushed
206c4e1b7ce6: Pushed
20210629124332: digest: sha256:a6a930714d7ec025741108fbacb114dd302c5686a6f2c54329508b2a53216e6a size: 4310

> Task :sdks:java:testing:load-tests:run
Jun 29, 2021 12:48:00 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Jun 29, 2021 12:48:00 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 190 files. Enable logging at DEBUG level to see which files will be staged.
Jun 29, 2021 12:48:01 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Jun 29, 2021 12:48:01 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Jun 29, 2021 12:48:03 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Jun 29, 2021 12:48:03 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <112268 bytes, hash 093c02c5c7578e9fbe48ff46318ab2caac85ee13454694c9cc03514270ddc110> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-CTwCxcdXjp--SP9GMYqyyqyF7hNFRpTJzANRQnDdwRA.pb
Jun 29, 2021 12:48:05 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 190 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Jun 29, 2021 12:48:05 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 190 files cached, 0 files newly uploaded in 0 seconds
Jun 29, 2021 12:48:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Jun 29, 2021 12:48:06 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@117d32e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@51d387d3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2370ac7a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@534180a2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6cf0a747, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@21505815, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7e53339, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58fef7f7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@31ddb930, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12eedfee, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c3c4a71, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1352434e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4f9a6c2d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2b6fcb9f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@75de6341, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@74170687, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@68f0f72c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3d96fa9e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3b545206, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77bb48d5]
Jun 29, 2021 12:48:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Jun 29, 2021 12:48:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Jun 29, 2021 12:48:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Jun 29, 2021 12:48:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Jun 29, 2021 12:48:06 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@64aad809, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1f03fba0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@17dd671f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3bbf6abe, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@9679750, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@9b9a327, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@75a0c890, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@358ab600, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@e26af6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@44065156, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3791f50e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@574b7f4a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@44ed0a8f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@32177fa5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@a96d56c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6ab4a5b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2abe9173, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@235d29d6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1fdca564, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43f9dd56]
Jun 29, 2021 12:48:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Jun 29, 2021 12:48:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Jun 29, 2021 12:48:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Jun 29, 2021 12:48:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Jun 29, 2021 12:48:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Jun 29, 2021 12:48:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Jun 29, 2021 12:48:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Jun 29, 2021 12:48:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Jun 29, 2021 12:48:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Jun 29, 2021 12:48:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Jun 29, 2021 12:48:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Jun 29, 2021 12:48:06 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.32.0-SNAPSHOT
Jun 29, 2021 12:48:07 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-06-29_05_48_06-5962777165539570269?project=apache-beam-testing
Jun 29, 2021 12:48:07 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-06-29_05_48_06-5962777165539570269
Jun 29, 2021 12:48:07 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-06-29_05_48_06-5962777165539570269
Jun 29, 2021 12:48:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-06-29T12:48:10.573Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-06-ytp2. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Jun 29, 2021 12:48:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-29T12:48:14.452Z: Worker configuration: n1-standard-2 in us-central1-f.
Jun 29, 2021 12:48:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-29T12:48:14.939Z: Expanding SplittableParDo operations into optimizable parts.
Jun 29, 2021 12:48:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-29T12:48:14.973Z: Expanding CollectionToSingleton operations into optimizable parts.
Jun 29, 2021 12:48:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-29T12:48:15.040Z: Expanding CoGroupByKey operations into optimizable parts.
Jun 29, 2021 12:48:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-29T12:48:15.128Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Jun 29, 2021 12:48:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-29T12:48:15.164Z: Expanding GroupByKey operations into streaming Read/Write steps
Jun 29, 2021 12:48:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-29T12:48:15.245Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jun 29, 2021 12:48:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-29T12:48:15.356Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jun 29, 2021 12:48:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-29T12:48:15.392Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Jun 29, 2021 12:48:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-29T12:48:15.433Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Jun 29, 2021 12:48:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-29T12:48:15.462Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Jun 29, 2021 12:48:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-29T12:48:15.500Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Jun 29, 2021 12:48:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-29T12:48:15.533Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jun 29, 2021 12:48:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-29T12:48:15.568Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jun 29, 2021 12:48:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-29T12:48:15.610Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jun 29, 2021 12:48:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-29T12:48:15.636Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Jun 29, 2021 12:48:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-29T12:48:15.672Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Jun 29, 2021 12:48:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-29T12:48:15.707Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Jun 29, 2021 12:48:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-29T12:48:15.752Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Jun 29, 2021 12:48:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-29T12:48:15.778Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jun 29, 2021 12:48:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-29T12:48:15.822Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jun 29, 2021 12:48:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-29T12:48:15.861Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jun 29, 2021 12:48:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-29T12:48:15.891Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Jun 29, 2021 12:48:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-29T12:48:15.941Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Jun 29, 2021 12:48:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-29T12:48:15.976Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Jun 29, 2021 12:48:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-29T12:48:16.004Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Jun 29, 2021 12:48:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-29T12:48:16.029Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Jun 29, 2021 12:48:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-29T12:48:16.064Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Jun 29, 2021 12:48:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-29T12:48:16.097Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Jun 29, 2021 12:48:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-29T12:48:16.137Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Jun 29, 2021 12:48:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-29T12:48:16.565Z: Starting 5 ****s in us-central1-f...
Jun 29, 2021 12:48:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-29T12:48:32.950Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jun 29, 2021 12:49:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-29T12:49:02.252Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Jun 29, 2021 12:49:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-29T12:49:36.827Z: Workers have started successfully.
Jun 29, 2021 12:49:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-29T12:49:36.862Z: Workers have started successfully.
Jun 29, 2021 4:00:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-29T16:00:35.196Z: Cancel request is committed for workflow job: 2021-06-29_05_48_06-5962777165539570269.
Jun 29, 2021 4:00:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-29T16:00:35.284Z: Cleaning up.
Jun 29, 2021 4:00:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-29T16:00:35.389Z: Stopping **** pool...
Jun 29, 2021 4:00:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-29T16:00:35.462Z: Stopping **** pool...
Jun 29, 2021 4:01:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-29T16:01:36.016Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Jun 29, 2021 4:01:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-29T16:01:36.055Z: Worker pool stopped.
Jun 29, 2021 4:01:45 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-06-29_05_48_06-5962777165539570269 finished with status CANCELLED.
Load test results for test (ID): 339330fa-5532-4c64-b9f0-82d759a30dd6 and timestamp: 2021-06-29T12:48:00.750000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11411.881
dataflow_v2_java11_total_bytes_count               6.2842614E9
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210629124332
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:a6a930714d7ec025741108fbacb114dd302c5686a6f2c54329508b2a53216e6a
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:a6a930714d7ec025741108fbacb114dd302c5686a6f2c54329508b2a53216e6a
  Associated tags:
 - 20210629124332
Tags:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210629124332
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210629124332].
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:a6a930714d7ec025741108fbacb114dd302c5686a6f2c54329508b2a53216e6a].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 18m 28s
104 actionable tasks: 74 executed, 28 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/vybao5g2ushdm

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #11

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/11/display/redirect>

Changes:


------------------------------------------
[...truncated 64.84 KB...]
b3db0f45d3ac: Waiting
fd92fca19cd0: Pushed
0500effdfd0e: Pushed
8bcff52119ff: Pushed
2d15723a8693: Pushed
b3db0f45d3ac: Pushed
9f3ba0b01c7f: Pushed
1544a67c3095: Pushed
12dbc05e142a: Pushed
82f8ebb33b27: Layer already exists
c0848348e2f7: Layer already exists
79c550eb7bd2: Layer already exists
7095af798ace: Layer already exists
fe6a4fdbedc0: Layer already exists
fd7f1f80f342: Pushed
9926dec5ef66: Pushed
e4d0e810d54a: Layer already exists
4e006334a6fd: Layer already exists
eace477778d4: Pushed
b86a5adcb56c: Pushed
20210628124333: digest: sha256:2b407d52c08c9d46cb1441c2fd04674d9297ec5e3569eae4e7313ceb33ddc17c size: 4310

> Task :sdks:java:testing:load-tests:run
Jun 28, 2021 12:48:11 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Jun 28, 2021 12:48:11 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 190 files. Enable logging at DEBUG level to see which files will be staged.
Jun 28, 2021 12:48:12 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Jun 28, 2021 12:48:12 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Jun 28, 2021 12:48:15 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Jun 28, 2021 12:48:15 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <112268 bytes, hash 2b47dfad496dd055d96099c0f58a0e265d2e04248154130bfe58811af8ede687> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-K0ffrUlt0FXZYJnA9YoOJl0uBCSBVBML_liBGvjt5oc.pb
Jun 28, 2021 12:48:16 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 190 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Jun 28, 2021 12:48:17 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 190 files cached, 0 files newly uploaded in 0 seconds
Jun 28, 2021 12:48:17 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Jun 28, 2021 12:48:17 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@534180a2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6cf0a747, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@21505815, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7e53339, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58fef7f7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@31ddb930, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12eedfee, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c3c4a71, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1352434e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4f9a6c2d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2b6fcb9f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@75de6341, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@74170687, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@68f0f72c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3d96fa9e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3b545206, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77bb48d5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@181d8899, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12d5c30e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@b887730]
Jun 28, 2021 12:48:17 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Jun 28, 2021 12:48:17 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Jun 28, 2021 12:48:17 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Jun 28, 2021 12:48:17 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Jun 28, 2021 12:48:17 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3bbf6abe, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@9679750, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@9b9a327, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@75a0c890, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@358ab600, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@e26af6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@44065156, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3791f50e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@574b7f4a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@44ed0a8f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@32177fa5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@a96d56c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6ab4a5b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2abe9173, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@235d29d6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1fdca564, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43f9dd56, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1d12e953, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@57cb70be, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2d4608a6]
Jun 28, 2021 12:48:17 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Jun 28, 2021 12:48:17 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Jun 28, 2021 12:48:17 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Jun 28, 2021 12:48:17 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Jun 28, 2021 12:48:17 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Jun 28, 2021 12:48:17 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Jun 28, 2021 12:48:17 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Jun 28, 2021 12:48:17 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Jun 28, 2021 12:48:17 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Jun 28, 2021 12:48:17 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Jun 28, 2021 12:48:17 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Jun 28, 2021 12:48:17 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.32.0-SNAPSHOT
Jun 28, 2021 12:48:18 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-06-28_05_48_17-6823670410136647254?project=apache-beam-testing
Jun 28, 2021 12:48:18 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-06-28_05_48_17-6823670410136647254
Jun 28, 2021 12:48:18 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-06-28_05_48_17-6823670410136647254
Jun 28, 2021 12:48:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-06-28T12:48:22.398Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-06-nax8. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Jun 28, 2021 12:48:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-28T12:48:26.150Z: Worker configuration: n1-standard-2 in us-central1-f.
Jun 28, 2021 12:48:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-28T12:48:26.795Z: Expanding SplittableParDo operations into optimizable parts.
Jun 28, 2021 12:48:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-28T12:48:26.831Z: Expanding CollectionToSingleton operations into optimizable parts.
Jun 28, 2021 12:48:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-28T12:48:26.888Z: Expanding CoGroupByKey operations into optimizable parts.
Jun 28, 2021 12:48:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-28T12:48:26.960Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Jun 28, 2021 12:48:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-28T12:48:26.990Z: Expanding GroupByKey operations into streaming Read/Write steps
Jun 28, 2021 12:48:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-28T12:48:27.043Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jun 28, 2021 12:48:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-28T12:48:27.160Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jun 28, 2021 12:48:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-28T12:48:27.194Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Jun 28, 2021 12:48:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-28T12:48:27.221Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Jun 28, 2021 12:48:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-28T12:48:27.246Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Jun 28, 2021 12:48:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-28T12:48:27.282Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Jun 28, 2021 12:48:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-28T12:48:27.312Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jun 28, 2021 12:48:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-28T12:48:27.347Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jun 28, 2021 12:48:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-28T12:48:27.378Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jun 28, 2021 12:48:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-28T12:48:27.411Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Jun 28, 2021 12:48:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-28T12:48:27.445Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Jun 28, 2021 12:48:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-28T12:48:27.496Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Jun 28, 2021 12:48:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-28T12:48:27.524Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Jun 28, 2021 12:48:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-28T12:48:27.562Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jun 28, 2021 12:48:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-28T12:48:27.597Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jun 28, 2021 12:48:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-28T12:48:27.622Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jun 28, 2021 12:48:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-28T12:48:27.655Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Jun 28, 2021 12:48:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-28T12:48:27.681Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Jun 28, 2021 12:48:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-28T12:48:27.716Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Jun 28, 2021 12:48:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-28T12:48:27.748Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Jun 28, 2021 12:48:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-28T12:48:27.782Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Jun 28, 2021 12:48:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-28T12:48:27.805Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Jun 28, 2021 12:48:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-28T12:48:27.865Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Jun 28, 2021 12:48:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-28T12:48:27.894Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Jun 28, 2021 12:48:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-28T12:48:28.235Z: Starting 5 ****s in us-central1-f...
Jun 28, 2021 12:49:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-28T12:48:58.413Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jun 28, 2021 12:49:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-28T12:49:17.819Z: Autoscaling: Raised the number of ****s to 4 so that the pipeline can catch up with its backlog and keep up with its input rate.
Jun 28, 2021 12:49:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-28T12:49:17.850Z: Resized **** pool to 4, though goal was 5.  This could be a quota issue.
Jun 28, 2021 12:49:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-28T12:49:28.261Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Jun 28, 2021 12:49:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-28T12:49:56.994Z: Workers have started successfully.
Jun 28, 2021 12:49:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-28T12:49:57.020Z: Workers have started successfully.
Jun 28, 2021 2:03:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-06-28T14:03:28.356Z: Staged package animal-sniffer-annotations-1.20-vt1E38otwrj1wIzR1vDgznQJTsZ3gSYJaOA_wOd1Iqw.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/animal-sniffer-annotations-1.20-vt1E38otwrj1wIzR1vDgznQJTsZ3gSYJaOA_wOd1Iqw.jar' is inaccessible.
Jun 28, 2021 2:03:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-06-28T14:03:28.522Z: Staged package aws-java-sdk-cloudwatch-1.11.974-Ub2yicrz1p27ushPndFcVBigc3SuIZ4G-Yaq45B1iUo.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/aws-java-sdk-cloudwatch-1.11.974-Ub2yicrz1p27ushPndFcVBigc3SuIZ4G-Yaq45B1iUo.jar' is inaccessible.
Jun 28, 2021 2:03:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-06-28T14:03:28.568Z: Staged package aws-java-sdk-core-1.11.974-s3lPhICKsq1xExGOyj03VQHRqOsuX5YUc-UUlnhiA64.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/aws-java-sdk-core-1.11.974-s3lPhICKsq1xExGOyj03VQHRqOsuX5YUc-UUlnhiA64.jar' is inaccessible.
Jun 28, 2021 2:03:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-06-28T14:03:28.699Z: Staged package aws-java-sdk-dynamodb-1.11.974-RLvIDrX-TDIDN06gNnaHm9dQbuHCD_XSBesOuniSTk4.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/aws-java-sdk-dynamodb-1.11.974-RLvIDrX-TDIDN06gNnaHm9dQbuHCD_XSBesOuniSTk4.jar' is inaccessible.
Jun 28, 2021 2:03:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-06-28T14:03:28.759Z: Staged package aws-java-sdk-kinesis-1.11.974-kbU_CjbRpkrhhFeoyIMQA81nTYmniJY77w8Smd7PLpY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/aws-java-sdk-kinesis-1.11.974-kbU_CjbRpkrhhFeoyIMQA81nTYmniJY77w8Smd7PLpY.jar' is inaccessible.
Jun 28, 2021 2:03:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-06-28T14:03:28.792Z: Staged package aws-java-sdk-kms-1.11.974-HWH_mHclw3Wv4_GJ56x62xFDHSPSkZWjeKAlx40XrTk.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/aws-java-sdk-kms-1.11.974-HWH_mHclw3Wv4_GJ56x62xFDHSPSkZWjeKAlx40XrTk.jar' is inaccessible.
Jun 28, 2021 2:03:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-06-28T14:03:28.838Z: Staged package aws-java-sdk-s3-1.11.974-Qgd0wgaOuI_uyxxQI4Ulljk9bZEq-3JZArQF0zU5yzA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/aws-java-sdk-s3-1.11.974-Qgd0wgaOuI_uyxxQI4Ulljk9bZEq-3JZArQF0zU5yzA.jar' is inaccessible.
Jun 28, 2021 2:03:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-06-28T14:03:29.354Z: Staged package commons-io-2.6--HfTBGYKwqFC84ZbrfyXHex-1zx0fH-NXS9ROcpzZRM.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/commons-io-2.6--HfTBGYKwqFC84ZbrfyXHex-1zx0fH-NXS9ROcpzZRM.jar' is inaccessible.
Jun 28, 2021 2:03:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-06-28T14:03:29.386Z: Staged package commons-lang-2.6-UPEbCfh3wpTVbyRGP0fSj5Kc9QRPZIZhwPDPuumi9Jw.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/commons-lang-2.6-UPEbCfh3wpTVbyRGP0fSj5Kc9QRPZIZhwPDPuumi9Jw.jar' is inaccessible.
Jun 28, 2021 2:03:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-06-28T14:03:30.586Z: Staged package ion-java-1.0.2-DRJ7IFofzgq8KjdXoEF0hlG8ZsFc9MBZusWDOyfUcaU.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/ion-java-1.0.2-DRJ7IFofzgq8KjdXoEF0hlG8ZsFc9MBZusWDOyfUcaU.jar' is inaccessible.
Jun 28, 2021 2:03:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-06-28T14:03:30.769Z: Staged package jmespath-java-1.11.974-QaUt5Hnw6wd7ULTQ5Oj4j_zpRr-rnMCdfGwiwkSo8-0.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/jmespath-java-1.11.974-QaUt5Hnw6wd7ULTQ5Oj4j_zpRr-rnMCdfGwiwkSo8-0.jar' is inaccessible.
Jun 28, 2021 2:03:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-06-28T14:03:31.133Z: Staged package opencensus-contrib-http-util-0.28.0-ScPbKinx_bL3OSjL6pab0dQPq3zFu2JzAiur2W96eJs.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/opencensus-contrib-http-util-0.28.0-ScPbKinx_bL3OSjL6pab0dQPq3zFu2JzAiur2W96eJs.jar' is inaccessible.
Jun 28, 2021 2:03:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-06-28T14:03:31.591Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jun 28, 2021 2:06:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-06-28T14:06:31.172Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jun 28, 2021 4:00:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-28T16:00:36.164Z: Cancel request is committed for workflow job: 2021-06-28_05_48_17-6823670410136647254.
Jun 28, 2021 4:00:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-28T16:00:36.221Z: Cleaning up.
Jun 28, 2021 4:00:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-28T16:00:36.333Z: Stopping **** pool...
Jun 28, 2021 4:00:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-28T16:00:36.377Z: Stopping **** pool...
Jun 28, 2021 4:01:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-28T16:01:28.634Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Jun 28, 2021 4:01:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-28T16:01:28.665Z: Worker pool stopped.
Jun 28, 2021 4:01:38 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-06-28_05_48_17-6823670410136647254 finished with status CANCELLED.
Load test results for test (ID): f4fbea02-beb6-479f-9e8a-3dc557134431 and timestamp: 2021-06-28T12:48:12.083000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11402.035
dataflow_v2_java11_total_bytes_count               3.0701141E9
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210628124333
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:2b407d52c08c9d46cb1441c2fd04674d9297ec5e3569eae4e7313ceb33ddc17c
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:2b407d52c08c9d46cb1441c2fd04674d9297ec5e3569eae4e7313ceb33ddc17c
  Associated tags:
 - 20210628124333
Tags:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210628124333
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210628124333].
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:2b407d52c08c9d46cb1441c2fd04674d9297ec5e3569eae4e7313ceb33ddc17c].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 18m 20s
104 actionable tasks: 74 executed, 28 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/i4virflcvcpbk

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 - Build # 10 - Aborted!

Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 - Build # 10 - Aborted:

Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/10/ to view the results.

Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #9

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/9/display/redirect?page=changes>

Changes:

[noreply] [BEAM-9547] Add implementation for from_dict, from_records (#15034)

[noreply] [BEAM-11951] added "Differences from Pandas" page for DataFrame (#15074)

[noreply] [BEAM-9547] reindex is order-sensitive (#15032)

[noreply] Minor: Fix incorrect s.apache.org links (#15084)

[noreply] Minor: Announce DataFrame API leaving experimental in 2.32.0 (#15053)

[noreply] [BEAM-9547] Add support for xs on DataFrame and Series (#15078)


------------------------------------------
[...truncated 64.91 KB...]
e4d0e810d54a: Layer already exists
4e006334a6fd: Layer already exists
d714e83033fc: Pushed
82277aaf4310: Pushed
20210626124333: digest: sha256:2ffcc82595e6fce6022a3697019a720e3b19662d27cb45754092baca0be6ad0f size: 4310

> Task :sdks:java:testing:load-tests:run
Jun 26, 2021 12:48:13 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Jun 26, 2021 12:48:13 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 190 files. Enable logging at DEBUG level to see which files will be staged.
Jun 26, 2021 12:48:14 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Jun 26, 2021 12:48:14 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Jun 26, 2021 12:48:17 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Jun 26, 2021 12:48:17 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <112268 bytes, hash c2bf55c9b786d1178af9f0423ba0f4d158cc32770f62476060aa9cc3db10ef0a> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-wr9VybeG0ReK-fBCO6D00VjMMncPYkdgYKqcw9sQ7wo.pb
Jun 26, 2021 12:48:18 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 190 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Jun 26, 2021 12:48:19 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 190 files cached, 0 files newly uploaded in 0 seconds
Jun 26, 2021 12:48:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Jun 26, 2021 12:48:19 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@117d32e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@51d387d3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2370ac7a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@534180a2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6cf0a747, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@21505815, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7e53339, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58fef7f7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@31ddb930, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12eedfee, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c3c4a71, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1352434e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4f9a6c2d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2b6fcb9f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@75de6341, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@74170687, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@68f0f72c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3d96fa9e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3b545206, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77bb48d5]
Jun 26, 2021 12:48:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Jun 26, 2021 12:48:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Jun 26, 2021 12:48:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Jun 26, 2021 12:48:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Jun 26, 2021 12:48:19 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@64aad809, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1f03fba0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@17dd671f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3bbf6abe, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@9679750, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@9b9a327, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@75a0c890, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@358ab600, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@e26af6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@44065156, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3791f50e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@574b7f4a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@44ed0a8f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@32177fa5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@a96d56c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6ab4a5b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2abe9173, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@235d29d6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1fdca564, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43f9dd56]
Jun 26, 2021 12:48:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Jun 26, 2021 12:48:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Jun 26, 2021 12:48:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Jun 26, 2021 12:48:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Jun 26, 2021 12:48:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Jun 26, 2021 12:48:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Jun 26, 2021 12:48:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Jun 26, 2021 12:48:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Jun 26, 2021 12:48:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Jun 26, 2021 12:48:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Jun 26, 2021 12:48:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Jun 26, 2021 12:48:19 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.32.0-SNAPSHOT
Jun 26, 2021 12:48:20 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-06-26_05_48_19-7305249796223110340?project=apache-beam-testing
Jun 26, 2021 12:48:20 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-06-26_05_48_19-7305249796223110340
Jun 26, 2021 12:48:20 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-06-26_05_48_19-7305249796223110340
Jun 26, 2021 12:48:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-06-26T12:48:24.765Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-06-n40q. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Jun 26, 2021 12:48:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-26T12:48:28.652Z: Worker configuration: n1-standard-2 in us-central1-f.
Jun 26, 2021 12:48:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-26T12:48:29.243Z: Expanding SplittableParDo operations into optimizable parts.
Jun 26, 2021 12:48:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-26T12:48:29.277Z: Expanding CollectionToSingleton operations into optimizable parts.
Jun 26, 2021 12:48:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-26T12:48:29.355Z: Expanding CoGroupByKey operations into optimizable parts.
Jun 26, 2021 12:48:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-26T12:48:29.438Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Jun 26, 2021 12:48:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-26T12:48:29.480Z: Expanding GroupByKey operations into streaming Read/Write steps
Jun 26, 2021 12:48:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-26T12:48:29.556Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jun 26, 2021 12:48:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-26T12:48:29.677Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jun 26, 2021 12:48:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-26T12:48:29.717Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Jun 26, 2021 12:48:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-26T12:48:29.742Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Jun 26, 2021 12:48:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-26T12:48:29.772Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Jun 26, 2021 12:48:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-26T12:48:29.822Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Jun 26, 2021 12:48:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-26T12:48:29.866Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jun 26, 2021 12:48:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-26T12:48:29.897Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jun 26, 2021 12:48:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-26T12:48:29.930Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jun 26, 2021 12:48:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-26T12:48:29.963Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Jun 26, 2021 12:48:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-26T12:48:30.014Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Jun 26, 2021 12:48:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-26T12:48:30.049Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Jun 26, 2021 12:48:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-26T12:48:30.090Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Jun 26, 2021 12:48:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-26T12:48:30.121Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jun 26, 2021 12:48:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-26T12:48:30.156Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jun 26, 2021 12:48:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-26T12:48:30.191Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jun 26, 2021 12:48:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-26T12:48:30.214Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Jun 26, 2021 12:48:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-26T12:48:30.250Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Jun 26, 2021 12:48:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-26T12:48:30.278Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Jun 26, 2021 12:48:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-26T12:48:30.320Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Jun 26, 2021 12:48:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-26T12:48:30.352Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Jun 26, 2021 12:48:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-26T12:48:30.383Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Jun 26, 2021 12:48:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-26T12:48:30.411Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Jun 26, 2021 12:48:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-26T12:48:30.440Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Jun 26, 2021 12:48:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-26T12:48:30.852Z: Starting 5 ****s in us-central1-f...
Jun 26, 2021 12:48:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-26T12:48:39.031Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jun 26, 2021 12:49:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-26T12:49:11.381Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Jun 26, 2021 12:49:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-26T12:49:49.553Z: Workers have started successfully.
Jun 26, 2021 12:49:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-26T12:49:49.593Z: Workers have started successfully.
Jun 26, 2021 3:15:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-06-26T15:15:30.952Z: Staged package amazon-kinesis-producer-0.14.1-5YYT_oyDFV17VWs8Of43O7N08sRfHu0SuJeqVSFCuLg.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/amazon-kinesis-producer-0.14.1-5YYT_oyDFV17VWs8Of43O7N08sRfHu0SuJeqVSFCuLg.jar' is inaccessible.
Jun 26, 2021 3:15:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-06-26T15:15:33.654Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jun 26, 2021 3:18:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-06-26T15:18:33.452Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jun 26, 2021 3:21:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-06-26T15:21:30.962Z: Staged package amazon-kinesis-producer-0.14.1-5YYT_oyDFV17VWs8Of43O7N08sRfHu0SuJeqVSFCuLg.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/amazon-kinesis-producer-0.14.1-5YYT_oyDFV17VWs8Of43O7N08sRfHu0SuJeqVSFCuLg.jar' is inaccessible.
Jun 26, 2021 3:21:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-06-26T15:21:33.695Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jun 26, 2021 3:24:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-06-26T15:24:33.665Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jun 26, 2021 3:27:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-06-26T15:27:30.945Z: Staged package amazon-kinesis-producer-0.14.1-5YYT_oyDFV17VWs8Of43O7N08sRfHu0SuJeqVSFCuLg.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/amazon-kinesis-producer-0.14.1-5YYT_oyDFV17VWs8Of43O7N08sRfHu0SuJeqVSFCuLg.jar' is inaccessible.
Jun 26, 2021 3:27:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-06-26T15:27:33.294Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jun 26, 2021 3:30:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-06-26T15:30:33.297Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jun 26, 2021 3:33:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-06-26T15:33:30.927Z: Staged package amazon-kinesis-producer-0.14.1-5YYT_oyDFV17VWs8Of43O7N08sRfHu0SuJeqVSFCuLg.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/amazon-kinesis-producer-0.14.1-5YYT_oyDFV17VWs8Of43O7N08sRfHu0SuJeqVSFCuLg.jar' is inaccessible.
Jun 26, 2021 3:33:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-06-26T15:33:33.038Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jun 26, 2021 3:36:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-06-26T15:36:33.771Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jun 26, 2021 3:39:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-06-26T15:39:30.940Z: Staged package amazon-kinesis-producer-0.14.1-5YYT_oyDFV17VWs8Of43O7N08sRfHu0SuJeqVSFCuLg.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/amazon-kinesis-producer-0.14.1-5YYT_oyDFV17VWs8Of43O7N08sRfHu0SuJeqVSFCuLg.jar' is inaccessible.
Jun 26, 2021 3:39:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-06-26T15:39:33.067Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jun 26, 2021 3:42:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-06-26T15:42:33.039Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jun 26, 2021 3:45:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-06-26T15:45:30.954Z: Staged package amazon-kinesis-producer-0.14.1-5YYT_oyDFV17VWs8Of43O7N08sRfHu0SuJeqVSFCuLg.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/amazon-kinesis-producer-0.14.1-5YYT_oyDFV17VWs8Of43O7N08sRfHu0SuJeqVSFCuLg.jar' is inaccessible.
Jun 26, 2021 3:45:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-06-26T15:45:33.550Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jun 26, 2021 3:48:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-06-26T15:48:33.312Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jun 26, 2021 3:51:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-06-26T15:51:30.925Z: Staged package amazon-kinesis-producer-0.14.1-5YYT_oyDFV17VWs8Of43O7N08sRfHu0SuJeqVSFCuLg.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/amazon-kinesis-producer-0.14.1-5YYT_oyDFV17VWs8Of43O7N08sRfHu0SuJeqVSFCuLg.jar' is inaccessible.
Jun 26, 2021 3:51:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-06-26T15:51:33.055Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jun 26, 2021 3:54:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-06-26T15:54:33.161Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jun 26, 2021 3:57:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-06-26T15:57:30.931Z: Staged package amazon-kinesis-producer-0.14.1-5YYT_oyDFV17VWs8Of43O7N08sRfHu0SuJeqVSFCuLg.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/amazon-kinesis-producer-0.14.1-5YYT_oyDFV17VWs8Of43O7N08sRfHu0SuJeqVSFCuLg.jar' is inaccessible.
Jun 26, 2021 3:57:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-06-26T15:57:33.112Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jun 26, 2021 4:00:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-26T16:00:33.321Z: Cancel request is committed for workflow job: 2021-06-26_05_48_19-7305249796223110340.
Jun 26, 2021 4:00:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-06-26T16:00:34.166Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Jun 26, 2021 4:00:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-26T16:00:34.196Z: Cleaning up.
Jun 26, 2021 4:00:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-26T16:00:34.269Z: Stopping **** pool...
Jun 26, 2021 4:00:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-26T16:00:34.316Z: Stopping **** pool...
Jun 26, 2021 4:01:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-26T16:01:25.445Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Jun 26, 2021 4:01:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-26T16:01:25.478Z: Worker pool stopped.
Jun 26, 2021 4:01:33 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-06-26_05_48_19-7305249796223110340 finished with status CANCELLED.
Load test results for test (ID): 3720721b-d468-4169-8a97-014bc7f0fab8 and timestamp: 2021-06-26T12:48:14.082000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11278.074
dataflow_v2_java11_total_bytes_count                5.294723E8
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210626124333
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:2ffcc82595e6fce6022a3697019a720e3b19662d27cb45754092baca0be6ad0f
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:2ffcc82595e6fce6022a3697019a720e3b19662d27cb45754092baca0be6ad0f
  Associated tags:
 - 20210626124333
Tags:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210626124333
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210626124333].
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:2ffcc82595e6fce6022a3697019a720e3b19662d27cb45754092baca0be6ad0f].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 18m 15s
104 actionable tasks: 74 executed, 28 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/w7mhe5emvufcm

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #8

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/8/display/redirect?page=changes>

Changes:

[noreply] [BEAM-11514] Make schema error more helpful (#15075)

[noreply] Minor: Fix missing f-string in nlargest/nsmallest error messages


------------------------------------------
[...truncated 64.84 KB...]
82f8ebb33b27: Waiting
5c94749a5b87: Waiting
fe6a4fdbedc0: Waiting
4e006334a6fd: Waiting
e4d0e810d54a: Waiting
abcc4c5a4e90: Pushed
1fdeb01c66b5: Pushed
889d8f0fa13e: Pushed
e4d8e8008109: Pushed
49bd1ba324c5: Pushed
dfb2bb4c59c2: Pushed
04e91a227de3: Pushed
ecc6bd6394dd: Pushed
82f8ebb33b27: Layer already exists
c0848348e2f7: Layer already exists
79c550eb7bd2: Layer already exists
7095af798ace: Layer already exists
fe6a4fdbedc0: Layer already exists
e4d0e810d54a: Layer already exists
c1727b074e61: Pushed
4e006334a6fd: Layer already exists
e63e35a8c551: Pushed
aa331c176bff: Pushed
5c94749a5b87: Pushed
20210625124335: digest: sha256:5658c18895bb8ebe24eb19ea5e025ab0c842e9f0eaa04fe64918a9f4c5a80c2c size: 4310

> Task :sdks:java:testing:load-tests:run
Jun 25, 2021 12:48:15 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Jun 25, 2021 12:48:15 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 190 files. Enable logging at DEBUG level to see which files will be staged.
Jun 25, 2021 12:48:16 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Jun 25, 2021 12:48:16 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Jun 25, 2021 12:48:18 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Jun 25, 2021 12:48:18 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <112268 bytes, hash d57f8aee6cf2434870843921bfdf4142a50fd21010d111d21cf29224df17466f> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-1X-K7mzyQ0hwhDkhv99BQqUP0hAQ0RHSHPKSJN8XRm8.pb
Jun 25, 2021 12:48:20 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 190 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Jun 25, 2021 12:48:21 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 190 files cached, 0 files newly uploaded in 0 seconds
Jun 25, 2021 12:48:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Jun 25, 2021 12:48:21 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@117d32e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@51d387d3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2370ac7a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@534180a2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6cf0a747, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@21505815, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7e53339, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58fef7f7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@31ddb930, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12eedfee, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c3c4a71, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1352434e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4f9a6c2d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2b6fcb9f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@75de6341, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@74170687, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@68f0f72c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3d96fa9e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3b545206, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77bb48d5]
Jun 25, 2021 12:48:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Jun 25, 2021 12:48:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Jun 25, 2021 12:48:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Jun 25, 2021 12:48:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Jun 25, 2021 12:48:21 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@64aad809, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1f03fba0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@17dd671f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3bbf6abe, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@9679750, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@9b9a327, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@75a0c890, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@358ab600, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@e26af6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@44065156, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3791f50e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@574b7f4a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@44ed0a8f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@32177fa5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@a96d56c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6ab4a5b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2abe9173, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@235d29d6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1fdca564, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43f9dd56]
Jun 25, 2021 12:48:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Jun 25, 2021 12:48:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Jun 25, 2021 12:48:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Jun 25, 2021 12:48:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Jun 25, 2021 12:48:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Jun 25, 2021 12:48:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Jun 25, 2021 12:48:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Jun 25, 2021 12:48:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Jun 25, 2021 12:48:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Jun 25, 2021 12:48:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Jun 25, 2021 12:48:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Jun 25, 2021 12:48:21 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.32.0-SNAPSHOT
Jun 25, 2021 12:48:22 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-06-25_05_48_21-15246061112316458490?project=apache-beam-testing
Jun 25, 2021 12:48:22 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-06-25_05_48_21-15246061112316458490
Jun 25, 2021 12:48:22 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-06-25_05_48_21-15246061112316458490
Jun 25, 2021 12:48:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-06-25T12:48:26.366Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-06-v5fy. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Jun 25, 2021 12:48:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-25T12:48:30.922Z: Worker configuration: n1-standard-2 in us-central1-f.
Jun 25, 2021 12:48:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-25T12:48:31.515Z: Expanding SplittableParDo operations into optimizable parts.
Jun 25, 2021 12:48:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-25T12:48:31.552Z: Expanding CollectionToSingleton operations into optimizable parts.
Jun 25, 2021 12:48:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-25T12:48:31.650Z: Expanding CoGroupByKey operations into optimizable parts.
Jun 25, 2021 12:48:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-25T12:48:31.703Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Jun 25, 2021 12:48:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-25T12:48:31.731Z: Expanding GroupByKey operations into streaming Read/Write steps
Jun 25, 2021 12:48:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-25T12:48:31.786Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jun 25, 2021 12:48:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-25T12:48:31.874Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jun 25, 2021 12:48:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-25T12:48:31.899Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Jun 25, 2021 12:48:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-25T12:48:31.931Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Jun 25, 2021 12:48:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-25T12:48:31.968Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Jun 25, 2021 12:48:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-25T12:48:31.996Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Jun 25, 2021 12:48:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-25T12:48:32.027Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jun 25, 2021 12:48:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-25T12:48:32.069Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jun 25, 2021 12:48:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-25T12:48:32.107Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jun 25, 2021 12:48:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-25T12:48:32.139Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Jun 25, 2021 12:48:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-25T12:48:32.163Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Jun 25, 2021 12:48:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-25T12:48:32.194Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Jun 25, 2021 12:48:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-25T12:48:32.231Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Jun 25, 2021 12:48:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-25T12:48:32.259Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jun 25, 2021 12:48:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-25T12:48:32.288Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jun 25, 2021 12:48:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-25T12:48:32.323Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jun 25, 2021 12:48:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-25T12:48:32.357Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Jun 25, 2021 12:48:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-25T12:48:32.392Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Jun 25, 2021 12:48:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-25T12:48:32.430Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Jun 25, 2021 12:48:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-25T12:48:32.459Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Jun 25, 2021 12:48:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-25T12:48:32.488Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Jun 25, 2021 12:48:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-25T12:48:32.532Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Jun 25, 2021 12:48:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-25T12:48:32.563Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Jun 25, 2021 12:48:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-25T12:48:32.604Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Jun 25, 2021 12:48:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-25T12:48:32.977Z: Starting 5 ****s in us-central1-f...
Jun 25, 2021 12:48:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-25T12:48:39.574Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jun 25, 2021 12:49:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-25T12:49:24.153Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Jun 25, 2021 12:50:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-25T12:50:00.740Z: Workers have started successfully.
Jun 25, 2021 12:50:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-25T12:50:00.769Z: Workers have started successfully.
Jun 25, 2021 4:00:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-25T16:00:33.601Z: Cancel request is committed for workflow job: 2021-06-25_05_48_21-15246061112316458490.
Jun 25, 2021 4:00:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-25T16:00:36.517Z: Cleaning up.
Jun 25, 2021 4:00:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-25T16:00:36.620Z: Stopping **** pool...
Jun 25, 2021 4:00:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-25T16:00:36.665Z: Stopping **** pool...
Jun 25, 2021 4:01:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-25T16:01:27.670Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Jun 25, 2021 4:01:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-25T16:01:27.704Z: Worker pool stopped.
Jun 25, 2021 4:01:32 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-06-25_05_48_21-15246061112316458490 finished with status CANCELLED.
Load test results for test (ID): 2f8ab986-f277-4481-9901-10157f760d8b and timestamp: 2021-06-25T12:48:16.314000000Z:
                 Metric:                    Value:
dataflow_v2_java11_runtime_sec                 11402.563
dataflow_v2_java11_total_bytes_count               8.1762279E9
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
	at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
	at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
	at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)

> Task :sdks:java:testing:load-tests:run FAILED

> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210625124335
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:5658c18895bb8ebe24eb19ea5e025ab0c842e9f0eaa04fe64918a9f4c5a80c2c
Deleted: sha256:18a94fbdfba851980c0db98bb3735f7c6a297628b46f2b3f4a6b4803356955f3
Deleted: sha256:53ce7c1ab0c5cca28d832e45cfb8eae152be9605cc0083cd990751c225948808
Deleted: sha256:0e79882f90838b8f55ce3931d58b5f3c1b86a3ed9b13e99e106fddf2cbce11b9
Deleted: sha256:9d4cc4e20ccf30259fdc2cb21d096e414f28e1b7351669716fc000d5e0dcd73e
Deleted: sha256:eb79a0c1b70840a34df9dd7660c0e2dfb843691b6a48d5c5cf37df97a732bec5
Deleted: sha256:f17ce8d48a94435b2ab9b58661d6dd978d73957555d908907e30a1dd416e1a4a
Deleted: sha256:3015e6c9dc1e800e46c5cd03546655893033003070620fc877169949f7e03827
Deleted: sha256:46f042a3e0ba06e6dfb67f79453e6e30978d175bfecb8d8f8304ee9f447838d5
Deleted: sha256:25e1ce95e36c076cbcff27162643f6cddd1c8dfa812ceeae747c423827c01dcd
Deleted: sha256:5d22eba74ea090786719605427a779d0647875d71597fffe7ac185412d049be8
Deleted: sha256:cf76f2b43bfca995551adf45539e9096e197e4103b786f499b4ff4ee8b745448
Deleted: sha256:2d153c3bbba0f58375eb80f2c5618beb15f9510d2907bd0d53e83b1517cccda2
Deleted: sha256:bd7a55eebfd57906fd767c2ff1531f712be837cb648ddf9ce9b2dd51f34b4070
Deleted: sha256:f8528491136fe24173ed3b93a9d1194d5b6d922675e5677c4d706f97d5106623
Deleted: sha256:7e2cc630c55cc385ecef5485527831aa74d26819a41bca36a9c2ca7b4ff45aeb
Deleted: sha256:7d14ff02efdb61bd0b1f5a22502a80226027a60cb85103376c41b08f74ec94d7
Deleted: sha256:aacbff2a0ca4af28e8a56e12d80dd9fe45508243a3de1093074d319cda4c22b4
Deleted: sha256:930f21695c58a23b1db277e1bab9fa597fc7f1cd09fd4d118a5b15a0b93d8388
Deleted: sha256:76fe5cb6e3ec1fdb19719b4565b8858db46669cbde17369e88ba7ec7dc81eb01
Deleted: sha256:ee69f5546d6151c1737e48d065985c90ff8bd094fe9fab8a9b57e4af01f0285a
Deleted: sha256:1d3f6e191e6c8ac7634ec1556e99685d3568925bd089a328fbb27a7c36b8a20a
Deleted: sha256:39ad013de18891ad6c212d56dcf4324f6aaac76fb77fe4940d4228b2499d4546
Deleted: sha256:3465b3a5829a386d8eea8da28e0eda9751769d59afb942c921dcf50120a38463
Deleted: sha256:acbe24b3f745bf0d41e7d058dac442b4509c1decb14fffd3c5c30ef48e542ff4
Deleted: sha256:49de7db6d4d13553603cc093b17db054234710e844c3444fc5959b626ebce379
Deleted: sha256:e864c630d81aa956b0f263329edb061928cdd10d299929022068dd2500857ffd
Deleted: sha256:03dc7f6bd75bc921ba54e4bec807c9e9ca6122fceb00cc0323dd1768cec534be
Deleted: sha256:9a6f82638921beece2376539e60b7c55782bace36d14b9a03eece4b390689446
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:5658c18895bb8ebe24eb19ea5e025ab0c842e9f0eaa04fe64918a9f4c5a80c2c
  Associated tags:
 - 20210625124335
Tags:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210625124335
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210625124335].
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:5658c18895bb8ebe24eb19ea5e025ab0c842e9f0eaa04fe64918a9f4c5a80c2c].

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3h 18m 14s
104 actionable tasks: 74 executed, 28 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/jvod4em36n6gc

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 - Build # 7 - Aborted!

Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 - Build # 7 - Aborted:

Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/7/ to view the results.

Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11 #6

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/6/display/redirect?page=changes>

Changes:

[egalpin] [BEAM-12421] Migrates ElasticsearchIOTest to use testContainers

[egalpin] Adds factory method for creating consistent ES containers

[egalpin] Renames elasticsearchIOTestContainerFactory to createTestContainer

[egalpin] Reduces number of docs for UTests from 400 to 100

[Robert Bradshaw] [BEAM-12475] Reject splits for previous bundles.

[noreply] Add 2.30 release date

[noreply] Remove placeholder entries from 2.30

[Robert Bradshaw] mypy

[Boyuan Zhang] [BEAM-12475] Ensure the BundleProcessor.getId() is null before calling

[benjamin.gonzalez] [BEAM-37137] Fix xlang dependency error

[heejong] Update Dataflow Python container version to 20210622

[Robert Burke] [BEAM-12528] Discard failed plans.

[noreply] [BEAM-9547] Add support for `Series.dt` methods (#14940)


------------------------------------------
[...truncated 60.81 KB...]
Resolving cached github.com/etcd-io/etcd: commit='11214aa33bf5a47d3d9d8dafe0f6b97237dfe921', urls=[https://github.com/etcd-io/etcd.git, git@github.com:etcd-io/etcd.git]

> Task :sdks:go:installDependencies
> Task :sdks:go:buildLinuxAmd64
> Task :sdks:go:goBuild

> Task :sdks:java:container:resolveBuildDependencies
Resolving ./github.com/apache/beam/sdks/go@/home/jenkins/jenkins-slave/workspace/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/src/sdks/go

> Task :sdks:java:container:installDependencies
> Task :sdks:java:container:buildLinuxAmd64
> Task :sdks:java:container:goBuild
> Task :sdks:java:container:java11:copySdkHarnessLauncher
> Task :sdks:java:container:java11:dockerPrepare
> Task :sdks:java:container:java11:docker

> Task :runners:google-cloud-dataflow-java:buildAndPushDockerContainer
WARNING: `gcloud docker` will not be supported for Docker client versions above 18.03.

As an alternative, use `gcloud auth configure-docker` to configure `docker` to
use `gcloud` as a credential helper, then use `docker` as you would for non-GCR
registries, e.g. `docker pull gcr.io/project-id/my-image`. Add
`--verbosity=error` to silence this warning: `gcloud docker
--verbosity=error -- pull gcr.io/project-id/my-image`.

See: https://cloud.google.com/container-registry/docs/support/deprecation-notices#gcloud-docker

The push refers to repository [us.gcr.io/apache-beam-testing/java-postcommit-it/java]
6181f1c01a4d: Preparing
0b3edb8b86bc: Preparing
2af05e0e7a2f: Preparing
6cafd47bf8c8: Preparing
a1094303e5be: Preparing
1d7819584159: Preparing
6642dc835280: Preparing
bb1ae6d05e71: Preparing
5ab462059aa5: Preparing
21bed984b435: Preparing
f0f679e0fe89: Preparing
632f6283daff: Preparing
82f8ebb33b27: Preparing
c0848348e2f7: Preparing
79c550eb7bd2: Preparing
7095af798ace: Preparing
fe6a4fdbedc0: Preparing
e4d0e810d54a: Preparing
4e006334a6fd: Preparing
5ab462059aa5: Waiting
1d7819584159: Waiting
7095af798ace: Waiting
21bed984b435: Waiting
f0f679e0fe89: Waiting
82f8ebb33b27: Waiting
6642dc835280: Waiting
c0848348e2f7: Waiting
79c550eb7bd2: Waiting
bb1ae6d05e71: Waiting
fe6a4fdbedc0: Waiting
632f6283daff: Waiting
e4d0e810d54a: Waiting
a1094303e5be: Pushed
2af05e0e7a2f: Pushed
0b3edb8b86bc: Pushed
6181f1c01a4d: Pushed
1d7819584159: Pushed
6cafd47bf8c8: Pushed
bb1ae6d05e71: Pushed
5ab462059aa5: Pushed
82f8ebb33b27: Layer already exists
c0848348e2f7: Layer already exists
79c550eb7bd2: Layer already exists
7095af798ace: Layer already exists
fe6a4fdbedc0: Layer already exists
f0f679e0fe89: Pushed
4e006334a6fd: Layer already exists
e4d0e810d54a: Layer already exists
632f6283daff: Pushed
6642dc835280: Pushed
21bed984b435: Pushed
20210623124329: digest: sha256:45c49b9f0f09b4dc391a347cfc65bc639f421de7b74a5474a5dcddd30e969dad size: 4310

> Task :sdks:java:testing:load-tests:run
Jun 23, 2021 12:47:46 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Jun 23, 2021 12:47:47 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 190 files. Enable logging at DEBUG level to see which files will be staged.
Jun 23, 2021 12:47:48 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Jun 23, 2021 12:47:48 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Jun 23, 2021 12:47:49 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Jun 23, 2021 12:47:49 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <112268 bytes, hash 587b02300459150f997c1857396654d0a1edbd323fef455416300ae162415453> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-WHsCMARZFQ-ZfBhXOWZU0KHtvTI_70VUFjAK4WJBVFM.pb
Jun 23, 2021 12:47:51 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 190 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Jun 23, 2021 12:47:51 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 190 files cached, 0 files newly uploaded in 0 seconds
Jun 23, 2021 12:47:51 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Jun 23, 2021 12:47:51 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@ccf91df, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@117d32e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@51d387d3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2370ac7a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@534180a2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6cf0a747, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@21505815, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7e53339, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58fef7f7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@31ddb930, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12eedfee, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c3c4a71, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1352434e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4f9a6c2d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2b6fcb9f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@75de6341, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@74170687, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@68f0f72c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3d96fa9e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3b545206]
Jun 23, 2021 12:47:51 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Jun 23, 2021 12:47:51 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Jun 23, 2021 12:47:51 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Jun 23, 2021 12:47:51 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Jun 23, 2021 12:47:51 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1886991b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@64aad809, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1f03fba0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@17dd671f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3bbf6abe, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@9679750, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@9b9a327, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@75a0c890, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@358ab600, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@e26af6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@44065156, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3791f50e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@574b7f4a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@44ed0a8f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@32177fa5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@a96d56c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6ab4a5b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2abe9173, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@235d29d6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1fdca564]
Jun 23, 2021 12:47:51 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Jun 23, 2021 12:47:51 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Jun 23, 2021 12:47:51 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Jun 23, 2021 12:47:51 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Jun 23, 2021 12:47:51 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Jun 23, 2021 12:47:51 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Jun 23, 2021 12:47:51 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Jun 23, 2021 12:47:51 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Jun 23, 2021 12:47:51 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Jun 23, 2021 12:47:51 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Jun 23, 2021 12:47:51 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Jun 23, 2021 12:47:52 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.32.0-SNAPSHOT
Jun 23, 2021 12:47:53 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-06-23_05_47_52-16587949485078404680?project=apache-beam-testing
Jun 23, 2021 12:47:53 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-06-23_05_47_52-16587949485078404680
Jun 23, 2021 12:47:53 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-06-23_05_47_52-16587949485078404680
Jun 23, 2021 12:47:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-06-23T12:47:57.529Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-06-1ujm. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Jun 23, 2021 12:48:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-23T12:48:01.476Z: Worker configuration: n1-standard-2 in us-central1-f.
Jun 23, 2021 12:48:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-23T12:48:01.994Z: Expanding SplittableParDo operations into optimizable parts.
Jun 23, 2021 12:48:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-23T12:48:02.023Z: Expanding CollectionToSingleton operations into optimizable parts.
Jun 23, 2021 12:48:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-23T12:48:02.086Z: Expanding CoGroupByKey operations into optimizable parts.
Jun 23, 2021 12:48:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-23T12:48:02.144Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Jun 23, 2021 12:48:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-23T12:48:02.173Z: Expanding GroupByKey operations into streaming Read/Write steps
Jun 23, 2021 12:48:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-23T12:48:02.222Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Jun 23, 2021 12:48:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-23T12:48:02.332Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Jun 23, 2021 12:48:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-23T12:48:02.373Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Jun 23, 2021 12:48:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-23T12:48:02.404Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Jun 23, 2021 12:48:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-23T12:48:02.430Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Jun 23, 2021 12:48:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-23T12:48:02.464Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Jun 23, 2021 12:48:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-23T12:48:02.491Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jun 23, 2021 12:48:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-23T12:48:02.527Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jun 23, 2021 12:48:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-23T12:48:02.559Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jun 23, 2021 12:48:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-23T12:48:02.597Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Jun 23, 2021 12:48:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-23T12:48:02.629Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Jun 23, 2021 12:48:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-23T12:48:02.663Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Jun 23, 2021 12:48:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-23T12:48:02.699Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Jun 23, 2021 12:48:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-23T12:48:02.730Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Jun 23, 2021 12:48:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-23T12:48:02.775Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Jun 23, 2021 12:48:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-23T12:48:02.814Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Jun 23, 2021 12:48:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-23T12:48:02.842Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Jun 23, 2021 12:48:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-23T12:48:02.869Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Jun 23, 2021 12:48:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-23T12:48:02.895Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Jun 23, 2021 12:48:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-23T12:48:02.929Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Jun 23, 2021 12:48:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-23T12:48:02.954Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Jun 23, 2021 12:48:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-23T12:48:02.976Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Jun 23, 2021 12:48:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-23T12:48:03.014Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Jun 23, 2021 12:48:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-23T12:48:03.044Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Jun 23, 2021 12:48:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-23T12:48:03.439Z: Starting 5 ****s in us-central1-f...
Jun 23, 2021 12:48:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-23T12:48:10.712Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jun 23, 2021 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-23T12:48:37.891Z: Autoscaling: Raised the number of ****s to 1 so that the pipeline can catch up with its backlog and keep up with its input rate.
Jun 23, 2021 12:48:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-23T12:48:37.920Z: Resized **** pool to 1, though goal was 5.  This could be a quota issue.
Jun 23, 2021 12:48:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-23T12:48:48.246Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Jun 23, 2021 12:49:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-23T12:49:24.907Z: Workers have started successfully.
Jun 23, 2021 12:49:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-06-23T12:49:24.943Z: Workers have started successfully.
FATAL: command execution failed
java.io.IOException: Backing channel 'apache-beam-jenkins-15' is disconnected.
	at hudson.remoting.RemoteInvocationHandler.channelOrFail(RemoteInvocationHandler.java:216)
	at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:286)
	at com.sun.proxy.$Proxy145.isAlive(Unknown Source)
	at hudson.Launcher$RemoteLauncher$ProcImpl.isAlive(Launcher.java:1211)
	at hudson.Launcher$RemoteLauncher$ProcImpl.join(Launcher.java:1203)
	at hudson.Launcher$ProcStarter.join(Launcher.java:523)
	at hudson.plugins.gradle.Gradle.perform(Gradle.java:317)
	at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
	at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:804)
	at hudson.model.Build$BuildExecution.build(Build.java:197)
	at hudson.model.Build$BuildExecution.doRun(Build.java:163)
	at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:514)
	at hudson.model.Run.execute(Run.java:1907)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
	at hudson.model.ResourceController.execute(ResourceController.java:97)
	at hudson.model.Executor.run(Executor.java:429)
Caused by: java.io.IOException: Pipe closed after 0 cycles
	at org.apache.sshd.common.channel.ChannelPipedInputStream.read(ChannelPipedInputStream.java:118)
	at org.apache.sshd.common.channel.ChannelPipedInputStream.read(ChannelPipedInputStream.java:101)
	at hudson.remoting.FlightRecorderInputStream.read(FlightRecorderInputStream.java:92)
	at hudson.remoting.ChunkedInputStream.readHeader(ChunkedInputStream.java:73)
	at hudson.remoting.ChunkedInputStream.readUntilBreak(ChunkedInputStream.java:103)
	at hudson.remoting.ChunkedCommandTransport.readBlock(ChunkedCommandTransport.java:39)
	at hudson.remoting.AbstractSynchronousByteArrayCommandTransport.read(AbstractSynchronousByteArrayCommandTransport.java:34)
	at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:63)
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
ERROR: apache-beam-jenkins-15 is offline; cannot locate jdk_1.8_latest

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org