You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@beam.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2022/03/31 17:05:14 UTC
beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17 - Build # 101 - Aborted!
beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17 - Build # 101 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17/101/ to view the results.
Jenkins build is back to normal : beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17 #142
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17/142/display/redirect?page=changes>
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17 #141
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17/141/display/redirect?page=changes>
Changes:
[andyye333] Add extra details to PubSub matcher errors
[noreply] Merge pull request #17559 from [BEAM-14423] Add exception injection
[noreply] [BEAM-11104] Allow self-checkpointing SDFs to return without finishing
[noreply] Merge pull request #17544 from [BEAM-14415] Exception handling tests for
[noreply] Merge pull request #17565 from [BEAM-14413] add Kafka exception test
[noreply] Merge pull request #17555 from [BEAM-14417] Adding exception handling
[noreply] [BEAM-14433] Improve Go split error message. (#17575)
[noreply] [BEAM-14429] Force java load test on dataflow runner v2
[noreply] Merge pull request #17577 from [BEAM-14435] Adding exception handling
[noreply] [BEAM-14347] Add generic registration functions for iters and emitters
[noreply] [BEAM-14169] Add Credentials rotation cron job for clusters (#17383)
[noreply] [BEAM-14347] Add generic registration for accumulators (#17579)
------------------------------------------
[...truncated 50.68 KB...]
8d3c4b8a056c: Preparing
779e60d40a60: Preparing
6e94b4ebcad2: Preparing
3a7cd99b8479: Preparing
c49c9f3cb678: Preparing
5a27e519e570: Waiting
3fe3663cf18e: Preparing
3634662e7534: Preparing
779e60d40a60: Waiting
6e94b4ebcad2: Waiting
267e49f181ed: Preparing
3bc383470c05: Preparing
e93827457889: Preparing
08fa02ce37eb: Preparing
a037458de4e0: Preparing
bafdbe68e4ae: Preparing
a13c519c6361: Preparing
08fa02ce37eb: Waiting
a13c519c6361: Waiting
a037458de4e0: Waiting
8d3c4b8a056c: Waiting
e93827457889: Waiting
3a7cd99b8479: Waiting
267e49f181ed: Waiting
c49c9f3cb678: Waiting
3bc383470c05: Waiting
83bac70d0466: Waiting
aa1c4990f94b: Pushed
5d41c137d1a8: Pushed
7d9293365344: Pushed
38be9522fc38: Pushed
a51548066d49: Pushed
8d3c4b8a056c: Pushed
5a27e519e570: Pushed
779e60d40a60: Pushed
6e94b4ebcad2: Pushed
c49c9f3cb678: Pushed
3a7cd99b8479: Pushed
3bc383470c05: Layer already exists
83bac70d0466: Pushed
e93827457889: Layer already exists
08fa02ce37eb: Layer already exists
a037458de4e0: Layer already exists
bafdbe68e4ae: Layer already exists
a13c519c6361: Layer already exists
3634662e7534: Pushed
267e49f181ed: Pushed
3fe3663cf18e: Pushed
20220510123038: digest: sha256:681c0d5f2a230de72485a61d99b98d34569d0303dda01320734c2ebe64c4a3da size: 4729
> Task :sdks:java:testing:load-tests:run
May 10, 2022 12:32:49 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
May 10, 2022 12:32:50 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 222 files. Enable logging at DEBUG level to see which files will be staged.
May 10, 2022 12:32:51 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
May 10, 2022 12:32:51 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
May 10, 2022 12:32:53 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 222 files from PipelineOptions.filesToStage to staging location to prepare for execution.
May 10, 2022 12:32:54 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 222 files cached, 0 files newly uploaded in 0 seconds
May 10, 2022 12:32:54 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
May 10, 2022 12:32:54 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <120279 bytes, hash 90d5250ac2615b6f15499695641c4bbc2e25c31d4352ec686073b7925252605d> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-kNUlCsJhW28VSZaVZBxLvC4lwx1DUuxoYHO3klJSYF0.pb
May 10, 2022 12:32:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
May 10, 2022 12:32:56 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 1 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4232b34a]
May 10, 2022 12:32:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
May 10, 2022 12:32:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
May 10, 2022 12:32:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
May 10, 2022 12:32:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
May 10, 2022 12:32:56 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 1 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@ffd4cba]
May 10, 2022 12:32:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
May 10, 2022 12:32:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
May 10, 2022 12:32:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
May 10, 2022 12:32:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
May 10, 2022 12:32:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
May 10, 2022 12:32:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
May 10, 2022 12:32:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
May 10, 2022 12:32:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
May 10, 2022 12:32:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
May 10, 2022 12:32:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
May 10, 2022 12:32:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
May 10, 2022 12:32:56 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.40.0-SNAPSHOT
May 10, 2022 12:32:57 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-05-10_05_32_56-7371717862294852586?project=apache-beam-testing
May 10, 2022 12:32:57 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2022-05-10_05_32_56-7371717862294852586
May 10, 2022 12:32:57 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2022-05-10_05_32_56-7371717862294852586
May 10, 2022 12:33:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2022-05-10T12:33:02.262Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java170dataflow0v20streaming0cogbk01-jenkins-05-ahvo. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
May 10, 2022 12:33:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-10T12:33:12.593Z: Worker configuration: e2-standard-2 in us-central1-b.
May 10, 2022 12:33:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-10T12:33:13.927Z: Expanding SplittableParDo operations into optimizable parts.
May 10, 2022 12:33:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-10T12:33:13.960Z: Expanding CollectionToSingleton operations into optimizable parts.
May 10, 2022 12:33:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-10T12:33:14.024Z: Expanding CoGroupByKey operations into optimizable parts.
May 10, 2022 12:33:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-10T12:33:14.093Z: Expanding SplittableProcessKeyed operations into optimizable parts.
May 10, 2022 12:33:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-10T12:33:14.125Z: Expanding GroupByKey operations into streaming Read/Write steps
May 10, 2022 12:33:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-10T12:33:14.186Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 10, 2022 12:33:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-10T12:33:14.308Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 10, 2022 12:33:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-10T12:33:14.336Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
May 10, 2022 12:33:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-10T12:33:14.371Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
May 10, 2022 12:33:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-10T12:33:14.405Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
May 10, 2022 12:33:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-10T12:33:14.439Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
May 10, 2022 12:33:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-10T12:33:14.476Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
May 10, 2022 12:33:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-10T12:33:14.506Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
May 10, 2022 12:33:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-10T12:33:14.543Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
May 10, 2022 12:33:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-10T12:33:14.575Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
May 10, 2022 12:33:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-10T12:33:14.606Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
May 10, 2022 12:33:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-10T12:33:14.659Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
May 10, 2022 12:33:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-10T12:33:14.694Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
May 10, 2022 12:33:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-10T12:33:14.738Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
May 10, 2022 12:33:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-10T12:33:14.772Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
May 10, 2022 12:33:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-10T12:33:14.797Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
May 10, 2022 12:33:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-10T12:33:14.827Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
May 10, 2022 12:33:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-10T12:33:14.864Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
May 10, 2022 12:33:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-10T12:33:14.889Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
May 10, 2022 12:33:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-10T12:33:14.918Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
May 10, 2022 12:33:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-10T12:33:14.952Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
May 10, 2022 12:33:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-10T12:33:14.987Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
May 10, 2022 12:33:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-10T12:33:15.020Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
May 10, 2022 12:33:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-10T12:33:15.047Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
May 10, 2022 12:33:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-10T12:33:15.240Z: Running job using Streaming Engine
May 10, 2022 12:33:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-10T12:33:15.523Z: Starting 5 ****s in us-central1-b...
May 10, 2022 12:33:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-10T12:33:21.076Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 10, 2022 12:33:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-10T12:33:38.137Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
May 10, 2022 12:34:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-10T12:34:46.474Z: Workers have started successfully.
May 10, 2022 2:13:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-10T14:13:36.440Z: Cleaning up.
May 10, 2022 2:13:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-10T14:13:36.646Z: Stopping **** pool...
May 10, 2022 2:13:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-10T14:13:36.700Z: Stopping **** pool...
May 10, 2022 2:14:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-10T14:14:11.917Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
May 10, 2022 2:14:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-10T14:14:11.960Z: Worker pool stopped.
May 10, 2022 2:14:18 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2022-05-10_05_32_56-7371717862294852586 finished with status DONE.
Load test results for test (ID): 8cdbdbfb-6fcb-49fa-a861-a1bb58fd0ad5 and timestamp: 2022-05-10T12:32:50.673000000Z:
Metric: Value:
dataflow_v2_java17_runtime_sec 5813.232
dataflow_v2_java17_total_bytes_count 6.0274294E9
> Task :runners:google-cloud-dataflow-java:cleanUpDockerJavaImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220510123038
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:681c0d5f2a230de72485a61d99b98d34569d0303dda01320734c2ebe64c4a3da
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220510123038]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:681c0d5f2a230de72485a61d99b98d34569d0303dda01320734c2ebe64c4a3da]
Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220510123038] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:681c0d5f2a230de72485a61d99b98d34569d0303dda01320734c2ebe64c4a3da])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:e016287d22625030f983cbf6f23e489ff06d1e82e3b1276c012f318dcaa366bf
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:e016287d22625030f983cbf6f23e489ff06d1e82e3b1276c012f318dcaa366bf
ERROR: (gcloud.container.images.delete) Not found: response: {'docker-distribution-api-version': 'registry/2.0', 'content-type': 'application/json', 'date': 'Tue, 10 May 2022 14:14:25 GMT', 'server': 'Docker Registry', 'cache-control': 'private', 'x-xss-protection': '0', 'x-frame-options': 'SAMEORIGIN', 'transfer-encoding': 'chunked', 'status': '404', 'content-length': '168', '-content-encoding': 'gzip'}
Failed to compute blob liveness for manifest: 'sha256:e016287d22625030f983cbf6f23e489ff06d1e82e3b1276c012f318dcaa366bf': None
> Task :runners:google-cloud-dataflow-java:cleanUpDockerJavaImages FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17/ws/src/runners/google-cloud-dataflow-java/build.gradle'> line: 297
* What went wrong:
Execution failed for task ':runners:google-cloud-dataflow-java:cleanUpDockerJavaImages'.
> Process 'command './scripts/cleanup_untagged_gcr_images.sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.
BUILD FAILED in 1h 44m
110 actionable tasks: 73 executed, 33 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/iters5gdlxc5s
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17 #140
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17/140/display/redirect?page=changes>
Changes:
[elias.segundo] Changing elegibility to AllNodeElegibility
[chamikaramj] Adds code reviewers for GCP I/O connectors and KafkaIO to Beam OWNERS
------------------------------------------
[...truncated 50.04 KB...]
e93827457889: Preparing
08fa02ce37eb: Preparing
a037458de4e0: Preparing
bafdbe68e4ae: Preparing
a13c519c6361: Preparing
4558b07d7c6c: Waiting
7572dcf244a7: Waiting
a037458de4e0: Waiting
bafdbe68e4ae: Waiting
a13c519c6361: Waiting
76cbfa6ab6de: Waiting
3bc383470c05: Waiting
212e3bed8e24: Waiting
a1cdc24dc423: Waiting
08fa02ce37eb: Waiting
e93827457889: Waiting
e71737e48960: Waiting
9eded4dea7e1: Waiting
d25a54d106dd: Waiting
65c0e27593f2: Waiting
b0a0cfc0c1c7: Pushed
99b1b0c017c9: Pushed
70d12e40179b: Pushed
74d7ad42a174: Pushed
a7b750c44030: Pushed
76cbfa6ab6de: Pushed
212e3bed8e24: Pushed
65c0e27593f2: Pushed
e71737e48960: Pushed
9eded4dea7e1: Pushed
a1cdc24dc423: Pushed
3bc383470c05: Layer already exists
e93827457889: Layer already exists
08fa02ce37eb: Layer already exists
4fdbf0a79d70: Pushed
a037458de4e0: Layer already exists
bafdbe68e4ae: Layer already exists
a13c519c6361: Layer already exists
4558b07d7c6c: Pushed
7572dcf244a7: Pushed
d25a54d106dd: Pushed
20220509123035: digest: sha256:ae661c08a40bbf8a47368c1570d2cbdb9dbb0865ef587bd35d60f43cbee2a87d size: 4729
> Task :sdks:java:testing:load-tests:run
May 09, 2022 12:32:39 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
May 09, 2022 12:32:39 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 222 files. Enable logging at DEBUG level to see which files will be staged.
May 09, 2022 12:32:40 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
May 09, 2022 12:32:40 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
May 09, 2022 12:32:43 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 222 files from PipelineOptions.filesToStage to staging location to prepare for execution.
May 09, 2022 12:32:44 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 222 files cached, 0 files newly uploaded in 0 seconds
May 09, 2022 12:32:44 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
May 09, 2022 12:32:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <120277 bytes, hash 703fa7d7284d03a6724f18a00f996b599f49c2d0600f6d8db28805bcee16d47a> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-cD-n1yhNA6ZyTxigD5lrWZ9JwtBgD22NsogFvO4W1Ho.pb
May 09, 2022 12:32:45 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
May 09, 2022 12:32:45 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@751ae8a4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@235d659c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4232b34a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2da16263, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@f5ce0bb, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@47e51549, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@101a461c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@360e9c06, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5ebffb44, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@311ff287, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7377781e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@31db34da, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@109f8c7e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5edacf20, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@16a5eb6d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7e307087, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1220ef43, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1a8b81e8, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@234cff57, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5e8507f1]
May 09, 2022 12:32:45 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
May 09, 2022 12:32:45 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
May 09, 2022 12:32:45 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
May 09, 2022 12:32:46 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
May 09, 2022 12:32:46 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3bf54172, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2c9a6717, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b3cde6f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6d091cad, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7c663eaf, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3bb5ceb, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1e692555, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ba0ae41, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@76fe6cdc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2ffb3aec, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@786ff1cb, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@46039a21, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@431e86b1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@35c4e864, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@32a2a6be, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@682af059, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5f36c8e3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4da39ca9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6a9344f5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5584d9c6]
May 09, 2022 12:32:46 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
May 09, 2022 12:32:46 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
May 09, 2022 12:32:46 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
May 09, 2022 12:32:46 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
May 09, 2022 12:32:46 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
May 09, 2022 12:32:46 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
May 09, 2022 12:32:46 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
May 09, 2022 12:32:46 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
May 09, 2022 12:32:46 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
May 09, 2022 12:32:46 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
May 09, 2022 12:32:46 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
May 09, 2022 12:32:46 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.40.0-SNAPSHOT
May 09, 2022 12:32:46 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-05-09_05_32_46-14665142041617253802?project=apache-beam-testing
May 09, 2022 12:32:46 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2022-05-09_05_32_46-14665142041617253802
May 09, 2022 12:32:46 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2022-05-09_05_32_46-14665142041617253802
May 09, 2022 12:32:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2022-05-09T12:32:51.088Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java170dataflow0v20streaming0cogbk01-jenkins-05-ithr. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
May 09, 2022 12:32:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-09T12:32:56.474Z: Worker configuration: e2-standard-2 in us-central1-b.
May 09, 2022 12:33:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-09T12:32:57.352Z: Expanding SplittableParDo operations into optimizable parts.
May 09, 2022 12:33:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-09T12:32:57.408Z: Expanding CollectionToSingleton operations into optimizable parts.
May 09, 2022 12:33:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-09T12:32:57.481Z: Expanding CoGroupByKey operations into optimizable parts.
May 09, 2022 12:33:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-09T12:32:57.573Z: Expanding SplittableProcessKeyed operations into optimizable parts.
May 09, 2022 12:33:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-09T12:32:57.613Z: Expanding GroupByKey operations into streaming Read/Write steps
May 09, 2022 12:33:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-09T12:32:57.689Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 09, 2022 12:33:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-09T12:32:57.803Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 09, 2022 12:33:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-09T12:32:57.838Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
May 09, 2022 12:33:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-09T12:32:57.881Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
May 09, 2022 12:33:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-09T12:32:57.914Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
May 09, 2022 12:33:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-09T12:32:57.954Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
May 09, 2022 12:33:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-09T12:32:57.985Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
May 09, 2022 12:33:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-09T12:32:58.023Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
May 09, 2022 12:33:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-09T12:32:58.058Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
May 09, 2022 12:33:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-09T12:32:58.093Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
May 09, 2022 12:33:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-09T12:32:58.124Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
May 09, 2022 12:33:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-09T12:32:58.158Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
May 09, 2022 12:33:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-09T12:32:58.194Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
May 09, 2022 12:33:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-09T12:32:58.227Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
May 09, 2022 12:33:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-09T12:32:58.251Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
May 09, 2022 12:33:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-09T12:32:58.284Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
May 09, 2022 12:33:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-09T12:32:58.319Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
May 09, 2022 12:33:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-09T12:32:58.353Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
May 09, 2022 12:33:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-09T12:32:58.400Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
May 09, 2022 12:33:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-09T12:32:58.429Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
May 09, 2022 12:33:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-09T12:32:58.456Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
May 09, 2022 12:33:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-09T12:32:58.492Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
May 09, 2022 12:33:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-09T12:32:58.525Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
May 09, 2022 12:33:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-09T12:32:58.575Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
May 09, 2022 12:33:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-09T12:32:58.758Z: Running job using Streaming Engine
May 09, 2022 12:33:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-09T12:32:58.982Z: Starting 5 ****s in us-central1-b...
May 09, 2022 12:33:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-09T12:33:21.818Z: Autoscaling: Raised the number of ****s to 4 so that the pipeline can catch up with its backlog and keep up with its input rate.
May 09, 2022 12:33:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-09T12:33:21.846Z: Resized **** pool to 4, though goal was 5. This could be a quota issue.
May 09, 2022 12:33:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-09T12:33:29.180Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 09, 2022 12:33:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-09T12:33:32.071Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
May 09, 2022 12:34:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-09T12:34:33.040Z: Workers have started successfully.
May 09, 2022 4:01:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-09T16:01:00.503Z: Cancel request is committed for workflow job: 2022-05-09_05_32_46-14665142041617253802.
May 09, 2022 4:01:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-09T16:01:00.620Z: Cleaning up.
May 09, 2022 4:01:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-09T16:01:00.718Z: Stopping **** pool...
May 09, 2022 4:01:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-09T16:01:00.768Z: Stopping **** pool...
May 09, 2022 4:01:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-09T16:01:32.713Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
May 09, 2022 4:01:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-09T16:01:32.750Z: Worker pool stopped.
May 09, 2022 4:01:41 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2022-05-09_05_32_46-14665142041617253802 finished with status CANCELLED.
Load test results for test (ID): a6fbb7f6-1e3f-46ae-9feb-1181e26d3207 and timestamp: 2022-05-09T12:32:40.038000000Z:
Metric: Value:
dataflow_v2_java17_runtime_sec 12338.085
dataflow_v2_java17_total_bytes_count 2.86964364E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)
> Task :sdks:java:testing:load-tests:run FAILED
> Task :runners:google-cloud-dataflow-java:cleanUpDockerJavaImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220509123035
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ae661c08a40bbf8a47368c1570d2cbdb9dbb0865ef587bd35d60f43cbee2a87d
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220509123035]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ae661c08a40bbf8a47368c1570d2cbdb9dbb0865ef587bd35d60f43cbee2a87d]
Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220509123035] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ae661c08a40bbf8a47368c1570d2cbdb9dbb0865ef587bd35d60f43cbee2a87d])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ae661c08a40bbf8a47368c1570d2cbdb9dbb0865ef587bd35d60f43cbee2a87d
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ae661c08a40bbf8a47368c1570d2cbdb9dbb0865ef587bd35d60f43cbee2a87d
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ae661c08a40bbf8a47368c1570d2cbdb9dbb0865ef587bd35d60f43cbee2a87d].
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.
BUILD FAILED in 3h 31m 27s
110 actionable tasks: 73 executed, 33 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/5fdkmmdldigby
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17 #139
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17/139/display/redirect>
Changes:
------------------------------------------
[...truncated 50.84 KB...]
08fa02ce37eb: Preparing
a037458de4e0: Preparing
bafdbe68e4ae: Preparing
a13c519c6361: Preparing
fced9607bc47: Waiting
e91b08f16562: Waiting
713082863eee: Waiting
08e8a29c256a: Waiting
00d4a583dbe4: Waiting
875a60575306: Waiting
c92dc92e5b42: Waiting
721b8ef4b4ba: Waiting
b900d4df52f8: Waiting
5b26c8721c69: Waiting
3bc383470c05: Waiting
bafdbe68e4ae: Waiting
a13c519c6361: Waiting
e93827457889: Waiting
08fa02ce37eb: Waiting
a037458de4e0: Waiting
fbe1b45afd3e: Pushed
c8df78b28fbb: Pushed
e023246e7a9a: Pushed
d580cb531531: Pushed
de4ea1bb1e74: Pushed
875a60575306: Pushed
721b8ef4b4ba: Pushed
b900d4df52f8: Pushed
5b26c8721c69: Pushed
713082863eee: Pushed
e91b08f16562: Pushed
fced9607bc47: Pushed
e93827457889: Layer already exists
08fa02ce37eb: Layer already exists
3bc383470c05: Layer already exists
a037458de4e0: Layer already exists
bafdbe68e4ae: Layer already exists
a13c519c6361: Layer already exists
00d4a583dbe4: Pushed
c92dc92e5b42: Pushed
08e8a29c256a: Pushed
20220508123043: digest: sha256:3e2bc6f9b92f30686d7581e866cc3457a9e65cb36f5ee4f62689dd7b0f578fb4 size: 4729
> Task :sdks:java:testing:load-tests:run
May 08, 2022 12:33:00 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
May 08, 2022 12:33:01 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 222 files. Enable logging at DEBUG level to see which files will be staged.
May 08, 2022 12:33:01 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
May 08, 2022 12:33:01 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
May 08, 2022 12:33:05 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 222 files from PipelineOptions.filesToStage to staging location to prepare for execution.
May 08, 2022 12:33:06 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 222 files cached, 0 files newly uploaded in 0 seconds
May 08, 2022 12:33:06 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
May 08, 2022 12:33:06 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <120277 bytes, hash 6c4231b8b37219127e2cbe678df05f694b012a4e308249a674ec8a16db5c1949> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-bEIxuLNyGRJ-LL5njfBfaUsBKk4wgkmmdOyKFttcGUk.pb
May 08, 2022 12:33:08 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
May 08, 2022 12:33:08 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@751ae8a4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@235d659c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4232b34a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2da16263, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@f5ce0bb, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@47e51549, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@101a461c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@360e9c06, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5ebffb44, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@311ff287, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7377781e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@31db34da, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@109f8c7e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5edacf20, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@16a5eb6d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7e307087, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1220ef43, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1a8b81e8, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@234cff57, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5e8507f1]
May 08, 2022 12:33:08 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
May 08, 2022 12:33:08 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
May 08, 2022 12:33:08 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
May 08, 2022 12:33:08 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
May 08, 2022 12:33:08 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3bf54172, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2c9a6717, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b3cde6f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6d091cad, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7c663eaf, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3bb5ceb, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1e692555, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ba0ae41, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@76fe6cdc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2ffb3aec, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@786ff1cb, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@46039a21, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@431e86b1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@35c4e864, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@32a2a6be, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@682af059, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5f36c8e3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4da39ca9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6a9344f5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5584d9c6]
May 08, 2022 12:33:08 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
May 08, 2022 12:33:08 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
May 08, 2022 12:33:08 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
May 08, 2022 12:33:08 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
May 08, 2022 12:33:08 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
May 08, 2022 12:33:08 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
May 08, 2022 12:33:08 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
May 08, 2022 12:33:08 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
May 08, 2022 12:33:08 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
May 08, 2022 12:33:08 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
May 08, 2022 12:33:08 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
May 08, 2022 12:33:08 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.40.0-SNAPSHOT
May 08, 2022 12:33:09 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-05-08_05_33_08-12380385818833013014?project=apache-beam-testing
May 08, 2022 12:33:09 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2022-05-08_05_33_08-12380385818833013014
May 08, 2022 12:33:09 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2022-05-08_05_33_08-12380385818833013014
May 08, 2022 12:33:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2022-05-08T12:33:14.423Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java170dataflow0v20streaming0cogbk01-jenkins-05-v0tj. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
May 08, 2022 12:33:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-08T12:33:25.065Z: Worker configuration: e2-standard-2 in us-central1-b.
May 08, 2022 12:33:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-08T12:33:25.642Z: Expanding SplittableParDo operations into optimizable parts.
May 08, 2022 12:33:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-08T12:33:25.737Z: Expanding CollectionToSingleton operations into optimizable parts.
May 08, 2022 12:33:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-08T12:33:25.818Z: Expanding CoGroupByKey operations into optimizable parts.
May 08, 2022 12:33:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-08T12:33:25.888Z: Expanding SplittableProcessKeyed operations into optimizable parts.
May 08, 2022 12:33:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-08T12:33:25.917Z: Expanding GroupByKey operations into streaming Read/Write steps
May 08, 2022 12:33:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-08T12:33:25.983Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 08, 2022 12:33:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-08T12:33:26.099Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 08, 2022 12:33:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-08T12:33:26.130Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
May 08, 2022 12:33:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-08T12:33:26.164Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
May 08, 2022 12:33:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-08T12:33:26.199Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
May 08, 2022 12:33:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-08T12:33:26.230Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
May 08, 2022 12:33:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-08T12:33:26.270Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
May 08, 2022 12:33:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-08T12:33:26.306Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
May 08, 2022 12:33:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-08T12:33:26.340Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
May 08, 2022 12:33:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-08T12:33:26.371Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
May 08, 2022 12:33:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-08T12:33:26.410Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
May 08, 2022 12:33:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-08T12:33:26.443Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
May 08, 2022 12:33:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-08T12:33:26.476Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
May 08, 2022 12:33:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-08T12:33:26.509Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
May 08, 2022 12:33:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-08T12:33:26.542Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
May 08, 2022 12:33:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-08T12:33:26.568Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
May 08, 2022 12:33:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-08T12:33:26.605Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
May 08, 2022 12:33:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-08T12:33:26.639Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
May 08, 2022 12:33:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-08T12:33:26.736Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
May 08, 2022 12:33:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-08T12:33:26.771Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
May 08, 2022 12:33:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-08T12:33:26.802Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
May 08, 2022 12:33:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-08T12:33:26.836Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
May 08, 2022 12:33:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-08T12:33:26.872Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
May 08, 2022 12:33:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-08T12:33:26.897Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
May 08, 2022 12:33:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-08T12:33:27.139Z: Running job using Streaming Engine
May 08, 2022 12:33:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-08T12:33:27.534Z: Starting 5 ****s in us-central1-b...
May 08, 2022 12:33:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-08T12:33:41.960Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 08, 2022 12:33:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-08T12:33:49.913Z: Autoscaling: Raised the number of ****s to 4 so that the pipeline can catch up with its backlog and keep up with its input rate.
May 08, 2022 12:33:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-08T12:33:49.939Z: Resized **** pool to 4, though goal was 5. This could be a quota issue.
May 08, 2022 12:34:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-08T12:34:00.147Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
May 08, 2022 12:34:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-08T12:34:57.660Z: Workers have started successfully.
May 08, 2022 4:01:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-08T16:01:12.049Z: Cancel request is committed for workflow job: 2022-05-08_05_33_08-12380385818833013014.
May 08, 2022 4:01:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-08T16:01:12.227Z: Cleaning up.
May 08, 2022 4:01:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-08T16:01:12.337Z: Stopping **** pool...
May 08, 2022 4:01:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-08T16:01:12.380Z: Stopping **** pool...
May 08, 2022 4:01:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-08T16:01:45.060Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
May 08, 2022 4:01:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-08T16:01:45.128Z: Worker pool stopped.
May 08, 2022 4:01:52 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2022-05-08_05_33_08-12380385818833013014 finished with status CANCELLED.
Load test results for test (ID): 6898ae29-0c8e-440a-aed1-3e8098cdf3b0 and timestamp: 2022-05-08T12:33:01.540000000Z:
Metric: Value:
dataflow_v2_java17_runtime_sec 12302.848
dataflow_v2_java17_total_bytes_count 2.44242771E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)
> Task :sdks:java:testing:load-tests:run FAILED
> Task :runners:google-cloud-dataflow-java:cleanUpDockerJavaImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220508123043
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:3e2bc6f9b92f30686d7581e866cc3457a9e65cb36f5ee4f62689dd7b0f578fb4
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220508123043]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:3e2bc6f9b92f30686d7581e866cc3457a9e65cb36f5ee4f62689dd7b0f578fb4]
Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220508123043] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:3e2bc6f9b92f30686d7581e866cc3457a9e65cb36f5ee4f62689dd7b0f578fb4])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:3e2bc6f9b92f30686d7581e866cc3457a9e65cb36f5ee4f62689dd7b0f578fb4
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:3e2bc6f9b92f30686d7581e866cc3457a9e65cb36f5ee4f62689dd7b0f578fb4
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:3e2bc6f9b92f30686d7581e866cc3457a9e65cb36f5ee4f62689dd7b0f578fb4].
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.
BUILD FAILED in 3h 31m 31s
110 actionable tasks: 73 executed, 33 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/a3nj7pq4szqmc
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17 #138
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17/138/display/redirect?page=changes>
Changes:
[kevinsijo] Setting up a basic directory
[kevinsijo] Mirroring Python SDK's directory structure
[kerrydc] Adds initial tests
[kevinsijo] 'runners' is the correct directory name
[Pablo Estrada] sketching the core API for JS SDK
[jonathanlui] add .gitignore for node/ts project
[Robert Bradshaw] Worker directory.
[Robert Bradshaw] Fix complile errors with explicit any for callables.
[Robert Bradshaw] Add worker entry point.
[Robert Bradshaw] Add proto generation code.
[Robert Bradshaw] Add generated proto files.
[Robert Bradshaw] Attempts to get ts protos to compile.
[Robert Bradshaw] Exclude ts protos for now.
[Robert Bradshaw] More changes to get ts protos working.
[Robert Bradshaw] Update scripts and config to get protos compiling.
[Robert Bradshaw] Update geenrated files.
[jonathanlui] add build and clean script to compile ts
[Robert Bradshaw] Generate server for loopback worker.
[Robert Bradshaw] Generated grpc servers for loopback.
[Robert Bradshaw] Add typescript formatter.
[Robert Bradshaw] Loopback server (that does nothing).
[Robert Bradshaw] Working server.
[Pablo Estrada] Starting expansion of primitive transforms
[Pablo Estrada] Starting to implement and support standard coders
[Robert Bradshaw] Also generate grpc clients.
[Robert Bradshaw] Basic implementation of worker harness.
[Pablo Estrada] fix the build
[Robert Bradshaw] Add some missing files for worker harness.
[Robert Bradshaw] Refactor operators to use registration.
[jonathanlui] enable ts in mocha
[jonathanlui] update readme
[jonathanlui] --save-dev @types/mocha
[jonathanlui] translate core_test.js to typescript
[Robert Bradshaw] Encapsulate worker service in a class.
[Kenneth Knowles] Port standard_coders_test to typescript (superficially)
[Pablo Estrada] Starting the proto translation of Impulse, ParDo, GBK
[Robert Bradshaw] Add some tests for the worker code.
[Robert Bradshaw] Fixing old lock file error.
[Pablo Estrada] Adding transform names and fixing GBK coder issue
[Robert Bradshaw] npx tsfmt -r src/apache_beam/base.ts src/apache_beam/transforms/core.ts
[Kenneth Knowles] switch to import style require() statements
[Kenneth Knowles] Add Coder interface using protobufjs classes
[Kenneth Knowles] BytesCoder with some failures
[noreply] Added GeneralObjectCoder and using it as coder for most transforms (#9)
[Kenneth Knowles] Fix order of arguments to deepEqual
[Kenneth Knowles] Encode expected encoding as binary
[Robert Bradshaw] Refactor API to allow for composites.
[jrmccluskey] Initial setup for automated Java expansion startup
[jrmccluskey] Update exp_service.ts
[Kenneth Knowles] Fix up coder deserialization
[Robert Bradshaw] Simplify GBK coder computation.
[Robert Bradshaw] Remove top-level PValue.
[Pablo Estrada] Make tests green
[Robert Bradshaw] Rename PValueish to PValue.
[jonathanlui] node runner
[jonathanlui] whitespaces
[Robert Bradshaw] Make Runner.run async.
[jonathanlui] bson and fast-deep-equal should not be listed as devdependency
[jrmccluskey] Add basic Dockerfile that starts ExternalWorkerPool
[Robert Bradshaw] Direct runner.
[kevinsijo] Testing expansion service communication
[Robert Bradshaw] Added flatten, assertion checkers.
[Pablo Estrada] progress on basic coders
[Robert Bradshaw] Fixing the build.
[Robert Bradshaw] Cleanup, simplify access.
[Pablo Estrada] Adding limited support for KVCoder and IterableCoder
[Robert Bradshaw] Introduce PipelineContext.
[Robert Bradshaw] Add toProto to all coders.
[Robert Bradshaw] Some work with coders.
[Robert Bradshaw] Remove debug logging.
[Robert Bradshaw] Use coders over data channel.
[Kenneth Knowles] explicitly sequence sub-coder serializations
[Kenneth Knowles] no more need to extend FakeCoder
[Kenneth Knowles] actually advance reader
[Kenneth Knowles] autoformat
[Kenneth Knowles] protobufjs already can write and read signed varints
[Kenneth Knowles] with improved test harness, kv has many more failures
[Kenneth Knowles] read bytescoder from correct position
[Kenneth Knowles] no more fake coders
[Kenneth Knowles] varint examples all work
[Kenneth Knowles] simplify coder value parsing
[Kenneth Knowles] global window coder
[Kenneth Knowles] fix swapEndian32
[Robert Bradshaw] Add P(...) operator.
[kevinsijo] Implementing RowCoder encoding.
[jrmccluskey] remove unused container dir
[kevinsijo] Corrected sorting of encoded positions to reflect an argsort instead.
[Robert Bradshaw] Populate environments.
[kevinsijo] Implementing RowCoder decoding.
[Kenneth Knowles] preliminary unbounded iterable coder
[Kenneth Knowles] friendlier description of standard coder test case
[Kenneth Knowles] fix test harness; iterable works
[jrmccluskey] first pass at boot.go
[jonathanlui] update package-lock.json
[jonathanlui] make NodeRunner a subclass of Runner
[jonathanlui] add waitUntilFinish interface member
[Pablo Estrada] Adding double coder
[Kenneth Knowles] scaffolding for windowed values
[Pablo Estrada] Adding type information to PColleciton and PTransform
[jonathanlui] fix direct runner
[Pablo Estrada] Adding typing information for DoFns
[Kenneth Knowles] add interval window
[Robert Bradshaw] Export PValue.
[Robert Bradshaw] Add CombineFn interface.
[Robert Bradshaw] Typed flatten.
[jonathanlui] add runAsync method to base.Runner
[Kenneth Knowles] add Long package
[Pablo Estrada] Adding more types. Making PValue typed
[Kenneth Knowles] instant coder draft
[Robert Bradshaw] Return job state from direct runner.
[Kenneth Knowles] type instant = long
[jonathanlui] implement NodeRunner.runPipeline
[Kenneth Knowles] autoformat
[kevinsijo] Completed implementation of basic row coder
[Kenneth Knowles] Fix IntervalWindowCoder, almost
[Kenneth Knowles] fix interval window coder
[Kenneth Knowles] autoformat
[Robert Bradshaw] loopback runner works
[Kenneth Knowles] move core element types into values.ts
[Kenneth Knowles] just build object directly to be cool
[Robert Bradshaw] GBK working on ULR.
[Robert Bradshaw] Async transforms.
[Robert Bradshaw] External transform grpah splicing.
[Kenneth Knowles] progress on windowed value: paneinfo encoding
[Robert Bradshaw] Fix merge.
[Robert Bradshaw] autoformat
[Kenneth Knowles] full windowed value coder
[kerrydc] Updates tests to use correct types, adds generics where needed to DoFns
[Robert Bradshaw] Add serialization librarires.'
[Robert Bradshaw] Add Split() PTransform, for producing multiple outputs from a single
[Robert Bradshaw] Schema-encoded external payloads.
[kevinsijo] Adding Schema inference from JSON
[Pablo Estrada] Removing unused directories
[Pablo Estrada] Support for finishBundle and improving typing annotations.
[Pablo Estrada] A base implementation of combiners with GBK/ParDo
[Robert Bradshaw] Fully propagate windowing information in both remote and direct runner.
[Robert Bradshaw] Make args and kwargs optional for python external transform.
[Robert Bradshaw] Infer schema for external transforms.
[Pablo Estrada] Implementing a custom combine fn as an example. Small fixes
[Robert Bradshaw] Fix missing windowing information in combiners.
[Robert Bradshaw] PostShuffle needn't group by key as that's already done.
[Robert Bradshaw] Guard pre-combine for global window only.
[Robert Bradshaw] WindowInto
[Robert Bradshaw] Fix optional kwargs.
[Robert Bradshaw] A couple of tweaks for js + py
[Robert Bradshaw] Add windowing file.
[Robert Bradshaw] CombineBy transform, stand-alone WordCount.
[Robert Bradshaw] cleanup
[Robert Bradshaw] Actually fix optional external kwargs.
[Robert Bradshaw] Demo2, textio read.
[Robert Bradshaw] Add command lines for starting up the servers.
[Robert Bradshaw] Run prettier on the full codebase.
[Robert Bradshaw] Update deps.
[Pablo Estrada] Adding docstrings for core.ts. Prettier dependency
[Pablo Estrada] Documenting coder interfaces
[Pablo Estrada] Added documentation for a few standard coders
[Robert Bradshaw] Unified grouping and combining.
[Robert Bradshaw] Allow PCollection ids to be lazy.
[Robert Bradshaw] Reorganize module structure.
[Robert Bradshaw] A couple more renames.
[Robert Bradshaw] Simplify.
[Robert Bradshaw] Consolidation.
[Robert Bradshaw] Fix build.
[Robert Bradshaw] Add optional context to ParDo.
[Robert Bradshaw] fixup: iterable coder endian sign issue
[Robert Bradshaw] omit context for map(console.log)
[Robert Bradshaw] Fix ReadFromText coders.
[Robert Bradshaw] Flesh out README with overview and current state.
[noreply] Readme typo
[Robert Bradshaw] Two more TODOs.
[noreply] Add a pointer to the example wordcount to the readme.
[Pablo Estrada] Documenting coders and implementing unknown-length method
[Robert Bradshaw] UIID dependency.
[Robert Bradshaw] Artifact handling.
[Robert Bradshaw] Properly wait on data channel for bundle completion.
[Robert Bradshaw] Automatic java expansion service startup.
[Robert Bradshaw] Process promises.
[Robert Bradshaw] Implement side inputs.
[Robert Bradshaw] Cleanup.
[Robert Bradshaw] Put complex constext stuff in its own file.
[Robert Bradshaw] Rename BoundedWindow to just Window.
[Robert Bradshaw] Alternative splitter class.
[Pablo Estrada] Documenting internal functions
[Robert Bradshaw] Take a pass clarifying the TODOs.
[Robert Bradshaw] Sql transform wrapper.
[Robert Bradshaw] Incorporate some feedback into the TODOs.
[Robert Bradshaw] More TODOs.
[Robert Bradshaw] Remove app placeholder.
[Robert Bradshaw] Apache license headers.
[Robert Bradshaw] More TODOs
[jankuehle] Suggestions for TypeScript todos
[dannymccormick] Add actions for typescript sdk
[dannymccormick] Fix test command
[noreply] Add missing version
[dannymccormick] Fix codecovTest command
[noreply] Only do prettier check on linux
[noreply] Only get codecov on linux
[Robert Bradshaw] Resolve some comments.
[Robert Bradshaw] Fix compile errors.
[Robert Bradshaw] Prettier.
[Robert Bradshaw] Re-order expandInternal arguments pending unification.
[Robert Bradshaw] More consistent and stricter PTransform naming.
[Robert Bradshaw] Notes on explicit, if less idiomatic, use of classes.
[Robert Bradshaw] Let DoFn be an interface rather than a class.
[Robert Bradshaw] Provide DoFn context to start and finish bundle.
[Robert Bradshaw] Optional promise code simplification.
[Robert Bradshaw] Cleanup todos.
[Robert Bradshaw] Avoid any type where not needed.
[Robert Bradshaw] Apache RAT excludes for typescript.
[Robert Bradshaw] Remove empty READMEs.
[Robert Bradshaw] Add licences statement to readme files.
[Robert Bradshaw] More RAT fixes.
[Robert Bradshaw] Another unsupported coder.
[Robert Bradshaw] Remove debugging code.
[noreply] Fix automatic naming with code coverage.
[Robert Bradshaw] Coders cleanup.
[Robert Bradshaw] Add tests for RowCoder.
[Robert Bradshaw] Normalize capitalization, comments.
[Robert Bradshaw] Install typescript closure packages.
[Robert Bradshaw] npm audit fix
[Robert Bradshaw] Move more imports out of base.
[Robert Bradshaw] Changes needed to compile with ts closure plugin.
[Robert Bradshaw] Use ttsc and ts-closure-transform plugin.
[Robert Bradshaw] Serialization registration to actually get serialization working.
[Robert Bradshaw] Container images working on local runner.
[Robert Bradshaw] Add a portable job server that proxies the Dataflow backend. (#17189)
[Robert Bradshaw] Improvements to dataflow job service for non-Python jobs.
[Robert Bradshaw] Get dataflow working.
[Robert Bradshaw] User friendly pipeline options.
[Robert Bradshaw] Less classes, more functions.
[Robert Bradshaw] Add new nullable standard coder.
[Robert Bradshaw] Make Apache Rat happy.
[Robert Bradshaw] Disable broken codecov.
[Robert Bradshaw] Remove last uses of base.ts.
[Robert Bradshaw] Remove unneedd file.
[Robert Bradshaw] Remove more uneeded/unused files.
[Robert Bradshaw] Cleanup tests.
[Robert Bradshaw] Minor cleanups to coder tests.
[noreply] Quote pip install package name
[noreply] [BEAM-14374] Fix module import error in FullyQualifiedNamedTransform
[Robert Bradshaw] Addressing issues from the review.
[noreply] Apply suggestions from code review.
[Robert Bradshaw] Post-merge fixes.
[dannymccormick] Delete tags.go
[Robert Bradshaw] Update tests to use our actual serialization libraries.
[Robert Bradshaw] Another pass at TODOs, removing finished items.
[Heejong Lee] [BEAM-14146] Python Streaming job failing to drain with BigQueryIO write
[Kenneth Knowles] Add parameter for service account impersonation in GCP credentials
[Heejong Lee] add test
[noreply] Merge pull request #17490 from [BEAM-14370] [Website] Add new page about
[noreply] [BEAM-14332] Refactored cluster management for Flink on Dataproc
[noreply] [BEAM-13988] Update mtime to use time.UnixMilli() calls (#17578)
[noreply] Fixing patching error on missing dependencies (#17564)
[noreply] Merge pull request #17517 from [BEAM-14383] Improve "FailedRows" errors
[Heejong Lee] add test without mock
------------------------------------------
[...truncated 50.18 KB...]
2d05db1e2a4b: Preparing
3bc383470c05: Preparing
e93827457889: Preparing
08fa02ce37eb: Preparing
a037458de4e0: Preparing
662540b7db15: Waiting
1b1b55724774: Waiting
bafdbe68e4ae: Preparing
0e2e00df1023: Waiting
d507830c9b22: Waiting
a13c519c6361: Preparing
08fa02ce37eb: Waiting
82227587df8f: Waiting
e93827457889: Waiting
3bc383470c05: Waiting
bafdbe68e4ae: Waiting
a13c519c6361: Waiting
1252636fdc5b: Waiting
b0bdfb1e5c67: Waiting
8db9d37c9268: Waiting
2d05db1e2a4b: Waiting
a037458de4e0: Waiting
413656da2079: Pushed
98faf80f3452: Pushed
94899fdeb521: Pushed
3c9402bf3f9b: Pushed
1e44127ca56a: Pushed
662540b7db15: Pushed
104aa3e0eab7: Pushed
82227587df8f: Pushed
d507830c9b22: Pushed
1252636fdc5b: Pushed
1b1b55724774: Pushed
3bc383470c05: Layer already exists
e93827457889: Layer already exists
b0bdfb1e5c67: Pushed
08fa02ce37eb: Layer already exists
a037458de4e0: Layer already exists
bafdbe68e4ae: Layer already exists
a13c519c6361: Layer already exists
8db9d37c9268: Pushed
2d05db1e2a4b: Pushed
0e2e00df1023: Pushed
20220507123039: digest: sha256:a201febb4501343aed27f34781031d37aebfe84ff97646ce1d23d15df7f7d15a size: 4729
> Task :sdks:java:testing:load-tests:run
May 07, 2022 12:32:44 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
May 07, 2022 12:32:44 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 222 files. Enable logging at DEBUG level to see which files will be staged.
May 07, 2022 12:32:45 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
May 07, 2022 12:32:45 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
May 07, 2022 12:32:48 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 222 files from PipelineOptions.filesToStage to staging location to prepare for execution.
May 07, 2022 12:32:48 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 222 files cached, 0 files newly uploaded in 0 seconds
May 07, 2022 12:32:48 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
May 07, 2022 12:32:48 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <120277 bytes, hash a6ab5cbe90343949e56d702d9a136db2fa719819f168bdcc66299093c41072f1> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-pqtcvpA0OUnlbXAtmhNtsvpxmBnxaL3MZimQk8QQcvE.pb
May 07, 2022 12:32:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
May 07, 2022 12:32:50 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@751ae8a4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@235d659c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4232b34a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2da16263, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@f5ce0bb, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@47e51549, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@101a461c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@360e9c06, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5ebffb44, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@311ff287, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7377781e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@31db34da, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@109f8c7e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5edacf20, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@16a5eb6d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7e307087, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1220ef43, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1a8b81e8, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@234cff57, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5e8507f1]
May 07, 2022 12:32:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
May 07, 2022 12:32:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
May 07, 2022 12:32:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
May 07, 2022 12:32:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
May 07, 2022 12:32:50 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3bf54172, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2c9a6717, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b3cde6f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6d091cad, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7c663eaf, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3bb5ceb, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1e692555, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ba0ae41, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@76fe6cdc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2ffb3aec, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@786ff1cb, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@46039a21, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@431e86b1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@35c4e864, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@32a2a6be, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@682af059, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5f36c8e3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4da39ca9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6a9344f5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5584d9c6]
May 07, 2022 12:32:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
May 07, 2022 12:32:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
May 07, 2022 12:32:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
May 07, 2022 12:32:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
May 07, 2022 12:32:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
May 07, 2022 12:32:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
May 07, 2022 12:32:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
May 07, 2022 12:32:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
May 07, 2022 12:32:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
May 07, 2022 12:32:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
May 07, 2022 12:32:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
May 07, 2022 12:32:50 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.40.0-SNAPSHOT
May 07, 2022 12:32:51 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-05-07_05_32_50-18293218377134002814?project=apache-beam-testing
May 07, 2022 12:32:51 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2022-05-07_05_32_50-18293218377134002814
May 07, 2022 12:32:51 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2022-05-07_05_32_50-18293218377134002814
May 07, 2022 12:32:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2022-05-07T12:32:54.377Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java170dataflow0v20streaming0cogbk01-jenkins-05-e677. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
May 07, 2022 12:33:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-07T12:33:03.852Z: Worker configuration: e2-standard-2 in us-central1-b.
May 07, 2022 12:33:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-07T12:33:04.691Z: Expanding SplittableParDo operations into optimizable parts.
May 07, 2022 12:33:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-07T12:33:04.716Z: Expanding CollectionToSingleton operations into optimizable parts.
May 07, 2022 12:33:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-07T12:33:04.786Z: Expanding CoGroupByKey operations into optimizable parts.
May 07, 2022 12:33:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-07T12:33:04.935Z: Expanding SplittableProcessKeyed operations into optimizable parts.
May 07, 2022 12:33:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-07T12:33:05.134Z: Expanding GroupByKey operations into streaming Read/Write steps
May 07, 2022 12:33:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-07T12:33:05.412Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 07, 2022 12:33:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-07T12:33:05.501Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 07, 2022 12:33:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-07T12:33:05.614Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 07, 2022 12:33:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-07T12:33:05.650Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
May 07, 2022 12:33:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-07T12:33:05.687Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
May 07, 2022 12:33:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-07T12:33:05.719Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
May 07, 2022 12:33:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-07T12:33:05.753Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
May 07, 2022 12:33:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-07T12:33:05.786Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
May 07, 2022 12:33:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-07T12:33:05.820Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
May 07, 2022 12:33:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-07T12:33:05.851Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
May 07, 2022 12:33:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-07T12:33:05.884Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
May 07, 2022 12:33:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-07T12:33:05.917Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
May 07, 2022 12:33:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-07T12:33:05.942Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
May 07, 2022 12:33:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-07T12:33:05.975Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
May 07, 2022 12:33:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-07T12:33:06.008Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
May 07, 2022 12:33:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-07T12:33:06.041Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
May 07, 2022 12:33:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-07T12:33:06.101Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
May 07, 2022 12:33:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-07T12:33:06.123Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
May 07, 2022 12:33:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-07T12:33:06.166Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
May 07, 2022 12:33:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-07T12:33:06.199Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
May 07, 2022 12:33:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-07T12:33:06.232Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
May 07, 2022 12:33:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-07T12:33:06.266Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
May 07, 2022 12:33:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-07T12:33:06.300Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
May 07, 2022 12:33:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-07T12:33:06.333Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
May 07, 2022 12:33:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-07T12:33:06.365Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
May 07, 2022 12:33:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-07T12:33:06.562Z: Running job using Streaming Engine
May 07, 2022 12:33:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-07T12:33:06.802Z: Starting 5 ****s in us-central1-b...
May 07, 2022 12:33:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-07T12:33:32.472Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
May 07, 2022 12:34:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-07T12:34:39.607Z: Workers have started successfully.
May 07, 2022 1:13:26 PM org.apache.beam.sdk.metrics.MetricsEnvironment getCurrentContainer
WARNING: Reporting metrics are not supported in the current execution environment.
May 07, 2022 4:01:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-07T16:00:58.218Z: Cancel request is committed for workflow job: 2022-05-07_05_32_50-18293218377134002814.
May 07, 2022 4:01:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-07T16:00:59.490Z: Cleaning up.
May 07, 2022 4:01:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-07T16:00:59.579Z: Stopping **** pool...
May 07, 2022 4:01:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-07T16:00:59.623Z: Stopping **** pool...
May 07, 2022 4:01:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-07T16:01:35.077Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
May 07, 2022 4:01:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-07T16:01:35.117Z: Worker pool stopped.
May 07, 2022 4:01:45 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2022-05-07_05_32_50-18293218377134002814 finished with status CANCELLED.
Load test results for test (ID): d380543f-25c8-4015-8b36-36a9fa03eb93 and timestamp: 2022-05-07T12:32:45.055000000Z:
Metric: Value:
dataflow_v2_java17_runtime_sec 12349.746
dataflow_v2_java17_total_bytes_count 3.37999092E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)
> Task :sdks:java:testing:load-tests:run FAILED
> Task :runners:google-cloud-dataflow-java:cleanUpDockerJavaImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220507123039
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:a201febb4501343aed27f34781031d37aebfe84ff97646ce1d23d15df7f7d15a
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220507123039]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:a201febb4501343aed27f34781031d37aebfe84ff97646ce1d23d15df7f7d15a]
Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220507123039] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:a201febb4501343aed27f34781031d37aebfe84ff97646ce1d23d15df7f7d15a])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:a201febb4501343aed27f34781031d37aebfe84ff97646ce1d23d15df7f7d15a
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:a201febb4501343aed27f34781031d37aebfe84ff97646ce1d23d15df7f7d15a
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:a201febb4501343aed27f34781031d37aebfe84ff97646ce1d23d15df7f7d15a].
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.
BUILD FAILED in 3h 31m 26s
110 actionable tasks: 73 executed, 33 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/icdmjakku6vew
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17 #137
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17/137/display/redirect?page=changes>
Changes:
[zyichi] Move master readme.md to 2.40.0
[noreply] [BEAM-14173] Fix Go Loadtests on Dataflow & partial fix for Flink
[noreply] Upgrade python sdk container requirements. (#17549)
[noreply] Merge pull request #17497: [BEAM-11205] Update GCP Libraries BOM version
[noreply] [BEAM-12603] Add retry on grpc data channel and remove retry from test.
[noreply] Merge pull request #17359: [BEAM-14303] Add a way to exclude output
[noreply] [BEAM-14347] Allow users to optimize DoFn execution with a single
[noreply] [BEAM-5878] Add (failing) kwonly-argument test (#17509)
------------------------------------------
[...truncated 47.03 KB...]
6736907aa128: Pushed
3bc383470c05: Layer already exists
e93827457889: Layer already exists
e0251c2cfd97: Pushed
08fa02ce37eb: Layer already exists
a037458de4e0: Layer already exists
bafdbe68e4ae: Layer already exists
a13c519c6361: Layer already exists
c21785f068bb: Pushed
feb9704976ce: Pushed
a73b38700550: Pushed
20220506123132: digest: sha256:df7a799483a8dfcf0e4964b95e5dc6ef838d5908578aa974f5608b4b2f79498f size: 4729
> Task :sdks:java:testing:load-tests:run
May 06, 2022 12:35:55 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
May 06, 2022 12:35:56 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 222 files. Enable logging at DEBUG level to see which files will be staged.
May 06, 2022 12:35:57 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
May 06, 2022 12:35:57 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
May 06, 2022 12:36:03 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 222 files from PipelineOptions.filesToStage to staging location to prepare for execution.
May 06, 2022 12:36:05 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 222 files cached, 0 files newly uploaded in 2 seconds
May 06, 2022 12:36:05 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
May 06, 2022 12:36:05 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <120277 bytes, hash 4c05446f1cc0b7cb866bcf3be6df3ca8b2912a2d7b39757dd38920d959660589> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-TAVEbxzAt8uGa8875t88qLKRKi17OXV904kg2VlmBYk.pb
May 06, 2022 12:36:08 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
May 06, 2022 12:36:08 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@751ae8a4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@235d659c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4232b34a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2da16263, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@f5ce0bb, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@47e51549, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@101a461c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@360e9c06, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5ebffb44, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@311ff287, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7377781e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@31db34da, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@109f8c7e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5edacf20, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@16a5eb6d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7e307087, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1220ef43, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1a8b81e8, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@234cff57, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5e8507f1]
May 06, 2022 12:36:08 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
May 06, 2022 12:36:08 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
May 06, 2022 12:36:08 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
May 06, 2022 12:36:08 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
May 06, 2022 12:36:08 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3b95a6db, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3bf54172, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2c9a6717, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b3cde6f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6d091cad, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7c663eaf, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3bb5ceb, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1e692555, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ba0ae41, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@76fe6cdc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2ffb3aec, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@786ff1cb, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@46039a21, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@431e86b1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@35c4e864, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@32a2a6be, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@682af059, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5f36c8e3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4da39ca9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6a9344f5]
May 06, 2022 12:36:08 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
May 06, 2022 12:36:08 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
May 06, 2022 12:36:08 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
May 06, 2022 12:36:08 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
May 06, 2022 12:36:08 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
May 06, 2022 12:36:08 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
May 06, 2022 12:36:08 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
May 06, 2022 12:36:08 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
May 06, 2022 12:36:08 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
May 06, 2022 12:36:08 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
May 06, 2022 12:36:08 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
May 06, 2022 12:36:08 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.40.0-SNAPSHOT
May 06, 2022 12:36:09 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-05-06_05_36_09-16672906884944557169?project=apache-beam-testing
May 06, 2022 12:36:09 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2022-05-06_05_36_09-16672906884944557169
May 06, 2022 12:36:09 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2022-05-06_05_36_09-16672906884944557169
May 06, 2022 12:36:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2022-05-06T12:36:16.319Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java170dataflow0v20streaming0cogbk01-jenkins-05-eke9. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
May 06, 2022 12:36:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-06T12:36:22.945Z: Worker configuration: e2-standard-2 in us-central1-b.
May 06, 2022 12:36:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-06T12:36:23.746Z: Expanding SplittableParDo operations into optimizable parts.
May 06, 2022 12:36:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-06T12:36:23.775Z: Expanding CollectionToSingleton operations into optimizable parts.
May 06, 2022 12:36:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-06T12:36:23.839Z: Expanding CoGroupByKey operations into optimizable parts.
May 06, 2022 12:36:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-06T12:36:23.900Z: Expanding SplittableProcessKeyed operations into optimizable parts.
May 06, 2022 12:36:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-06T12:36:23.919Z: Expanding GroupByKey operations into streaming Read/Write steps
May 06, 2022 12:36:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-06T12:36:23.983Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 06, 2022 12:36:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-06T12:36:24.087Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 06, 2022 12:36:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-06T12:36:24.120Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
May 06, 2022 12:36:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-06T12:36:24.154Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
May 06, 2022 12:36:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-06T12:36:24.188Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
May 06, 2022 12:36:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-06T12:36:24.220Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
May 06, 2022 12:36:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-06T12:36:24.254Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
May 06, 2022 12:36:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-06T12:36:24.288Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
May 06, 2022 12:36:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-06T12:36:24.332Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
May 06, 2022 12:36:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-06T12:36:24.352Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
May 06, 2022 12:36:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-06T12:36:24.379Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
May 06, 2022 12:36:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-06T12:36:24.412Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
May 06, 2022 12:36:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-06T12:36:24.448Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
May 06, 2022 12:36:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-06T12:36:24.470Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
May 06, 2022 12:36:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-06T12:36:24.502Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
May 06, 2022 12:36:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-06T12:36:24.528Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
May 06, 2022 12:36:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-06T12:36:24.559Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
May 06, 2022 12:36:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-06T12:36:24.627Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
May 06, 2022 12:36:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-06T12:36:24.654Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
May 06, 2022 12:36:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-06T12:36:24.697Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
May 06, 2022 12:36:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-06T12:36:24.733Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
May 06, 2022 12:36:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-06T12:36:24.755Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
May 06, 2022 12:36:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-06T12:36:24.787Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
May 06, 2022 12:36:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-06T12:36:24.820Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
May 06, 2022 12:36:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-06T12:36:24.993Z: Running job using Streaming Engine
May 06, 2022 12:36:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-06T12:36:25.292Z: Starting 5 ****s in us-central1-b...
May 06, 2022 12:36:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-06T12:36:35.180Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 06, 2022 12:36:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-06T12:36:47.859Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
May 06, 2022 12:37:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-06T12:37:47.668Z: Workers have started successfully.
May 06, 2022 4:01:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-06T16:01:11.959Z: Cancel request is committed for workflow job: 2022-05-06_05_36_09-16672906884944557169.
May 06, 2022 4:01:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-06T16:01:12.063Z: Cleaning up.
May 06, 2022 4:01:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-06T16:01:12.173Z: Stopping **** pool...
May 06, 2022 4:01:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-06T16:01:12.236Z: Stopping **** pool...
May 06, 2022 4:01:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-06T16:01:46.626Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
May 06, 2022 4:01:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-06T16:01:46.663Z: Worker pool stopped.
May 06, 2022 4:01:56 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2022-05-06_05_36_09-16672906884944557169 finished with status CANCELLED.
Load test results for test (ID): aea7067d-9935-4110-bf99-16a4bf6df520 and timestamp: 2022-05-06T12:35:56.758000000Z:
Metric: Value:
dataflow_v2_java17_runtime_sec 12160.557
dataflow_v2_java17_total_bytes_count 3.05775743E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)
> Task :sdks:java:testing:load-tests:run FAILED
> Task :runners:google-cloud-dataflow-java:cleanUpDockerJavaImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220506123132
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:df7a799483a8dfcf0e4964b95e5dc6ef838d5908578aa974f5608b4b2f79498f
Deleted: sha256:c61e2bf56473ec4563f54698aa2a8e107b9212d2f9cb4249e1b8926eebd47660
Deleted: sha256:9d9130c54f1baeb08b94a07964e9e22ecd4bab33862e7798a10319d1d8d4b08d
Deleted: sha256:8467a3fb70145cc93817e21ee5c868eec7532f4e86ca70e4d38f522af0cb398c
Deleted: sha256:59fbca7d507b996b1d51fe22b67984f3ba7e986f1c53f6e3a14639d6277f637d
Deleted: sha256:c5e291eab10e7283e0ca96918031f1a9e83e8b8411f7ac5b1f40ec2f30ba230e
Deleted: sha256:6f601d14f835a5e2a860e8838164b3f7b7d0e20a6c80cab2e040083a47b0b56f
Deleted: sha256:b495ad196c71a0563ffb223307fb959e64e64a20da8e12016bb4602bf19fa9a5
Deleted: sha256:7730c370c352c50337fe7f2bad2ce7e584bfef3be35d36987d4541fbb7c06a2d
Deleted: sha256:6ee922752cb80e19cfd4beb7f1111c74c090f8d970a315f577ca3ab541447ab4
Deleted: sha256:9e190c203486d33f72fe7036f544e381072667a81abf21b65a53927a8ca3ddd9
Deleted: sha256:f2bd20d129b02ee50849e5c9152aac79942dba3415537ea9cba062da73f12839
Deleted: sha256:2909b2457cefac16105342dbdf59f3e9288bd44393da551bd3665665ede39ac8
Deleted: sha256:00968e40a472f2b5ad0dc5d917c967cc8b28c86c62b3207ac0a6e3f943550041
Deleted: sha256:f4d6879504907af6782a76ee389504614fb7ab9455c4b107eb02f2c19046ed49
Deleted: sha256:4be40ff13744c31b7d2eb8e31c8e9e5dd9b9690ef2a3bec37134190d3c85cdb0
Deleted: sha256:36bcd0ee35e63597b3e643aef706aac4fbef299ba2773de693bc6ce03688f272
Deleted: sha256:da888297f316c3a48e5f9787b1fe1a9384df56871c0e907eeb0fd44a91012df7
Deleted: sha256:58d9461f9b85e6dc8dce3d9061ab1b7690a0a8b13fc305f246ef037183b06bf3
Deleted: sha256:9e3aed6009e7e4f7dbf0492e1853b2a2549a3e81cfd12daeaeabfb6094af94bf
Deleted: sha256:75fb1d5e10ded2a00433da41d70fbc05be0aa093ebdeb90611cba0b377d7fec6
Deleted: sha256:e999ffd0c7debe92dbcad4eabf1788922fefe6e251093e945a669a8f458b1c67
Deleted: sha256:9b05c11211514e7340a5e8839f29b37218653537f63497a47cede3c2125bd15b
Deleted: sha256:65e811058f16c4f2bfe8c6b1a1d910ea4dc0532e1a41c80eb57bbc9fded148cb
Deleted: sha256:b66d07df3cdafc664e5592122dc648cfb3250d33c251f4ec18d46206d483bb23
Deleted: sha256:1427b463374c90c14de7918ef0f0c7b26c7fd8a48ca5e4b9e3ef08203ad7c545
Deleted: sha256:d637282d3e86cd34969b59506e5e78e9bb4e3cde032e3c4f9e6e04c08a078941
Deleted: sha256:89a707572366f6809adb4362ae03d1c016718b4c4f45fe40192369f027375415
Deleted: sha256:4d36ad43cbe6758277403104e5f851d113c19fe8dd2e1cbd383416b8ac45d28d
Deleted: sha256:a7ce6b29ef0448179f5fc9bd58e3ff1eb05009c7417e54905a9dd6b2a10f7fff
Deleted: sha256:632d214ecd9402e629ff5d15ce8a4c318d85c083035e0043a09c7d41bdc02d30
Deleted: sha256:6eb47e9cb58a35946ffaab9ad85d0fbe80def98404dee173c9e679c55c36d57e
Deleted: sha256:affbfb0c7a73592504cf91a2cbde5d28f930f33acc93430d08460a0c68304935
Deleted: sha256:239810707ccfa57dd42da91f0a3261bf3902d2c487fa6c35dc7241f5e86b3335
Deleted: sha256:46e146925a73a269ca21a84f848012801e5f0e57e63c23c8dd5206c23efda7ec
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220506123132]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:df7a799483a8dfcf0e4964b95e5dc6ef838d5908578aa974f5608b4b2f79498f]
Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220506123132] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:df7a799483a8dfcf0e4964b95e5dc6ef838d5908578aa974f5608b4b2f79498f])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:df7a799483a8dfcf0e4964b95e5dc6ef838d5908578aa974f5608b4b2f79498f
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:df7a799483a8dfcf0e4964b95e5dc6ef838d5908578aa974f5608b4b2f79498f
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:df7a799483a8dfcf0e4964b95e5dc6ef838d5908578aa974f5608b4b2f79498f].
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.
BUILD FAILED in 3h 31m 24s
110 actionable tasks: 73 executed, 33 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/l5ae3vt3byunc
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17 #136
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17/136/display/redirect?page=changes>
Changes:
[Heejong Lee] [BEAM-9245] Unable to pull datatore Entity which contains dict
[bulat.safiullin] [BEAM-14382] [Website] add banner container for with css, images, html
[Jan Lukavský] [BEAM-14196] add test verifying output watermark propagation in bundle
[Jan Lukavský] [BEAM-14196] Fix FlinkRunner mid-bundle output watermark handling
[nielm] [BEAM-14405] Fix NPE when ProjectID is not specified in a template
[bulat.safiullin] [BEAM-14382] change mobile banner img, add padding to banner section
[ahmedabualsaud] fix test decotrator typo
[noreply] Merge pull request #17440 from [BEAM-14329] Enable exponential backoff
[noreply] [BEAM-11104] Fix output forwarding issue for ProcessContinuations
[noreply] re-add testing package to pydoc (#17524)
[Heejong Lee] add test
[noreply] [BEAM-14250] Amended the workaround (#17531)
[noreply] [BEAM-11104] Fix broken split result validation (#17546)
[noreply] Fixed a SQL and screenshots in the Beam SQL blog (#17545)
[noreply] Merge pull request #17417: [BEAM-14388] Address some performance
[noreply] [BEAM-14386] [Flink] Support for scala 2.12 (#17512)
[noreply] [BEAM-14294] Worker changes to support trivial Batched DoFns (#17384)
[zyichi] Moving to 2.40.0-SNAPSHOT on master branch.
[noreply] [BEAM-14048] [CdapIO] Add ConfigWrapper for building CDAP PluginConfigs
------------------------------------------
[...truncated 50.60 KB...]
a037458de4e0: Preparing
a0aeee58f513: Waiting
d7b56d57c324: Waiting
04d5df9ec8e8: Waiting
dc197617b2ee: Waiting
bafdbe68e4ae: Preparing
c85968a0c042: Waiting
3263c0e06234: Waiting
3bc383470c05: Waiting
673012901f20: Waiting
e93827457889: Waiting
a13c519c6361: Preparing
bafdbe68e4ae: Waiting
a13c519c6361: Waiting
a5830f62fc2b: Waiting
a037458de4e0: Waiting
8996c988e5df: Pushed
882944b20e14: Pushed
c72f16974de9: Pushed
0d42ebd3c794: Pushed
911b1a8b5e57: Pushed
a0aeee58f513: Pushed
a81a04dbaea7: Pushed
d7b56d57c324: Pushed
1d4e401c8952: Pushed
04d5df9ec8e8: Pushed
a5830f62fc2b: Pushed
3bc383470c05: Layer already exists
e93827457889: Layer already exists
c85968a0c042: Pushed
08fa02ce37eb: Layer already exists
a037458de4e0: Layer already exists
bafdbe68e4ae: Layer already exists
a13c519c6361: Layer already exists
3263c0e06234: Pushed
dc197617b2ee: Pushed
673012901f20: Pushed
20220505123041: digest: sha256:131e4451715f7df49079705cc0d0df461d263e958b7b9a5e9d379950b22ba9a0 size: 4729
> Task :sdks:java:testing:load-tests:run
May 05, 2022 12:32:46 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
May 05, 2022 12:32:46 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 221 files. Enable logging at DEBUG level to see which files will be staged.
May 05, 2022 12:32:47 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
May 05, 2022 12:32:47 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
May 05, 2022 12:32:49 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 221 files from PipelineOptions.filesToStage to staging location to prepare for execution.
May 05, 2022 12:32:49 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 221 files cached, 0 files newly uploaded in 0 seconds
May 05, 2022 12:32:49 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
May 05, 2022 12:32:50 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <119943 bytes, hash d2875933a8f7081a206ea4f4f77ec0d39d59892254cb85fcb163b4db2e98fae1> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-0odZM6j3CBogbqT0937A051ZiSJUy4X8sWO02y6Y-uE.pb
May 05, 2022 12:32:51 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
May 05, 2022 12:32:51 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@463561c5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@659feb22, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3468ee6e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2f4b98f6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@421def93, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58c1da09, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2b2954e1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58d6e55a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@751ae8a4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@235d659c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4232b34a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2da16263, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@f5ce0bb, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@47e51549, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@101a461c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@360e9c06, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5ebffb44, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@311ff287, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7377781e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@31db34da]
May 05, 2022 12:32:51 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
May 05, 2022 12:32:51 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
May 05, 2022 12:32:51 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
May 05, 2022 12:32:51 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
May 05, 2022 12:32:51 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3a2e9f5b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7da34b26, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@10c47c79, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3b2553d9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@753fd7a1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12a2585b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@21b6c9c2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@309cedb6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3b95a6db, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3bf54172, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2c9a6717, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b3cde6f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6d091cad, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7c663eaf, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3bb5ceb, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1e692555, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ba0ae41, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@76fe6cdc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2ffb3aec, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@786ff1cb]
May 05, 2022 12:32:51 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
May 05, 2022 12:32:51 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
May 05, 2022 12:32:51 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
May 05, 2022 12:32:51 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
May 05, 2022 12:32:51 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
May 05, 2022 12:32:51 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
May 05, 2022 12:32:51 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
May 05, 2022 12:32:51 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
May 05, 2022 12:32:51 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
May 05, 2022 12:32:51 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
May 05, 2022 12:32:51 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
May 05, 2022 12:32:51 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.40.0-SNAPSHOT
May 05, 2022 12:32:52 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-05-05_05_32_52-6914658191205046925?project=apache-beam-testing
May 05, 2022 12:32:52 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2022-05-05_05_32_52-6914658191205046925
May 05, 2022 12:32:52 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2022-05-05_05_32_52-6914658191205046925
May 05, 2022 12:33:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2022-05-05T12:33:04.755Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java170dataflow0v20streaming0cogbk01-jenkins-05-839r. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
May 05, 2022 12:33:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-05T12:33:12.969Z: Worker configuration: e2-standard-2 in us-central1-b.
May 05, 2022 12:33:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-05T12:33:13.766Z: Expanding SplittableParDo operations into optimizable parts.
May 05, 2022 12:33:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-05T12:33:13.798Z: Expanding CollectionToSingleton operations into optimizable parts.
May 05, 2022 12:33:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-05T12:33:13.898Z: Expanding CoGroupByKey operations into optimizable parts.
May 05, 2022 12:33:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-05T12:33:13.959Z: Expanding SplittableProcessKeyed operations into optimizable parts.
May 05, 2022 12:33:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-05T12:33:13.987Z: Expanding GroupByKey operations into streaming Read/Write steps
May 05, 2022 12:33:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-05T12:33:14.068Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 05, 2022 12:33:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-05T12:33:14.146Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 05, 2022 12:33:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-05T12:33:14.178Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
May 05, 2022 12:33:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-05T12:33:14.208Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
May 05, 2022 12:33:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-05T12:33:14.232Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
May 05, 2022 12:33:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-05T12:33:14.252Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
May 05, 2022 12:33:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-05T12:33:14.281Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
May 05, 2022 12:33:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-05T12:33:14.316Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
May 05, 2022 12:33:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-05T12:33:14.346Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
May 05, 2022 12:33:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-05T12:33:14.380Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
May 05, 2022 12:33:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-05T12:33:14.404Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
May 05, 2022 12:33:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-05T12:33:14.461Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
May 05, 2022 12:33:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-05T12:33:14.500Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
May 05, 2022 12:33:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-05T12:33:14.531Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
May 05, 2022 12:33:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-05T12:33:14.563Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
May 05, 2022 12:33:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-05T12:33:14.596Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
May 05, 2022 12:33:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-05T12:33:14.630Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
May 05, 2022 12:33:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-05T12:33:14.665Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
May 05, 2022 12:33:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-05T12:33:14.697Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
May 05, 2022 12:33:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-05T12:33:14.718Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
May 05, 2022 12:33:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-05T12:33:14.744Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
May 05, 2022 12:33:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-05T12:33:14.781Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
May 05, 2022 12:33:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-05T12:33:14.848Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
May 05, 2022 12:33:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-05T12:33:14.877Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
May 05, 2022 12:33:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-05T12:33:15.025Z: Running job using Streaming Engine
May 05, 2022 12:33:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-05T12:33:15.265Z: Starting 5 ****s in us-central1-b...
May 05, 2022 12:33:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-05T12:33:41.370Z: Autoscaling: Raised the number of ****s to 4 so that the pipeline can catch up with its backlog and keep up with its input rate.
May 05, 2022 12:33:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-05T12:33:41.398Z: Resized **** pool to 4, though goal was 5. This could be a quota issue.
May 05, 2022 12:33:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-05T12:33:46.902Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 05, 2022 12:33:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-05T12:33:51.587Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
May 05, 2022 12:34:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-05T12:34:50.485Z: Workers have started successfully.
May 05, 2022 3:05:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-05T15:05:19.624Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 05, 2022 3:05:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-05T15:05:24.302Z: Worker configuration: e2-standard-2 in us-central1-b.
May 05, 2022 4:00:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-05T16:00:49.572Z: Cancel request is committed for workflow job: 2022-05-05_05_32_52-6914658191205046925.
May 05, 2022 4:00:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-05T16:00:49.600Z: Cleaning up.
May 05, 2022 4:00:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-05T16:00:49.808Z: Stopping **** pool...
May 05, 2022 4:00:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-05T16:00:49.897Z: Stopping **** pool...
May 05, 2022 4:01:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-05T16:01:26.536Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
May 05, 2022 4:01:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-05T16:01:26.569Z: Worker pool stopped.
May 05, 2022 4:01:39 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2022-05-05_05_32_52-6914658191205046925 finished with status CANCELLED.
Load test results for test (ID): 3e4bb4f5-730b-4386-9ddb-8239732ef044 and timestamp: 2022-05-05T12:32:46.775000000Z:
Metric: Value:
dataflow_v2_java17_runtime_sec 12316.972
dataflow_v2_java17_total_bytes_count 3.41388437E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)
> Task :sdks:java:testing:load-tests:run FAILED
> Task :runners:google-cloud-dataflow-java:cleanUpDockerJavaImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220505123041
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:131e4451715f7df49079705cc0d0df461d263e958b7b9a5e9d379950b22ba9a0
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220505123041]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:131e4451715f7df49079705cc0d0df461d263e958b7b9a5e9d379950b22ba9a0]
Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220505123041] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:131e4451715f7df49079705cc0d0df461d263e958b7b9a5e9d379950b22ba9a0])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:131e4451715f7df49079705cc0d0df461d263e958b7b9a5e9d379950b22ba9a0
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:131e4451715f7df49079705cc0d0df461d263e958b7b9a5e9d379950b22ba9a0
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:131e4451715f7df49079705cc0d0df461d263e958b7b9a5e9d379950b22ba9a0].
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.
BUILD FAILED in 3h 31m 20s
110 actionable tasks: 76 executed, 30 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/vqaloxpdhzprm
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17 #135
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17/135/display/redirect?page=changes>
Changes:
[noreply] fix: JDBC config schema fields order
[Brian Hulette] Revert "Merge pull request #17255 from kileys/test-revert"
[Brian Hulette] BEAM-14231: bypass schema cache for
[noreply] [BEAM-13657] Follow up update version warning in __init__ (#17493)
[noreply] Merge pull request #17431 from [BEAM-14273] Add integration tests for BQ
[noreply] Merge pull request #17205 from [BEAM-14145] [Website] add carousel to
[noreply] [BEAM-14064] fix es io windowing (#17112)
[noreply] [BEAM-13670] Upgraded ipython from v7 to v8 (#17529)
[noreply] [BEAM-11104] Enable ProcessContinuation return values, add unit test
[Robert Bradshaw] [BEAM-14403] Allow Prime to be used with legacy workers.
[noreply] [BEAM-11106] Support drain in Go SDK (#17432)
[noreply] add __Init__ to inference. (#17514)
------------------------------------------
[...truncated 50.32 KB...]
3bc383470c05: Preparing
e93827457889: Preparing
08fa02ce37eb: Preparing
a037458de4e0: Preparing
bafdbe68e4ae: Preparing
a13c519c6361: Preparing
3bc70159237a: Waiting
a6e4d1804105: Waiting
2d543f6e6e21: Waiting
0307cef3be55: Waiting
a037458de4e0: Waiting
08fa02ce37eb: Waiting
8b92e4958fd1: Waiting
bafdbe68e4ae: Waiting
9969bcd1e3fd: Waiting
e93827457889: Waiting
3bc383470c05: Waiting
2520eb2e8567: Waiting
72bfc7510f27: Waiting
a4ad5e0bd783: Waiting
f116eab367a0: Pushed
331c9a0b820a: Pushed
cb7330ae032c: Pushed
25a6908e0e13: Pushed
b7bbab1ffc45: Pushed
a4ad5e0bd783: Pushed
3bc70159237a: Pushed
2520eb2e8567: Pushed
9745019c1f1b: Pushed
2d543f6e6e21: Pushed
72bfc7510f27: Pushed
0307cef3be55: Pushed
3bc383470c05: Layer already exists
e93827457889: Layer already exists
08fa02ce37eb: Layer already exists
a037458de4e0: Layer already exists
bafdbe68e4ae: Layer already exists
a13c519c6361: Layer already exists
8b92e4958fd1: Pushed
9969bcd1e3fd: Pushed
a6e4d1804105: Pushed
20220504123041: digest: sha256:b607c702456435e49fbe29707a5e8a335df87c310c8eae2069a24e6102daeb1f size: 4729
> Task :sdks:java:testing:load-tests:run
May 04, 2022 12:32:36 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
May 04, 2022 12:32:37 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 221 files. Enable logging at DEBUG level to see which files will be staged.
May 04, 2022 12:32:38 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
May 04, 2022 12:32:38 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
May 04, 2022 12:32:40 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 221 files from PipelineOptions.filesToStage to staging location to prepare for execution.
May 04, 2022 12:32:41 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 221 files cached, 0 files newly uploaded in 0 seconds
May 04, 2022 12:32:41 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
May 04, 2022 12:32:41 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <119943 bytes, hash a19c96ecdc20c9355bed3a3f2f46e859fa201abbc15dd86e5cf410cc6d045525> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-oZyW7NwgyTVb7To_L0boWfogGrvBXdhuXPQQzG0EVSU.pb
May 04, 2022 12:32:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
May 04, 2022 12:32:43 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@659feb22, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3468ee6e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2f4b98f6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@421def93, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58c1da09, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2b2954e1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58d6e55a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@751ae8a4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@235d659c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4232b34a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2da16263, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@f5ce0bb, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@47e51549, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@101a461c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@360e9c06, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5ebffb44, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@311ff287, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7377781e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@31db34da, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@109f8c7e]
May 04, 2022 12:32:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
May 04, 2022 12:32:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
May 04, 2022 12:32:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
May 04, 2022 12:32:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
May 04, 2022 12:32:43 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3bf54172, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2c9a6717, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b3cde6f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6d091cad, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7c663eaf, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3bb5ceb, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1e692555, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ba0ae41, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@76fe6cdc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2ffb3aec, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@786ff1cb, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@46039a21, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@431e86b1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@35c4e864, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@32a2a6be, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@682af059, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5f36c8e3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4da39ca9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6a9344f5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5584d9c6]
May 04, 2022 12:32:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
May 04, 2022 12:32:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
May 04, 2022 12:32:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
May 04, 2022 12:32:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
May 04, 2022 12:32:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
May 04, 2022 12:32:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
May 04, 2022 12:32:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
May 04, 2022 12:32:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
May 04, 2022 12:32:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
May 04, 2022 12:32:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
May 04, 2022 12:32:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
May 04, 2022 12:32:43 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.39.0-SNAPSHOT
May 04, 2022 12:32:44 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-05-04_05_32_43-7390128921444977935?project=apache-beam-testing
May 04, 2022 12:32:44 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2022-05-04_05_32_43-7390128921444977935
May 04, 2022 12:32:44 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2022-05-04_05_32_43-7390128921444977935
May 04, 2022 12:32:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2022-05-04T12:32:47.612Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java170dataflow0v20streaming0cogbk01-jenkins-05-q8mb. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
May 04, 2022 12:32:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-04T12:32:52.883Z: Worker configuration: e2-standard-2 in us-central1-b.
May 04, 2022 12:32:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-04T12:32:53.521Z: Expanding SplittableParDo operations into optimizable parts.
May 04, 2022 12:32:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-04T12:32:53.565Z: Expanding CollectionToSingleton operations into optimizable parts.
May 04, 2022 12:32:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-04T12:32:53.692Z: Expanding CoGroupByKey operations into optimizable parts.
May 04, 2022 12:32:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-04T12:32:53.790Z: Expanding SplittableProcessKeyed operations into optimizable parts.
May 04, 2022 12:32:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-04T12:32:53.843Z: Expanding GroupByKey operations into streaming Read/Write steps
May 04, 2022 12:32:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-04T12:32:53.962Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 04, 2022 12:32:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-04T12:32:54.105Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 04, 2022 12:32:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-04T12:32:54.144Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
May 04, 2022 12:32:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-04T12:32:54.191Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
May 04, 2022 12:32:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-04T12:32:54.237Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
May 04, 2022 12:32:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-04T12:32:54.311Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
May 04, 2022 12:32:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-04T12:32:54.350Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
May 04, 2022 12:32:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-04T12:32:54.403Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
May 04, 2022 12:32:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-04T12:32:54.441Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
May 04, 2022 12:32:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-04T12:32:54.483Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
May 04, 2022 12:32:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-04T12:32:54.527Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
May 04, 2022 12:32:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-04T12:32:54.581Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
May 04, 2022 12:32:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-04T12:32:54.620Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
May 04, 2022 12:32:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-04T12:32:54.659Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
May 04, 2022 12:32:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-04T12:32:54.705Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
May 04, 2022 12:32:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-04T12:32:54.742Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
May 04, 2022 12:32:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-04T12:32:54.779Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
May 04, 2022 12:32:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-04T12:32:54.819Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
May 04, 2022 12:32:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-04T12:32:54.858Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
May 04, 2022 12:32:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-04T12:32:54.925Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
May 04, 2022 12:32:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-04T12:32:54.966Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
May 04, 2022 12:32:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-04T12:32:55.004Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
May 04, 2022 12:32:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-04T12:32:55.043Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
May 04, 2022 12:32:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-04T12:32:55.123Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
May 04, 2022 12:32:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-04T12:32:55.382Z: Running job using Streaming Engine
May 04, 2022 12:32:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-04T12:32:55.912Z: Starting 5 ****s in us-central1-b...
May 04, 2022 12:33:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-04T12:33:17.504Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 04, 2022 12:33:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-04T12:33:20.346Z: Autoscaling: Raised the number of ****s to 4 so that the pipeline can catch up with its backlog and keep up with its input rate.
May 04, 2022 12:33:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-04T12:33:20.377Z: Resized **** pool to 4, though goal was 5. This could be a quota issue.
May 04, 2022 12:33:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-04T12:33:30.625Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
May 04, 2022 12:34:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-04T12:34:22.731Z: Workers have started successfully.
May 04, 2022 4:01:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-04T16:00:59.996Z: Cancel request is committed for workflow job: 2022-05-04_05_32_43-7390128921444977935.
May 04, 2022 4:01:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-04T16:01:00.111Z: Cleaning up.
May 04, 2022 4:01:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-04T16:01:00.314Z: Stopping **** pool...
May 04, 2022 4:01:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-04T16:01:00.383Z: Stopping **** pool...
May 04, 2022 4:01:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-04T16:01:35.915Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
May 04, 2022 4:01:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-04T16:01:35.987Z: Worker pool stopped.
May 04, 2022 4:01:44 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2022-05-04_05_32_43-7390128921444977935 finished with status CANCELLED.
Load test results for test (ID): 1fb5cbe1-adcf-4c95-8641-b4766f1d23bb and timestamp: 2022-05-04T12:32:37.757000000Z:
Metric: Value:
dataflow_v2_java17_runtime_sec 12363.386
dataflow_v2_java17_total_bytes_count 4.12943456E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)
> Task :sdks:java:testing:load-tests:run FAILED
> Task :runners:google-cloud-dataflow-java:cleanUpDockerJavaImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220504123041
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:b607c702456435e49fbe29707a5e8a335df87c310c8eae2069a24e6102daeb1f
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220504123041]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:b607c702456435e49fbe29707a5e8a335df87c310c8eae2069a24e6102daeb1f]
Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220504123041] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:b607c702456435e49fbe29707a5e8a335df87c310c8eae2069a24e6102daeb1f])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:b607c702456435e49fbe29707a5e8a335df87c310c8eae2069a24e6102daeb1f
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:b607c702456435e49fbe29707a5e8a335df87c310c8eae2069a24e6102daeb1f
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:b607c702456435e49fbe29707a5e8a335df87c310c8eae2069a24e6102daeb1f].
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.
BUILD FAILED in 3h 31m 26s
110 actionable tasks: 73 executed, 33 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/d7u3qhbfqkica
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17 #134
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17/134/display/redirect?page=changes>
Changes:
[yathu] [BEAM-14375] Fix Java Wordcount Dataflow postcommit
[Robert Bradshaw] Allow arithmetic between deferred scalars.
[noreply] [BEAM-14390] Set user-agent when pulling licenses to avoid 403s (#17521)
[noreply] [BEAM-8688] Upgrade GCSIO to 2.2.6 (#17486)
[noreply] [BEAM-14253] patch SubscriptionPartitionLoader to work around a dataflow
[noreply] Add website link log to notify user of pre-build workflow. (#17498)
[noreply] [BEAM-11105] Add timestamp observing watermark estimation (#17476)
[noreply] Merge pull request #17487 from Adding user-agent to GCS client in Python
[noreply] [BEAM-10265] Display error message if trying to infer recursive schema
[noreply] [BEAM-12575] Upgraded ipykernel from v5 to v6 (#17526)
[noreply] [BEAM-11105] Add docs + CHANGES.md entry for Go Watermark Estimation
[noreply] Merge pull request #17380 from [BEAM-14314][BEAM-9532] Add last_updated
------------------------------------------
[...truncated 45.93 KB...]
> Task :release:go-licenses:java:dockerRun
+ go-licenses save github.com/apache/beam/sdks/java/container --save_path=/output/licenses
+ go-licenses csv github.com/apache/beam/sdks/java/container
+ tee /output/licenses/list.csv
+ chmod -R a+w /output/licenses
> Task :release:go-licenses:java:createLicenses
> Task :sdks:java:container:java17:copyGolangLicenses
> Task :sdks:java:container:java17:dockerPrepare
> Task :sdks:java:container:java17:docker
> Task :runners:google-cloud-dataflow-java:buildAndPushDockerJavaContainer
WARNING: `gcloud docker` will not be supported for Docker client versions above 18.03.
As an alternative, use `gcloud auth configure-docker` to configure `docker` to
use `gcloud` as a credential helper, then use `docker` as you would for non-GCR
registries, e.g. `docker pull gcr.io/project-id/my-image`. Add
`--verbosity=error` to silence this warning: `gcloud docker
--verbosity=error -- pull gcr.io/project-id/my-image`.
See: https://cloud.google.com/container-registry/docs/support/deprecation-notices#gcloud-docker
The push refers to repository [us.gcr.io/apache-beam-testing/java-postcommit-it/java]
52eb74fe55c7: Preparing
a0047307b411: Preparing
22f5c719ac2a: Preparing
571a60fbcd54: Preparing
5cafb396ea19: Preparing
b9bb22d55e3e: Preparing
cf1017aad468: Preparing
d825907d653d: Preparing
8db3937692b0: Preparing
bac86cd751e4: Preparing
9fd422015e17: Preparing
5e79d87fff88: Preparing
51a379cd90f3: Preparing
ea4b0956d47c: Preparing
75795add637a: Preparing
3bc383470c05: Preparing
e93827457889: Preparing
08fa02ce37eb: Preparing
a037458de4e0: Preparing
bafdbe68e4ae: Preparing
a13c519c6361: Preparing
ea4b0956d47c: Waiting
75795add637a: Waiting
8db3937692b0: Waiting
a037458de4e0: Waiting
b9bb22d55e3e: Waiting
bac86cd751e4: Waiting
bafdbe68e4ae: Waiting
3bc383470c05: Waiting
a13c519c6361: Waiting
9fd422015e17: Waiting
51a379cd90f3: Waiting
e93827457889: Waiting
08fa02ce37eb: Waiting
d825907d653d: Waiting
cf1017aad468: Waiting
5cafb396ea19: Pushed
22f5c719ac2a: Pushed
a0047307b411: Pushed
571a60fbcd54: Pushed
52eb74fe55c7: Pushed
cf1017aad468: Pushed
d825907d653d: Pushed
bac86cd751e4: Pushed
8db3937692b0: Pushed
5e79d87fff88: Pushed
b9bb22d55e3e: Pushed
3bc383470c05: Layer already exists
e93827457889: Layer already exists
9fd422015e17: Pushed
08fa02ce37eb: Layer already exists
a037458de4e0: Layer already exists
bafdbe68e4ae: Layer already exists
a13c519c6361: Layer already exists
ea4b0956d47c: Pushed
75795add637a: Pushed
51a379cd90f3: Pushed
20220503123036: digest: sha256:d3e45bd5042d6ca189215f85538763fdb51a453a71b9f9ce4dd097d09e46a5dd size: 4729
> Task :sdks:java:testing:load-tests:run
May 03, 2022 12:32:36 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
May 03, 2022 12:32:36 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 221 files. Enable logging at DEBUG level to see which files will be staged.
May 03, 2022 12:32:37 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
May 03, 2022 12:32:37 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
May 03, 2022 12:32:41 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 221 files from PipelineOptions.filesToStage to staging location to prepare for execution.
May 03, 2022 12:32:43 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 221 files cached, 0 files newly uploaded in 2 seconds
May 03, 2022 12:32:43 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
May 03, 2022 12:32:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <119943 bytes, hash 59f916c5e19489bd88ce5811761090b62724c332f6f3411ef2c783946ee7ef48> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-WfkWxeGUib2IzlgRdhCQtickwzL280Ee8seDlG7n70g.pb
May 03, 2022 12:32:45 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
May 03, 2022 12:32:45 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@463561c5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@659feb22, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3468ee6e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2f4b98f6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@421def93, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58c1da09, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2b2954e1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58d6e55a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@751ae8a4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@235d659c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4232b34a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2da16263, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@f5ce0bb, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@47e51549, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@101a461c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@360e9c06, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5ebffb44, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@311ff287, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7377781e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@31db34da]
May 03, 2022 12:32:45 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
May 03, 2022 12:32:45 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
May 03, 2022 12:32:45 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
May 03, 2022 12:32:45 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
May 03, 2022 12:32:45 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3a2e9f5b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7da34b26, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@10c47c79, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3b2553d9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@753fd7a1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12a2585b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@21b6c9c2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@309cedb6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3b95a6db, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3bf54172, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2c9a6717, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b3cde6f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6d091cad, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7c663eaf, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3bb5ceb, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1e692555, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ba0ae41, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@76fe6cdc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2ffb3aec, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@786ff1cb]
May 03, 2022 12:32:45 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
May 03, 2022 12:32:45 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
May 03, 2022 12:32:45 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
May 03, 2022 12:32:45 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
May 03, 2022 12:32:45 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
May 03, 2022 12:32:45 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
May 03, 2022 12:32:45 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
May 03, 2022 12:32:45 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
May 03, 2022 12:32:45 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
May 03, 2022 12:32:45 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
May 03, 2022 12:32:45 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
May 03, 2022 12:32:45 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.39.0-SNAPSHOT
May 03, 2022 12:32:46 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-05-03_05_32_46-11901399501120612900?project=apache-beam-testing
May 03, 2022 12:32:46 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2022-05-03_05_32_46-11901399501120612900
May 03, 2022 12:32:46 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2022-05-03_05_32_46-11901399501120612900
May 03, 2022 12:32:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2022-05-03T12:32:50.653Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java170dataflow0v20streaming0cogbk01-jenkins-05-trww. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
May 03, 2022 12:32:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-03T12:32:58.016Z: Worker configuration: e2-standard-2 in us-central1-b.
May 03, 2022 12:33:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-03T12:32:58.605Z: Expanding SplittableParDo operations into optimizable parts.
May 03, 2022 12:33:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-03T12:32:58.682Z: Expanding CollectionToSingleton operations into optimizable parts.
May 03, 2022 12:33:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-03T12:32:59.174Z: Expanding CoGroupByKey operations into optimizable parts.
May 03, 2022 12:33:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-03T12:32:59.417Z: Expanding SplittableProcessKeyed operations into optimizable parts.
May 03, 2022 12:33:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-03T12:32:59.458Z: Expanding GroupByKey operations into streaming Read/Write steps
May 03, 2022 12:33:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-03T12:32:59.533Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 03, 2022 12:33:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-03T12:32:59.645Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 03, 2022 12:33:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-03T12:32:59.687Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
May 03, 2022 12:33:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-03T12:32:59.721Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
May 03, 2022 12:33:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-03T12:32:59.767Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
May 03, 2022 12:33:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-03T12:32:59.798Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
May 03, 2022 12:33:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-03T12:32:59.831Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
May 03, 2022 12:33:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-03T12:32:59.865Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
May 03, 2022 12:33:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-03T12:32:59.897Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
May 03, 2022 12:33:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-03T12:32:59.929Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
May 03, 2022 12:33:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-03T12:32:59.964Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
May 03, 2022 12:33:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-03T12:32:59.995Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
May 03, 2022 12:33:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-03T12:33:00.049Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
May 03, 2022 12:33:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-03T12:33:00.097Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
May 03, 2022 12:33:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-03T12:33:00.227Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
May 03, 2022 12:33:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-03T12:33:00.316Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
May 03, 2022 12:33:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-03T12:33:00.528Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
May 03, 2022 12:33:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-03T12:33:00.660Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
May 03, 2022 12:33:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-03T12:33:00.811Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
May 03, 2022 12:33:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-03T12:33:00.895Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
May 03, 2022 12:33:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-03T12:33:00.945Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
May 03, 2022 12:33:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-03T12:33:00.987Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
May 03, 2022 12:33:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-03T12:33:01.020Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
May 03, 2022 12:33:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-03T12:33:01.061Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
May 03, 2022 12:33:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-03T12:33:01.275Z: Running job using Streaming Engine
May 03, 2022 12:33:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-03T12:33:01.528Z: Starting 5 ****s in us-central1-b...
May 03, 2022 12:33:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-03T12:33:18.790Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 03, 2022 12:33:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-03T12:33:24.290Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
May 03, 2022 12:34:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-05-03T12:34:36.212Z: Workers have started successfully.
FATAL: command execution failed
java.io.IOException: Backing channel 'apache-beam-jenkins-10' is disconnected.
at hudson.remoting.RemoteInvocationHandler.channelOrFail(RemoteInvocationHandler.java:216)
at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:286)
at com.sun.proxy.$Proxy132.isAlive(Unknown Source)
at hudson.Launcher$RemoteLauncher$ProcImpl.isAlive(Launcher.java:1213)
at hudson.Launcher$RemoteLauncher$ProcImpl.join(Launcher.java:1205)
at hudson.Launcher$ProcStarter.join(Launcher.java:522)
at hudson.plugins.gradle.Gradle.perform(Gradle.java:317)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:806)
at hudson.model.Build$BuildExecution.build(Build.java:198)
at hudson.model.Build$BuildExecution.doRun(Build.java:163)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:514)
at hudson.model.Run.execute(Run.java:1888)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:99)
at hudson.model.Executor.run(Executor.java:432)
Caused by: hudson.remoting.Channel$OrderlyShutdown: Command Close created at
at hudson.remoting.Channel$CloseCommand.execute(Channel.java:1320)
at hudson.remoting.Channel$1.handle(Channel.java:607)
at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:81)
Caused by: Command Close created at
at hudson.remoting.Command.<init>(Command.java:70)
at hudson.remoting.Channel$CloseCommand.<init>(Channel.java:1313)
at hudson.remoting.Channel$CloseCommand.<init>(Channel.java:1311)
at hudson.remoting.Channel.close(Channel.java:1487)
at hudson.remoting.Channel.close(Channel.java:1454)
at hudson.remoting.Channel$CloseCommand.execute(Channel.java:1319)
... 2 more
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
ERROR: apache-beam-jenkins-10 is offline; cannot locate jdk_1.8_latest
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17 #133
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17/133/display/redirect?page=changes>
Changes:
[noreply] [BEAM-11105] Add manual watermark estimation (#17475)
------------------------------------------
[...truncated 388.69 KB...]
return opener.open(url, data, timeout)
File "/usr/lib/python3.8/urllib/request.py", line 531, in open
response = meth(req, response)
File "/usr/lib/python3.8/urllib/request.py", line 640, in http_response
response = self.parent.error(
File "/usr/lib/python3.8/urllib/request.py", line 569, in error
return self._call_chain(*args)
File "/usr/lib/python3.8/urllib/request.py", line 502, in _call_chain
result = func(*args)
File "/usr/lib/python3.8/urllib/request.py", line 649, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR:root:Invalid url for slf4j-api-1.7.30: http://www.opensource.org/licenses/mit-license.php after 9 retries.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17/ws/src/sdks/java/container/license_scripts/pull_licenses_java.py",> line 58, in pull_from_url
url_read = urlopen(url)
File "/usr/lib/python3.8/urllib/request.py", line 222, in urlopen
return opener.open(url, data, timeout)
File "/usr/lib/python3.8/urllib/request.py", line 531, in open
response = meth(req, response)
File "/usr/lib/python3.8/urllib/request.py", line 640, in http_response
response = self.parent.error(
File "/usr/lib/python3.8/urllib/request.py", line 569, in error
return self._call_chain(*args)
File "/usr/lib/python3.8/urllib/request.py", line 502, in _call_chain
result = func(*args)
File "/usr/lib/python3.8/urllib/request.py", line 649, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR:root:Invalid url for jdbc-1.16.3: http://opensource.org/licenses/MIT after 9 retries.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17/ws/src/sdks/java/container/license_scripts/pull_licenses_java.py",> line 58, in pull_from_url
url_read = urlopen(url)
File "/usr/lib/python3.8/urllib/request.py", line 222, in urlopen
return opener.open(url, data, timeout)
File "/usr/lib/python3.8/urllib/request.py", line 531, in open
response = meth(req, response)
File "/usr/lib/python3.8/urllib/request.py", line 640, in http_response
response = self.parent.error(
File "/usr/lib/python3.8/urllib/request.py", line 569, in error
return self._call_chain(*args)
File "/usr/lib/python3.8/urllib/request.py", line 502, in _call_chain
result = func(*args)
File "/usr/lib/python3.8/urllib/request.py", line 649, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR:root:Invalid url for kafka-1.16.3: http://opensource.org/licenses/MIT. Retrying...
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17/ws/src/sdks/java/container/license_scripts/pull_licenses_java.py",> line 58, in pull_from_url
url_read = urlopen(url)
File "/usr/lib/python3.8/urllib/request.py", line 222, in urlopen
return opener.open(url, data, timeout)
File "/usr/lib/python3.8/urllib/request.py", line 531, in open
response = meth(req, response)
File "/usr/lib/python3.8/urllib/request.py", line 640, in http_response
response = self.parent.error(
File "/usr/lib/python3.8/urllib/request.py", line 569, in error
return self._call_chain(*args)
File "/usr/lib/python3.8/urllib/request.py", line 502, in _call_chain
result = func(*args)
File "/usr/lib/python3.8/urllib/request.py", line 649, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR:root:Invalid url for postgresql-1.16.3: http://opensource.org/licenses/MIT after 9 retries.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17/ws/src/sdks/java/container/license_scripts/pull_licenses_java.py",> line 58, in pull_from_url
url_read = urlopen(url)
File "/usr/lib/python3.8/urllib/request.py", line 222, in urlopen
return opener.open(url, data, timeout)
File "/usr/lib/python3.8/urllib/request.py", line 531, in open
response = meth(req, response)
File "/usr/lib/python3.8/urllib/request.py", line 640, in http_response
response = self.parent.error(
File "/usr/lib/python3.8/urllib/request.py", line 569, in error
return self._call_chain(*args)
File "/usr/lib/python3.8/urllib/request.py", line 502, in _call_chain
result = func(*args)
File "/usr/lib/python3.8/urllib/request.py", line 649, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR:root:Invalid url for kafka-1.16.3: http://opensource.org/licenses/MIT. Retrying...
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17/ws/src/sdks/java/container/license_scripts/pull_licenses_java.py",> line 58, in pull_from_url
url_read = urlopen(url)
File "/usr/lib/python3.8/urllib/request.py", line 222, in urlopen
return opener.open(url, data, timeout)
File "/usr/lib/python3.8/urllib/request.py", line 531, in open
response = meth(req, response)
File "/usr/lib/python3.8/urllib/request.py", line 640, in http_response
response = self.parent.error(
File "/usr/lib/python3.8/urllib/request.py", line 569, in error
return self._call_chain(*args)
File "/usr/lib/python3.8/urllib/request.py", line 502, in _call_chain
result = func(*args)
File "/usr/lib/python3.8/urllib/request.py", line 649, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR:root:Invalid url for kafka-1.16.3: http://opensource.org/licenses/MIT. Retrying...
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17/ws/src/sdks/java/container/license_scripts/pull_licenses_java.py",> line 58, in pull_from_url
url_read = urlopen(url)
File "/usr/lib/python3.8/urllib/request.py", line 222, in urlopen
return opener.open(url, data, timeout)
File "/usr/lib/python3.8/urllib/request.py", line 531, in open
response = meth(req, response)
File "/usr/lib/python3.8/urllib/request.py", line 640, in http_response
response = self.parent.error(
File "/usr/lib/python3.8/urllib/request.py", line 569, in error
return self._call_chain(*args)
File "/usr/lib/python3.8/urllib/request.py", line 502, in _call_chain
result = func(*args)
File "/usr/lib/python3.8/urllib/request.py", line 649, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR:root:Invalid url for kafka-1.16.3: http://opensource.org/licenses/MIT. Retrying...
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17/ws/src/sdks/java/container/license_scripts/pull_licenses_java.py",> line 58, in pull_from_url
url_read = urlopen(url)
File "/usr/lib/python3.8/urllib/request.py", line 222, in urlopen
return opener.open(url, data, timeout)
File "/usr/lib/python3.8/urllib/request.py", line 531, in open
response = meth(req, response)
File "/usr/lib/python3.8/urllib/request.py", line 640, in http_response
response = self.parent.error(
File "/usr/lib/python3.8/urllib/request.py", line 569, in error
return self._call_chain(*args)
File "/usr/lib/python3.8/urllib/request.py", line 502, in _call_chain
result = func(*args)
File "/usr/lib/python3.8/urllib/request.py", line 649, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR:root:Invalid url for kafka-1.16.3: http://opensource.org/licenses/MIT. Retrying...
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17/ws/src/sdks/java/container/license_scripts/pull_licenses_java.py",> line 58, in pull_from_url
url_read = urlopen(url)
File "/usr/lib/python3.8/urllib/request.py", line 222, in urlopen
return opener.open(url, data, timeout)
File "/usr/lib/python3.8/urllib/request.py", line 531, in open
response = meth(req, response)
File "/usr/lib/python3.8/urllib/request.py", line 640, in http_response
response = self.parent.error(
File "/usr/lib/python3.8/urllib/request.py", line 569, in error
return self._call_chain(*args)
File "/usr/lib/python3.8/urllib/request.py", line 502, in _call_chain
result = func(*args)
File "/usr/lib/python3.8/urllib/request.py", line 649, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR:root:Invalid url for kafka-1.16.3: http://opensource.org/licenses/MIT. Retrying...
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17/ws/src/sdks/java/container/license_scripts/pull_licenses_java.py",> line 58, in pull_from_url
url_read = urlopen(url)
File "/usr/lib/python3.8/urllib/request.py", line 222, in urlopen
return opener.open(url, data, timeout)
File "/usr/lib/python3.8/urllib/request.py", line 531, in open
response = meth(req, response)
File "/usr/lib/python3.8/urllib/request.py", line 640, in http_response
response = self.parent.error(
File "/usr/lib/python3.8/urllib/request.py", line 569, in error
return self._call_chain(*args)
File "/usr/lib/python3.8/urllib/request.py", line 502, in _call_chain
result = func(*args)
File "/usr/lib/python3.8/urllib/request.py", line 649, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR:root:Invalid url for kafka-1.16.3: http://opensource.org/licenses/MIT. Retrying...
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17/ws/src/sdks/java/container/license_scripts/pull_licenses_java.py",> line 58, in pull_from_url
url_read = urlopen(url)
File "/usr/lib/python3.8/urllib/request.py", line 222, in urlopen
return opener.open(url, data, timeout)
File "/usr/lib/python3.8/urllib/request.py", line 531, in open
response = meth(req, response)
File "/usr/lib/python3.8/urllib/request.py", line 640, in http_response
response = self.parent.error(
File "/usr/lib/python3.8/urllib/request.py", line 569, in error
return self._call_chain(*args)
File "/usr/lib/python3.8/urllib/request.py", line 502, in _call_chain
result = func(*args)
File "/usr/lib/python3.8/urllib/request.py", line 649, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR:root:Invalid url for kafka-1.16.3: http://opensource.org/licenses/MIT. Retrying...
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17/ws/src/sdks/java/container/license_scripts/pull_licenses_java.py",> line 58, in pull_from_url
url_read = urlopen(url)
File "/usr/lib/python3.8/urllib/request.py", line 222, in urlopen
return opener.open(url, data, timeout)
File "/usr/lib/python3.8/urllib/request.py", line 531, in open
response = meth(req, response)
File "/usr/lib/python3.8/urllib/request.py", line 640, in http_response
response = self.parent.error(
File "/usr/lib/python3.8/urllib/request.py", line 569, in error
return self._call_chain(*args)
File "/usr/lib/python3.8/urllib/request.py", line 502, in _call_chain
result = func(*args)
File "/usr/lib/python3.8/urllib/request.py", line 649, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR:root:Invalid url for kafka-1.16.3: http://opensource.org/licenses/MIT after 9 retries.
ERROR:root:['system-rules-1.19.0', 'google-auth-library-oauth2-http-1.4.0', 'reflectasm-1.07', 'software-and-algorithms-1.0', 'zstd-jni-1.4.3-1', 'kryo-2.21', 'junit-quickcheck-generators-0.8', 'google-auth-library-credentials-1.4.0', 'protobuf-java-util-3.19.3', 'protobuf-java-3.19.3', 'classgraph-4.8.104', 'grpc-context-1.44.0', 'grpc-alts-1.44.0', 'grpc-protobuf-1.44.0', 'perfmark-api-0.23.0', 'junit-dep-4.11', 'minlog-1.2', 'zstd-jni-1.5.2-1', 'junit-quickcheck-core-0.8', 'checker-compat-qual-2.5.3', 'grpc-core-1.44.0', 'grpc-api-1.44.0', 'grpc-protobuf-lite-1.44.0', 'pcollections-2.1.2', 'hamcrest-2.1', 'slf4j-jdk14-1.7.30', 'duct-tape-1.0.8', 'database-commons-1.16.3', 'mysql-1.16.3', 'checker-compat-qual-2.5.5', 'grpc-grpclb-1.44.0', 'grpc-auth-1.44.0', 'grpc-stub-1.44.0', 'slf4j-simple-1.7.30', 'slf4j-api-1.7.30', 'jdbc-1.16.3', 'postgresql-1.16.3', 'kafka-1.16.3']
ERROR:root:**************************************** Licenses were not able to be pulled automatically for some dependencies. Please search source code of the dependencies on the internet and add "license" and "notice" (if available) field to <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17/ws/src/sdks/java/container/license_scripts/dep_urls_java.yaml> for each missing license. Dependency List: [checker-compat-qual-2.5.3,checker-compat-qual-2.5.5,classgraph-4.8.104,database-commons-1.16.3,duct-tape-1.0.8,google-auth-library-credentials-1.4.0,google-auth-library-oauth2-http-1.4.0,grpc-alts-1.44.0,grpc-api-1.44.0,grpc-auth-1.44.0,grpc-context-1.44.0,grpc-core-1.44.0,grpc-grpclb-1.44.0,grpc-protobuf-1.44.0,grpc-protobuf-lite-1.44.0,grpc-stub-1.44.0,hamcrest-2.1,jdbc-1.16.3,junit-dep-4.11,junit-quickcheck-core-0.8,junit-quickcheck-generators-0.8,kafka-1.16.3,kryo-2.21,minlog-1.2,mysql-1.16.3,pcollections-2.1.2,perfmark-api-0.23.0,postgresql-1.16.3,protobuf-java-3.19.3,protobuf-java-util-3.19.3,reflectasm-1.07,slf4j-api-1.7.30,slf4j-jdk14-1.7.30,slf4j-simple-1.7.30,software-and-algorithms-1.0,system-rules-1.19.0,zstd-jni-1.4.3-1,zstd-jni-1.5.2-1]
INFO:root:pull_licenses_java.py failed. It took 162.359743 seconds with 16 threads.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17/ws/src/sdks/java/container/license_scripts/pull_licenses_java.py",> line 321, in <module>
raise RuntimeError('{n} error(s) occurred.'.format(n=len(error_msg)),
RuntimeError: ('1 error(s) occurred.', ['**************************************** Licenses were not able to be pulled automatically for some dependencies. Please search source code of the dependencies on the internet and add "license" and "notice" (if available) field to <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17/ws/src/sdks/java/container/license_scripts/dep_urls_java.yaml> for each missing license. Dependency List: [checker-compat-qual-2.5.3,checker-compat-qual-2.5.5,classgraph-4.8.104,database-commons-1.16.3,duct-tape-1.0.8,google-auth-library-credentials-1.4.0,google-auth-library-oauth2-http-1.4.0,grpc-alts-1.44.0,grpc-api-1.44.0,grpc-auth-1.44.0,grpc-context-1.44.0,grpc-core-1.44.0,grpc-grpclb-1.44.0,grpc-protobuf-1.44.0,grpc-protobuf-lite-1.44.0,grpc-stub-1.44.0,hamcrest-2.1,jdbc-1.16.3,junit-dep-4.11,junit-quickcheck-core-0.8,junit-quickcheck-generators-0.8,kafka-1.16.3,kryo-2.21,minlog-1.2,mysql-1.16.3,pcollections-2.1.2,perfmark-api-0.23.0,postgresql-1.16.3,protobuf-java-3.19.3,protobuf-java-util-3.19.3,reflectasm-1.07,slf4j-api-1.7.30,slf4j-jdk14-1.7.30,slf4j-simple-1.7.30,software-and-algorithms-1.0,system-rules-1.19.0,zstd-jni-1.4.3-1,zstd-jni-1.5.2-1]'])
> Task :sdks:java:container:pullLicenses FAILED
> Task :sdks:java:container:goPrepare UP-TO-DATE
> Task :sdks:java:container:goBuild
/home/jenkins/go/bin/go1.16.12 build -o ./build/target/linux_amd64/boot boot.go
> Task :sdks:java:container:java17:copySdkHarnessLauncher
Execution optimizations have been disabled for task ':sdks:java:container:java17:copySdkHarnessLauncher' to ensure correctness due to the following reasons:
- Gradle detected a problem with the following location: '<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17/ws/src/sdks/java/container/build/target'.> Reason: Task ':sdks:java:container:java17:copySdkHarnessLauncher' uses this output of task ':sdks:java:container:downloadCloudProfilerAgent' without declaring an explicit or implicit dependency. This can lead to incorrect results being produced, depending on what order the tasks are executed. Please refer to https://docs.gradle.org/7.4/userguide/validation_problems.html#implicit_dependency for more details about this problem.
- Gradle detected a problem with the following location: '<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17/ws/src/sdks/java/container/build/target'.> Reason: Task ':sdks:java:container:java17:copySdkHarnessLauncher' uses this output of task ':sdks:java:container:pullLicenses' without declaring an explicit or implicit dependency. This can lead to incorrect results being produced, depending on what order the tasks are executed. Please refer to https://docs.gradle.org/7.4/userguide/validation_problems.html#implicit_dependency for more details about this problem.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:container:pullLicenses'.
> Process 'command './license_scripts/license_script.sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.
BUILD FAILED in 3m 15s
104 actionable tasks: 67 executed, 33 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/om7opgkgr7qm2
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17 #132
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17/132/display/redirect?page=changes>
Changes:
[noreply] Revert "Improvement to Seed job configuration to launch against PRs
[ilion.beyst] Minor: fix typo
[noreply] Merge pull request #17422 from [BEAM-14344]: remove tracing from
------------------------------------------
[...truncated 384.85 KB...]
return opener.open(url, data, timeout)
File "/usr/lib/python3.8/urllib/request.py", line 531, in open
response = meth(req, response)
File "/usr/lib/python3.8/urllib/request.py", line 640, in http_response
response = self.parent.error(
File "/usr/lib/python3.8/urllib/request.py", line 569, in error
return self._call_chain(*args)
File "/usr/lib/python3.8/urllib/request.py", line 502, in _call_chain
result = func(*args)
File "/usr/lib/python3.8/urllib/request.py", line 649, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR:root:Invalid url for grpc-stub-1.44.0: https://opensource.org/licenses/Apache-2.0 after 9 retries.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17/ws/src/sdks/java/container/license_scripts/pull_licenses_java.py",> line 58, in pull_from_url
url_read = urlopen(url)
File "/usr/lib/python3.8/urllib/request.py", line 222, in urlopen
return opener.open(url, data, timeout)
File "/usr/lib/python3.8/urllib/request.py", line 531, in open
response = meth(req, response)
File "/usr/lib/python3.8/urllib/request.py", line 640, in http_response
response = self.parent.error(
File "/usr/lib/python3.8/urllib/request.py", line 569, in error
return self._call_chain(*args)
File "/usr/lib/python3.8/urllib/request.py", line 502, in _call_chain
result = func(*args)
File "/usr/lib/python3.8/urllib/request.py", line 649, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR:root:Invalid url for slf4j-simple-1.7.30: http://www.opensource.org/licenses/mit-license.php after 9 retries.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17/ws/src/sdks/java/container/license_scripts/pull_licenses_java.py",> line 58, in pull_from_url
url_read = urlopen(url)
File "/usr/lib/python3.8/urllib/request.py", line 222, in urlopen
return opener.open(url, data, timeout)
File "/usr/lib/python3.8/urllib/request.py", line 531, in open
response = meth(req, response)
File "/usr/lib/python3.8/urllib/request.py", line 640, in http_response
response = self.parent.error(
File "/usr/lib/python3.8/urllib/request.py", line 569, in error
return self._call_chain(*args)
File "/usr/lib/python3.8/urllib/request.py", line 502, in _call_chain
result = func(*args)
File "/usr/lib/python3.8/urllib/request.py", line 649, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR:root:Invalid url for kafka-1.16.3: http://opensource.org/licenses/MIT. Retrying...
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17/ws/src/sdks/java/container/license_scripts/pull_licenses_java.py",> line 58, in pull_from_url
url_read = urlopen(url)
File "/usr/lib/python3.8/urllib/request.py", line 222, in urlopen
return opener.open(url, data, timeout)
File "/usr/lib/python3.8/urllib/request.py", line 531, in open
response = meth(req, response)
File "/usr/lib/python3.8/urllib/request.py", line 640, in http_response
response = self.parent.error(
File "/usr/lib/python3.8/urllib/request.py", line 569, in error
return self._call_chain(*args)
File "/usr/lib/python3.8/urllib/request.py", line 502, in _call_chain
result = func(*args)
File "/usr/lib/python3.8/urllib/request.py", line 649, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR:root:Invalid url for postgresql-1.16.3: http://opensource.org/licenses/MIT after 9 retries.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17/ws/src/sdks/java/container/license_scripts/pull_licenses_java.py",> line 58, in pull_from_url
url_read = urlopen(url)
File "/usr/lib/python3.8/urllib/request.py", line 222, in urlopen
return opener.open(url, data, timeout)
File "/usr/lib/python3.8/urllib/request.py", line 531, in open
response = meth(req, response)
File "/usr/lib/python3.8/urllib/request.py", line 640, in http_response
response = self.parent.error(
File "/usr/lib/python3.8/urllib/request.py", line 569, in error
return self._call_chain(*args)
File "/usr/lib/python3.8/urllib/request.py", line 502, in _call_chain
result = func(*args)
File "/usr/lib/python3.8/urllib/request.py", line 649, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR:root:Invalid url for kafka-1.16.3: http://opensource.org/licenses/MIT. Retrying...
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17/ws/src/sdks/java/container/license_scripts/pull_licenses_java.py",> line 58, in pull_from_url
url_read = urlopen(url)
File "/usr/lib/python3.8/urllib/request.py", line 222, in urlopen
return opener.open(url, data, timeout)
File "/usr/lib/python3.8/urllib/request.py", line 531, in open
response = meth(req, response)
File "/usr/lib/python3.8/urllib/request.py", line 640, in http_response
response = self.parent.error(
File "/usr/lib/python3.8/urllib/request.py", line 569, in error
return self._call_chain(*args)
File "/usr/lib/python3.8/urllib/request.py", line 502, in _call_chain
result = func(*args)
File "/usr/lib/python3.8/urllib/request.py", line 649, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR:root:Invalid url for kafka-1.16.3: http://opensource.org/licenses/MIT. Retrying...
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17/ws/src/sdks/java/container/license_scripts/pull_licenses_java.py",> line 58, in pull_from_url
url_read = urlopen(url)
File "/usr/lib/python3.8/urllib/request.py", line 222, in urlopen
return opener.open(url, data, timeout)
File "/usr/lib/python3.8/urllib/request.py", line 531, in open
response = meth(req, response)
File "/usr/lib/python3.8/urllib/request.py", line 640, in http_response
response = self.parent.error(
File "/usr/lib/python3.8/urllib/request.py", line 569, in error
return self._call_chain(*args)
File "/usr/lib/python3.8/urllib/request.py", line 502, in _call_chain
result = func(*args)
File "/usr/lib/python3.8/urllib/request.py", line 649, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR:root:Invalid url for kafka-1.16.3: http://opensource.org/licenses/MIT. Retrying...
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17/ws/src/sdks/java/container/license_scripts/pull_licenses_java.py",> line 58, in pull_from_url
url_read = urlopen(url)
File "/usr/lib/python3.8/urllib/request.py", line 222, in urlopen
return opener.open(url, data, timeout)
File "/usr/lib/python3.8/urllib/request.py", line 531, in open
response = meth(req, response)
File "/usr/lib/python3.8/urllib/request.py", line 640, in http_response
response = self.parent.error(
File "/usr/lib/python3.8/urllib/request.py", line 569, in error
return self._call_chain(*args)
File "/usr/lib/python3.8/urllib/request.py", line 502, in _call_chain
result = func(*args)
File "/usr/lib/python3.8/urllib/request.py", line 649, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR:root:Invalid url for kafka-1.16.3: http://opensource.org/licenses/MIT. Retrying...
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17/ws/src/sdks/java/container/license_scripts/pull_licenses_java.py",> line 58, in pull_from_url
url_read = urlopen(url)
File "/usr/lib/python3.8/urllib/request.py", line 222, in urlopen
return opener.open(url, data, timeout)
File "/usr/lib/python3.8/urllib/request.py", line 531, in open
response = meth(req, response)
File "/usr/lib/python3.8/urllib/request.py", line 640, in http_response
response = self.parent.error(
File "/usr/lib/python3.8/urllib/request.py", line 569, in error
return self._call_chain(*args)
File "/usr/lib/python3.8/urllib/request.py", line 502, in _call_chain
result = func(*args)
File "/usr/lib/python3.8/urllib/request.py", line 649, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR:root:Invalid url for kafka-1.16.3: http://opensource.org/licenses/MIT. Retrying...
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17/ws/src/sdks/java/container/license_scripts/pull_licenses_java.py",> line 58, in pull_from_url
url_read = urlopen(url)
File "/usr/lib/python3.8/urllib/request.py", line 222, in urlopen
return opener.open(url, data, timeout)
File "/usr/lib/python3.8/urllib/request.py", line 531, in open
response = meth(req, response)
File "/usr/lib/python3.8/urllib/request.py", line 640, in http_response
response = self.parent.error(
File "/usr/lib/python3.8/urllib/request.py", line 569, in error
return self._call_chain(*args)
File "/usr/lib/python3.8/urllib/request.py", line 502, in _call_chain
result = func(*args)
File "/usr/lib/python3.8/urllib/request.py", line 649, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR:root:Invalid url for kafka-1.16.3: http://opensource.org/licenses/MIT. Retrying...
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17/ws/src/sdks/java/container/license_scripts/pull_licenses_java.py",> line 58, in pull_from_url
url_read = urlopen(url)
File "/usr/lib/python3.8/urllib/request.py", line 222, in urlopen
return opener.open(url, data, timeout)
File "/usr/lib/python3.8/urllib/request.py", line 531, in open
response = meth(req, response)
File "/usr/lib/python3.8/urllib/request.py", line 640, in http_response
response = self.parent.error(
File "/usr/lib/python3.8/urllib/request.py", line 569, in error
return self._call_chain(*args)
File "/usr/lib/python3.8/urllib/request.py", line 502, in _call_chain
result = func(*args)
File "/usr/lib/python3.8/urllib/request.py", line 649, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR:root:Invalid url for kafka-1.16.3: http://opensource.org/licenses/MIT. Retrying...
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17/ws/src/sdks/java/container/license_scripts/pull_licenses_java.py",> line 58, in pull_from_url
url_read = urlopen(url)
File "/usr/lib/python3.8/urllib/request.py", line 222, in urlopen
return opener.open(url, data, timeout)
File "/usr/lib/python3.8/urllib/request.py", line 531, in open
response = meth(req, response)
File "/usr/lib/python3.8/urllib/request.py", line 640, in http_response
response = self.parent.error(
File "/usr/lib/python3.8/urllib/request.py", line 569, in error
return self._call_chain(*args)
File "/usr/lib/python3.8/urllib/request.py", line 502, in _call_chain
result = func(*args)
File "/usr/lib/python3.8/urllib/request.py", line 649, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR:root:Invalid url for kafka-1.16.3: http://opensource.org/licenses/MIT after 9 retries.
ERROR:root:['google-auth-library-oauth2-http-1.4.0', 'reflectasm-1.07', 'system-rules-1.19.0', 'zstd-jni-1.4.3-1', 'software-and-algorithms-1.0', 'google-auth-library-credentials-1.4.0', 'kryo-2.21', 'junit-quickcheck-generators-0.8', 'protobuf-java-util-3.19.3', 'classgraph-4.8.104', 'protobuf-java-3.19.3', 'grpc-context-1.44.0', 'grpc-alts-1.44.0', 'grpc-protobuf-1.44.0', 'perfmark-api-0.23.0', 'junit-dep-4.11', 'minlog-1.2', 'zstd-jni-1.5.2-1', 'junit-quickcheck-core-0.8', 'checker-compat-qual-2.5.3', 'grpc-core-1.44.0', 'grpc-api-1.44.0', 'grpc-protobuf-lite-1.44.0', 'pcollections-2.1.2', 'duct-tape-1.0.8', 'hamcrest-2.1', 'database-commons-1.16.3', 'slf4j-jdk14-1.7.30', 'mysql-1.16.3', 'checker-compat-qual-2.5.5', 'grpc-grpclb-1.44.0', 'slf4j-api-1.7.30', 'grpc-auth-1.44.0', 'jdbc-1.16.3', 'grpc-stub-1.44.0', 'slf4j-simple-1.7.30', 'postgresql-1.16.3', 'kafka-1.16.3']
ERROR:root:**************************************** Licenses were not able to be pulled automatically for some dependencies. Please search source code of the dependencies on the internet and add "license" and "notice" (if available) field to <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17/ws/src/sdks/java/container/license_scripts/dep_urls_java.yaml> for each missing license. Dependency List: [checker-compat-qual-2.5.3,checker-compat-qual-2.5.5,classgraph-4.8.104,database-commons-1.16.3,duct-tape-1.0.8,google-auth-library-credentials-1.4.0,google-auth-library-oauth2-http-1.4.0,grpc-alts-1.44.0,grpc-api-1.44.0,grpc-auth-1.44.0,grpc-context-1.44.0,grpc-core-1.44.0,grpc-grpclb-1.44.0,grpc-protobuf-1.44.0,grpc-protobuf-lite-1.44.0,grpc-stub-1.44.0,hamcrest-2.1,jdbc-1.16.3,junit-dep-4.11,junit-quickcheck-core-0.8,junit-quickcheck-generators-0.8,kafka-1.16.3,kryo-2.21,minlog-1.2,mysql-1.16.3,pcollections-2.1.2,perfmark-api-0.23.0,postgresql-1.16.3,protobuf-java-3.19.3,protobuf-java-util-3.19.3,reflectasm-1.07,slf4j-api-1.7.30,slf4j-jdk14-1.7.30,slf4j-simple-1.7.30,software-and-algorithms-1.0,system-rules-1.19.0,zstd-jni-1.4.3-1,zstd-jni-1.5.2-1]
INFO:root:pull_licenses_java.py failed. It took 162.258068 seconds with 16 threads.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17/ws/src/sdks/java/container/license_scripts/pull_licenses_java.py",> line 321, in <module>
raise RuntimeError('{n} error(s) occurred.'.format(n=len(error_msg)),
RuntimeError: ('1 error(s) occurred.', ['**************************************** Licenses were not able to be pulled automatically for some dependencies. Please search source code of the dependencies on the internet and add "license" and "notice" (if available) field to <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17/ws/src/sdks/java/container/license_scripts/dep_urls_java.yaml> for each missing license. Dependency List: [checker-compat-qual-2.5.3,checker-compat-qual-2.5.5,classgraph-4.8.104,database-commons-1.16.3,duct-tape-1.0.8,google-auth-library-credentials-1.4.0,google-auth-library-oauth2-http-1.4.0,grpc-alts-1.44.0,grpc-api-1.44.0,grpc-auth-1.44.0,grpc-context-1.44.0,grpc-core-1.44.0,grpc-grpclb-1.44.0,grpc-protobuf-1.44.0,grpc-protobuf-lite-1.44.0,grpc-stub-1.44.0,hamcrest-2.1,jdbc-1.16.3,junit-dep-4.11,junit-quickcheck-core-0.8,junit-quickcheck-generators-0.8,kafka-1.16.3,kryo-2.21,minlog-1.2,mysql-1.16.3,pcollections-2.1.2,perfmark-api-0.23.0,postgresql-1.16.3,protobuf-java-3.19.3,protobuf-java-util-3.19.3,reflectasm-1.07,slf4j-api-1.7.30,slf4j-jdk14-1.7.30,slf4j-simple-1.7.30,software-and-algorithms-1.0,system-rules-1.19.0,zstd-jni-1.4.3-1,zstd-jni-1.5.2-1]'])
> Task :sdks:java:container:pullLicenses FAILED
> Task :sdks:java:container:goPrepare UP-TO-DATE
> Task :sdks:java:container:goBuild
/home/jenkins/go/bin/go1.16.12 build -o ./build/target/linux_amd64/boot boot.go
> Task :sdks:java:container:java17:copySdkHarnessLauncher
Execution optimizations have been disabled for task ':sdks:java:container:java17:copySdkHarnessLauncher' to ensure correctness due to the following reasons:
- Gradle detected a problem with the following location: '<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17/ws/src/sdks/java/container/build/target'.> Reason: Task ':sdks:java:container:java17:copySdkHarnessLauncher' uses this output of task ':sdks:java:container:downloadCloudProfilerAgent' without declaring an explicit or implicit dependency. This can lead to incorrect results being produced, depending on what order the tasks are executed. Please refer to https://docs.gradle.org/7.4/userguide/validation_problems.html#implicit_dependency for more details about this problem.
- Gradle detected a problem with the following location: '<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17/ws/src/sdks/java/container/build/target'.> Reason: Task ':sdks:java:container:java17:copySdkHarnessLauncher' uses this output of task ':sdks:java:container:pullLicenses' without declaring an explicit or implicit dependency. This can lead to incorrect results being produced, depending on what order the tasks are executed. Please refer to https://docs.gradle.org/7.4/userguide/validation_problems.html#implicit_dependency for more details about this problem.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:container:pullLicenses'.
> Process 'command './license_scripts/license_script.sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.
BUILD FAILED in 3m 15s
104 actionable tasks: 68 executed, 32 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/fsmqg7svzemh4
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17 #131
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17/131/display/redirect?page=changes>
Changes:
[Robert Bradshaw] Add element weighting parameter to BatchElements.
[Robert Bradshaw] Clearer test.
[noreply] Revert "Merge pull request #17260 from [BEAM-13229] [Website] bug side
[noreply] [BEAM-14001] Add missing test cases to existing suites in exec package
[noreply] [BEAM-14243] Add staticcheck to Github Actions Precommits (#17479)
[noreply] [BEAM-14368][BEAM-13984]Change model loading from constructor to
[noreply] [BEAM-13983] changed file name from sklearn_loader to sklearn_inference
[noreply] Add SQL in Notebooks blog post (#17481)
[noreply] Merge pull request #17404: [BEAM-13990] support date and timestamp
------------------------------------------
[...truncated 48.53 KB...]
b66a2701ea27: Preparing
3bc383470c05: Preparing
69c1338cdef4: Waiting
e93827457889: Preparing
3810379bedde: Waiting
2b0de254a7c2: Waiting
08fa02ce37eb: Preparing
a037458de4e0: Preparing
b1d971149db0: Waiting
b66a2701ea27: Waiting
cba591b85723: Waiting
bafdbe68e4ae: Preparing
a13c519c6361: Preparing
3bc383470c05: Waiting
a13c519c6361: Waiting
bb84462952c7: Waiting
52766a5bccea: Waiting
da40c4782e7f: Waiting
bafdbe68e4ae: Waiting
a037458de4e0: Waiting
872a42936a09: Pushed
c32361a1a860: Pushed
e7c6c95e5083: Pushed
e9f9a1f648fa: Pushed
65541ee2f3b3: Pushed
69c1338cdef4: Pushed
69042358dd56: Pushed
3810379bedde: Pushed
52766a5bccea: Pushed
2b0de254a7c2: Pushed
cba591b85723: Pushed
b1d971149db0: Pushed
3bc383470c05: Layer already exists
e93827457889: Layer already exists
08fa02ce37eb: Layer already exists
a037458de4e0: Layer already exists
bafdbe68e4ae: Layer already exists
a13c519c6361: Layer already exists
bb84462952c7: Pushed
b66a2701ea27: Pushed
da40c4782e7f: Pushed
20220430123036: digest: sha256:0e47d17ca7537760a8c2dbd9549153d80ea0d48c42799e5647dcbcee5d1d52fc size: 4729
> Task :sdks:java:testing:load-tests:run
Apr 30, 2022 12:32:34 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Apr 30, 2022 12:32:35 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 210 files. Enable logging at DEBUG level to see which files will be staged.
Apr 30, 2022 12:32:36 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Apr 30, 2022 12:32:36 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 30, 2022 12:32:38 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 210 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 30, 2022 12:32:39 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 210 files cached, 0 files newly uploaded in 0 seconds
Apr 30, 2022 12:32:39 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 30, 2022 12:32:39 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <116094 bytes, hash 4491a2b601e674059ec8006d689c509ec298af1a0439951d10b800e6dd010454> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-RJGitgHmdAWeyABtaJxQnsKYrxoEOZUdELgA5t0BBFQ.pb
Apr 30, 2022 12:32:40 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Apr 30, 2022 12:32:40 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@d5af0a5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5981f4a6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@63dfada0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6f231ced, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@35a60674, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@63d4f0a2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7d78f3d5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@a4b5ce3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@f5b6e78, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4b4eced1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@71926a36, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@216e9ca3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@75120e58, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@48976e6d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2a367e93, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7f6874f2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1a6dc589, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@697a34af, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@70211df5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4c5228e7]
Apr 30, 2022 12:32:40 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Apr 30, 2022 12:32:40 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Apr 30, 2022 12:32:40 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Apr 30, 2022 12:32:40 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Apr 30, 2022 12:32:40 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b81616b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@15d42ccb, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@279dd959, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@46383a78, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@36c281ed, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@244418a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4b5a078a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4c361f63, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6ed922e1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4eb166a1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@554c4eaa, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@29fd8e67, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@e146f93, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4bd5849e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7cdbaa50, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@39909d1a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1455154c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7343922c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@526b2f3e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6f2e1024]
Apr 30, 2022 12:32:40 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Apr 30, 2022 12:32:40 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Apr 30, 2022 12:32:40 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Apr 30, 2022 12:32:40 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Apr 30, 2022 12:32:40 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Apr 30, 2022 12:32:40 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Apr 30, 2022 12:32:40 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Apr 30, 2022 12:32:40 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Apr 30, 2022 12:32:40 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Apr 30, 2022 12:32:40 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Apr 30, 2022 12:32:40 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Apr 30, 2022 12:32:41 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.39.0-SNAPSHOT
Apr 30, 2022 12:32:41 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-04-30_05_32_41-17059695585751445620?project=apache-beam-testing
Apr 30, 2022 12:32:41 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2022-04-30_05_32_41-17059695585751445620
Apr 30, 2022 12:32:41 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2022-04-30_05_32_41-17059695585751445620
Apr 30, 2022 12:32:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2022-04-30T12:32:52.695Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java170dataflow0v20streaming0cogbk01-jenkins-04-ltel. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Apr 30, 2022 12:32:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-30T12:32:59.118Z: Worker configuration: e2-standard-2 in us-central1-b.
Apr 30, 2022 12:33:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-30T12:33:00.085Z: Expanding SplittableParDo operations into optimizable parts.
Apr 30, 2022 12:33:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-30T12:33:00.133Z: Expanding CollectionToSingleton operations into optimizable parts.
Apr 30, 2022 12:33:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-30T12:33:00.242Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 30, 2022 12:33:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-30T12:33:00.391Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Apr 30, 2022 12:33:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-30T12:33:00.462Z: Expanding GroupByKey operations into streaming Read/Write steps
Apr 30, 2022 12:33:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-30T12:33:00.610Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 30, 2022 12:33:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-30T12:33:00.820Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 30, 2022 12:33:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-30T12:33:00.866Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Apr 30, 2022 12:33:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-30T12:33:00.931Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Apr 30, 2022 12:33:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-30T12:33:00.975Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Apr 30, 2022 12:33:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-30T12:33:01.008Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Apr 30, 2022 12:33:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-30T12:33:01.075Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Apr 30, 2022 12:33:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-30T12:33:01.139Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Apr 30, 2022 12:33:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-30T12:33:01.179Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Apr 30, 2022 12:33:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-30T12:33:01.240Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Apr 30, 2022 12:33:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-30T12:33:01.283Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Apr 30, 2022 12:33:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-30T12:33:01.310Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Apr 30, 2022 12:33:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-30T12:33:01.353Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Apr 30, 2022 12:33:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-30T12:33:01.400Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Apr 30, 2022 12:33:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-30T12:33:01.461Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Apr 30, 2022 12:33:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-30T12:33:01.498Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Apr 30, 2022 12:33:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-30T12:33:01.544Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Apr 30, 2022 12:33:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-30T12:33:01.580Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Apr 30, 2022 12:33:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-30T12:33:01.649Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Apr 30, 2022 12:33:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-30T12:33:01.701Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Apr 30, 2022 12:33:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-30T12:33:01.764Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Apr 30, 2022 12:33:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-30T12:33:01.804Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Apr 30, 2022 12:33:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-30T12:33:01.860Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Apr 30, 2022 12:33:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-30T12:33:01.897Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Apr 30, 2022 12:33:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-30T12:33:02.138Z: Running job using Streaming Engine
Apr 30, 2022 12:33:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-30T12:33:02.452Z: Starting 5 ****s in us-central1-b...
Apr 30, 2022 12:33:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-30T12:33:25.346Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 30, 2022 12:33:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-30T12:33:29.128Z: Autoscaling: Raised the number of ****s to 4 so that the pipeline can catch up with its backlog and keep up with its input rate.
Apr 30, 2022 12:33:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-30T12:33:29.181Z: Resized **** pool to 4, though goal was 5. This could be a quota issue.
Apr 30, 2022 12:33:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-30T12:33:39.423Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Apr 30, 2022 12:34:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-30T12:34:37.924Z: Workers have started successfully.
Apr 30, 2022 4:01:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-30T16:00:58.753Z: Cancel request is committed for workflow job: 2022-04-30_05_32_41-17059695585751445620.
Apr 30, 2022 4:01:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-30T16:00:58.828Z: Cleaning up.
Apr 30, 2022 4:01:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-30T16:00:58.936Z: Stopping **** pool...
Apr 30, 2022 4:01:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-30T16:00:58.989Z: Stopping **** pool...
Apr 30, 2022 4:01:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-30T16:01:34.198Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Apr 30, 2022 4:01:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-30T16:01:34.235Z: Worker pool stopped.
Apr 30, 2022 4:01:41 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2022-04-30_05_32_41-17059695585751445620 finished with status CANCELLED.
Load test results for test (ID): 772bdddf-7839-4d9d-b5d6-a0269f657b3b and timestamp: 2022-04-30T12:32:35.683000000Z:
Metric: Value:
dataflow_v2_java17_runtime_sec 12358.036
dataflow_v2_java17_total_bytes_count 3.44843553E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)
> Task :sdks:java:testing:load-tests:run FAILED
> Task :runners:google-cloud-dataflow-java:cleanUpDockerJavaImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220430123036
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:0e47d17ca7537760a8c2dbd9549153d80ea0d48c42799e5647dcbcee5d1d52fc
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220430123036]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:0e47d17ca7537760a8c2dbd9549153d80ea0d48c42799e5647dcbcee5d1d52fc]
Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220430123036] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:0e47d17ca7537760a8c2dbd9549153d80ea0d48c42799e5647dcbcee5d1d52fc])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:0e47d17ca7537760a8c2dbd9549153d80ea0d48c42799e5647dcbcee5d1d52fc
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:0e47d17ca7537760a8c2dbd9549153d80ea0d48c42799e5647dcbcee5d1d52fc
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:0e47d17ca7537760a8c2dbd9549153d80ea0d48c42799e5647dcbcee5d1d52fc].
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.
BUILD FAILED in 3h 31m 25s
110 actionable tasks: 74 executed, 32 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/ey4p62jcnji2q
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17 #130
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17/130/display/redirect?page=changes>
Changes:
[ihr] Update Java katas to Beam 2.38
[noreply] [BEAM-14369] Fix "target/options: no such file or directory" error while
[noreply] [BEAM-14297] Enable nullable key and value arrays for xlang kafka io
[noreply] Merge pull request #17444 from [BEAM-14310] [Website] bug home
[noreply] Merge pull request #17388 from [BEAM-14311] [Website] Home Page
[noreply] [BEAM-14376] Typo in method description doc
[noreply] Add default classpath when not present (#17491)
[thiagotnunes] fix: update javadocs for ChangeStreamMetrics
[noreply] Merge pull request #17443 from [BEAM-12164]: use the end timestamp for
[noreply] Merge pull request #17260 from [BEAM-13229] [Website] bug side nav
[noreply] [BEAM-14351] Fix the template and move the announcement to the next
------------------------------------------
[...truncated 50.05 KB...]
1a6c66a7e504: Preparing
9f2fba01df7f: Preparing
939d4a0c2578: Preparing
10bfe1d713b4: Preparing
3bc383470c05: Preparing
e93827457889: Preparing
08fa02ce37eb: Preparing
a037458de4e0: Preparing
bafdbe68e4ae: Preparing
a13c519c6361: Preparing
e93827457889: Waiting
3bc383470c05: Waiting
08fa02ce37eb: Waiting
1a6c66a7e504: Waiting
a037458de4e0: Waiting
9f2fba01df7f: Waiting
bafdbe68e4ae: Waiting
939d4a0c2578: Waiting
a13c519c6361: Waiting
53913aed19d2: Waiting
10bfe1d713b4: Waiting
7a763751f65a: Waiting
f30932317082: Waiting
c3f9929bdbbf: Waiting
89905dd1bd7f: Pushed
4e895253da30: Pushed
df122f2a33c4: Pushed
ad33fb1004d8: Pushed
f7689c509955: Pushed
53913aed19d2: Pushed
f30932317082: Pushed
e2ecd49ff6ec: Pushed
1a6c66a7e504: Pushed
86aa7f9706f2: Pushed
7a763751f65a: Pushed
3bc383470c05: Layer already exists
c3f9929bdbbf: Pushed
e93827457889: Layer already exists
08fa02ce37eb: Layer already exists
a037458de4e0: Layer already exists
bafdbe68e4ae: Layer already exists
a13c519c6361: Layer already exists
939d4a0c2578: Pushed
10bfe1d713b4: Pushed
9f2fba01df7f: Pushed
20220429123038: digest: sha256:0eff4209fd3bba2527bb7ad5bf3d3235bf96dd181813188181b9303e3ea58608 size: 4729
> Task :sdks:java:testing:load-tests:run
Apr 29, 2022 12:32:38 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Apr 29, 2022 12:32:39 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 208 files. Enable logging at DEBUG level to see which files will be staged.
Apr 29, 2022 12:32:39 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Apr 29, 2022 12:32:39 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 29, 2022 12:32:42 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 208 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 29, 2022 12:32:42 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 208 files cached, 0 files newly uploaded in 0 seconds
Apr 29, 2022 12:32:42 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 29, 2022 12:32:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <115405 bytes, hash b28ad4e8e1c0fbba6f24af826606c4176dfe00c843d23c387855f37925ad9698> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-sorU6OHA-7pvJK-CZgbEF23-AMhD0jw4eFXzeSWtlpg.pb
Apr 29, 2022 12:32:44 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Apr 29, 2022 12:32:44 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@57272109, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@59696551, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@648d0e6d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@79e66b2f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@17273273, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5f69e2b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@984169e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43f1bb92, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6d6bbd35, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5c5d6175, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7544ac86, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3b27b497, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@b1534d3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c74aa0d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6c841199, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6a818392, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@489091bd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@512d6e60, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1de9b505, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b122839]
Apr 29, 2022 12:32:44 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Apr 29, 2022 12:32:44 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Apr 29, 2022 12:32:44 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Apr 29, 2022 12:32:44 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Apr 29, 2022 12:32:44 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6a6f6c7e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6c5ddccd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1dbd580, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6c101cc1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7d0d91a1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7fb48179, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@201c3cda, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4c86da0c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5d97caa4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6732726, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@474821de, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3d64c581, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5ec5ea63, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4190bc8a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@47d023b7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5c83ae01, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2d64c100, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@69d45cca, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2fdf17dc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6e6d4780]
Apr 29, 2022 12:32:44 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Apr 29, 2022 12:32:44 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Apr 29, 2022 12:32:44 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Apr 29, 2022 12:32:44 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Apr 29, 2022 12:32:44 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Apr 29, 2022 12:32:44 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Apr 29, 2022 12:32:44 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Apr 29, 2022 12:32:44 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Apr 29, 2022 12:32:44 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Apr 29, 2022 12:32:44 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Apr 29, 2022 12:32:44 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Apr 29, 2022 12:32:44 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.39.0-SNAPSHOT
Apr 29, 2022 12:32:45 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-04-29_05_32_44-9382013507233506701?project=apache-beam-testing
Apr 29, 2022 12:32:45 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2022-04-29_05_32_44-9382013507233506701
Apr 29, 2022 12:32:45 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2022-04-29_05_32_44-9382013507233506701
Apr 29, 2022 12:32:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2022-04-29T12:32:50.482Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java170dataflow0v20streaming0cogbk01-jenkins-04-3d59. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Apr 29, 2022 12:32:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-29T12:32:57.076Z: Worker configuration: e2-standard-2 in us-central1-b.
Apr 29, 2022 12:32:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-29T12:32:57.618Z: Expanding SplittableParDo operations into optimizable parts.
Apr 29, 2022 12:32:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-29T12:32:57.643Z: Expanding CollectionToSingleton operations into optimizable parts.
Apr 29, 2022 12:32:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-29T12:32:57.715Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 29, 2022 12:32:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-29T12:32:57.776Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Apr 29, 2022 12:32:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-29T12:32:57.804Z: Expanding GroupByKey operations into streaming Read/Write steps
Apr 29, 2022 12:32:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-29T12:32:57.852Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 29, 2022 12:32:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-29T12:32:57.973Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 29, 2022 12:32:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-29T12:32:58.027Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Apr 29, 2022 12:32:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-29T12:32:58.056Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Apr 29, 2022 12:32:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-29T12:32:58.085Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Apr 29, 2022 12:32:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-29T12:32:58.109Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Apr 29, 2022 12:32:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-29T12:32:58.135Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Apr 29, 2022 12:32:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-29T12:32:58.169Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Apr 29, 2022 12:32:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-29T12:32:58.192Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Apr 29, 2022 12:32:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-29T12:32:58.215Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Apr 29, 2022 12:32:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-29T12:32:58.237Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Apr 29, 2022 12:32:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-29T12:32:58.261Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Apr 29, 2022 12:32:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-29T12:32:58.304Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Apr 29, 2022 12:32:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-29T12:32:58.336Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Apr 29, 2022 12:32:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-29T12:32:58.360Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Apr 29, 2022 12:32:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-29T12:32:58.388Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Apr 29, 2022 12:32:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-29T12:32:58.424Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Apr 29, 2022 12:32:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-29T12:32:58.447Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Apr 29, 2022 12:32:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-29T12:32:58.481Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Apr 29, 2022 12:32:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-29T12:32:58.507Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Apr 29, 2022 12:32:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-29T12:32:58.549Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Apr 29, 2022 12:32:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-29T12:32:58.578Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Apr 29, 2022 12:32:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-29T12:32:58.634Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Apr 29, 2022 12:32:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-29T12:32:58.664Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Apr 29, 2022 12:32:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-29T12:32:58.834Z: Running job using Streaming Engine
Apr 29, 2022 12:32:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-29T12:32:59.055Z: Starting 5 ****s in us-central1-b...
Apr 29, 2022 12:33:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-29T12:33:07.776Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 29, 2022 12:33:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-29T12:33:31.754Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Apr 29, 2022 12:34:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-29T12:34:35.673Z: Workers have started successfully.
Apr 29, 2022 4:01:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-29T16:01:02.934Z: Cancel request is committed for workflow job: 2022-04-29_05_32_44-9382013507233506701.
Apr 29, 2022 4:01:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-29T16:01:03.019Z: Cleaning up.
Apr 29, 2022 4:01:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-29T16:01:03.114Z: Stopping **** pool...
Apr 29, 2022 4:01:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-29T16:01:03.157Z: Stopping **** pool...
Apr 29, 2022 4:01:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-29T16:01:35.807Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Apr 29, 2022 4:01:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-29T16:01:35.840Z: Worker pool stopped.
Apr 29, 2022 4:01:45 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2022-04-29_05_32_44-9382013507233506701 finished with status CANCELLED.
Load test results for test (ID): 47ad33e9-9c96-4a7d-a563-f53a45509e9b and timestamp: 2022-04-29T12:32:39.443000000Z:
Metric: Value:
dataflow_v2_java17_runtime_sec 12346.36
dataflow_v2_java17_total_bytes_count 3.8917995E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)
> Task :sdks:java:testing:load-tests:run FAILED
> Task :runners:google-cloud-dataflow-java:cleanUpDockerJavaImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220429123038
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:0eff4209fd3bba2527bb7ad5bf3d3235bf96dd181813188181b9303e3ea58608
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220429123038]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:0eff4209fd3bba2527bb7ad5bf3d3235bf96dd181813188181b9303e3ea58608]
Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220429123038] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:0eff4209fd3bba2527bb7ad5bf3d3235bf96dd181813188181b9303e3ea58608])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:0eff4209fd3bba2527bb7ad5bf3d3235bf96dd181813188181b9303e3ea58608
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:0eff4209fd3bba2527bb7ad5bf3d3235bf96dd181813188181b9303e3ea58608
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:0eff4209fd3bba2527bb7ad5bf3d3235bf96dd181813188181b9303e3ea58608].
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.
BUILD FAILED in 3h 32m 11s
110 actionable tasks: 73 executed, 33 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/j7urcfn26q5km
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17 #129
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17/129/display/redirect?page=changes>
Changes:
[noreply] [BEAM-11104] Add Checkpointing split to Go SDK (#17386)
[noreply] Merge pull request #17226 from [BEAM-14204] [Playground] Tests for
[noreply] [BEAM-13015, BEAM-14184] Address unbounded number of messages being
[noreply] Improvement to Seed job configuration to launch against PRs (#17468)
[noreply] [BEAM-13983] Small changes to sklearn runinference (#17459)
[chamikaramj] Renames ExternalPythonTransform to PythonExternalTransform
[noreply] [BEAM-14351] Inherit from Coder. (#17437)
------------------------------------------
[...truncated 46.64 KB...]
022c02dc48eb: Pushed
d1fb25c57111: Pushed
286332b2b2d6: Pushed
0ed3aae1b811: Pushed
61b45dd0fa4a: Pushed
2e38b96b808c: Pushed
7aaa05e8eee5: Pushed
8d69c2f825d4: Pushed
3bc383470c05: Layer already exists
e93827457889: Layer already exists
e394883f3a23: Pushed
08fa02ce37eb: Layer already exists
a037458de4e0: Layer already exists
e3c4e9c86913: Pushed
bafdbe68e4ae: Layer already exists
a13c519c6361: Layer already exists
c2e2f9b1b251: Pushed
1d53b7ecd1e9: Pushed
20220428123039: digest: sha256:e150bab5afd54e7eb0a9e2e9d2379ba06eab09426d968d7a5ad2e2b44cfcd34c size: 4729
> Task :sdks:java:testing:load-tests:run
Apr 28, 2022 12:32:26 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Apr 28, 2022 12:32:26 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 208 files. Enable logging at DEBUG level to see which files will be staged.
Apr 28, 2022 12:32:27 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Apr 28, 2022 12:32:27 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 28, 2022 12:32:29 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 208 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 28, 2022 12:32:30 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 208 files cached, 0 files newly uploaded in 0 seconds
Apr 28, 2022 12:32:30 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 28, 2022 12:32:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <115405 bytes, hash 447bac8dad8fd3432a03289c1dcc4ee41d1147d664d6de8d48db944d17c2003b> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-RHusja2P00MqAyicHcxO5B0RR9Zk1t6NSNuUTRfCADs.pb
Apr 28, 2022 12:32:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Apr 28, 2022 12:32:32 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@57272109, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@59696551, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@648d0e6d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@79e66b2f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@17273273, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5f69e2b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@984169e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43f1bb92, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6d6bbd35, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5c5d6175, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7544ac86, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3b27b497, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@b1534d3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c74aa0d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6c841199, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6a818392, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@489091bd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@512d6e60, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1de9b505, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b122839]
Apr 28, 2022 12:32:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Apr 28, 2022 12:32:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Apr 28, 2022 12:32:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Apr 28, 2022 12:32:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Apr 28, 2022 12:32:32 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6a6f6c7e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6c5ddccd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1dbd580, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6c101cc1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7d0d91a1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7fb48179, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@201c3cda, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4c86da0c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5d97caa4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6732726, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@474821de, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3d64c581, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5ec5ea63, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4190bc8a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@47d023b7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5c83ae01, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2d64c100, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@69d45cca, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2fdf17dc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6e6d4780]
Apr 28, 2022 12:32:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Apr 28, 2022 12:32:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Apr 28, 2022 12:32:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Apr 28, 2022 12:32:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Apr 28, 2022 12:32:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Apr 28, 2022 12:32:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Apr 28, 2022 12:32:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Apr 28, 2022 12:32:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Apr 28, 2022 12:32:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Apr 28, 2022 12:32:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Apr 28, 2022 12:32:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Apr 28, 2022 12:32:32 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.39.0-SNAPSHOT
Apr 28, 2022 12:32:32 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-04-28_05_32_32-10235830869270665189?project=apache-beam-testing
Apr 28, 2022 12:32:32 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2022-04-28_05_32_32-10235830869270665189
Apr 28, 2022 12:32:32 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2022-04-28_05_32_32-10235830869270665189
Apr 28, 2022 12:32:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2022-04-28T12:32:38.272Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java170dataflow0v20streaming0cogbk01-jenkins-04-t8zh. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Apr 28, 2022 12:32:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-28T12:32:46.137Z: Worker configuration: e2-standard-2 in us-central1-f.
Apr 28, 2022 12:32:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-28T12:32:46.706Z: Expanding SplittableParDo operations into optimizable parts.
Apr 28, 2022 12:32:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-28T12:32:46.743Z: Expanding CollectionToSingleton operations into optimizable parts.
Apr 28, 2022 12:32:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-28T12:32:46.837Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 28, 2022 12:32:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-28T12:32:46.922Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Apr 28, 2022 12:32:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-28T12:32:46.958Z: Expanding GroupByKey operations into streaming Read/Write steps
Apr 28, 2022 12:32:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-28T12:32:47.022Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 28, 2022 12:32:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-28T12:32:47.130Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 28, 2022 12:32:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-28T12:32:47.155Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Apr 28, 2022 12:32:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-28T12:32:47.190Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Apr 28, 2022 12:32:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-28T12:32:47.211Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Apr 28, 2022 12:32:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-28T12:32:47.233Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Apr 28, 2022 12:32:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-28T12:32:47.268Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Apr 28, 2022 12:32:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-28T12:32:47.300Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Apr 28, 2022 12:32:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-28T12:32:47.334Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Apr 28, 2022 12:32:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-28T12:32:47.366Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Apr 28, 2022 12:32:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-28T12:32:47.400Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Apr 28, 2022 12:32:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-28T12:32:47.440Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Apr 28, 2022 12:32:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-28T12:32:47.468Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Apr 28, 2022 12:32:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-28T12:32:47.506Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Apr 28, 2022 12:32:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-28T12:32:47.538Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Apr 28, 2022 12:32:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-28T12:32:47.568Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Apr 28, 2022 12:32:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-28T12:32:47.600Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Apr 28, 2022 12:32:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-28T12:32:47.632Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Apr 28, 2022 12:32:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-28T12:32:47.669Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Apr 28, 2022 12:32:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-28T12:32:47.701Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Apr 28, 2022 12:32:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-28T12:32:47.723Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Apr 28, 2022 12:32:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-28T12:32:47.755Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Apr 28, 2022 12:32:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-28T12:32:47.777Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Apr 28, 2022 12:32:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-28T12:32:47.824Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Apr 28, 2022 12:32:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-28T12:32:48.025Z: Running job using Streaming Engine
Apr 28, 2022 12:32:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-28T12:32:48.320Z: Starting 5 ****s in us-central1-f...
Apr 28, 2022 12:33:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-28T12:33:09.513Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 28, 2022 12:33:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-28T12:33:10.851Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Apr 28, 2022 12:34:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-28T12:34:18.153Z: Workers have started successfully.
Apr 28, 2022 1:17:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-04-28T13:17:50.263Z: Staged package gax-grpc-2.8.1-Oaqma5KQUeKekzhWvuQ9BCsF3Uf1NF_DCF0Um3fxw3s.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/gax-grpc-2.8.1-Oaqma5KQUeKekzhWvuQ9BCsF3Uf1NF_DCF0Um3fxw3s.jar' is inaccessible.
Apr 28, 2022 1:17:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2022-04-28T13:17:56.765Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Apr 28, 2022 1:20:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2022-04-28T13:20:52.937Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Apr 28, 2022 4:01:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-28T16:01:05.061Z: Cancel request is committed for workflow job: 2022-04-28_05_32_32-10235830869270665189.
Apr 28, 2022 4:01:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-28T16:01:05.121Z: Cleaning up.
Apr 28, 2022 4:01:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-28T16:01:05.211Z: Stopping **** pool...
Apr 28, 2022 4:01:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-28T16:01:05.279Z: Stopping **** pool...
Apr 28, 2022 4:01:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-28T16:01:38.538Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Apr 28, 2022 4:01:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-28T16:01:38.609Z: Worker pool stopped.
Apr 28, 2022 4:01:56 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2022-04-28_05_32_32-10235830869270665189 finished with status CANCELLED.
Load test results for test (ID): 902d45ce-f2c6-48bd-aa04-223c5fdbd611 and timestamp: 2022-04-28T12:32:27.273000000Z:
Metric: Value:
dataflow_v2_java17_runtime_sec 12356.592
dataflow_v2_java17_total_bytes_count 4.5381291E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)
> Task :sdks:java:testing:load-tests:run FAILED
> Task :runners:google-cloud-dataflow-java:cleanUpDockerJavaImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220428123039
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:e150bab5afd54e7eb0a9e2e9d2379ba06eab09426d968d7a5ad2e2b44cfcd34c
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220428123039]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:e150bab5afd54e7eb0a9e2e9d2379ba06eab09426d968d7a5ad2e2b44cfcd34c]
Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220428123039] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:e150bab5afd54e7eb0a9e2e9d2379ba06eab09426d968d7a5ad2e2b44cfcd34c])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:1fc998e42139727d2c3786f1de49786ca539cf13c81d83afa97ab6cf29387608
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:1fc998e42139727d2c3786f1de49786ca539cf13c81d83afa97ab6cf29387608
ERROR: (gcloud.container.images.delete) Not found: response: {'docker-distribution-api-version': 'registry/2.0', 'content-type': 'application/json', 'date': 'Thu, 28 Apr 2022 16:02:03 GMT', 'server': 'Docker Registry', 'cache-control': 'private', 'x-xss-protection': '0', 'x-frame-options': 'SAMEORIGIN', 'transfer-encoding': 'chunked', 'status': '404', 'content-length': '168', '-content-encoding': 'gzip'}
Failed to compute blob liveness for manifest: 'sha256:1fc998e42139727d2c3786f1de49786ca539cf13c81d83afa97ab6cf29387608': None
> Task :runners:google-cloud-dataflow-java:cleanUpDockerJavaImages FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17/ws/src/runners/google-cloud-dataflow-java/build.gradle'> line: 297
* What went wrong:
Execution failed for task ':runners:google-cloud-dataflow-java:cleanUpDockerJavaImages'.
> Process 'command './scripts/cleanup_untagged_gcr_images.sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.
BUILD FAILED in 3h 31m 38s
110 actionable tasks: 73 executed, 33 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/23fok3zo25qsc
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17 #128
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17/128/display/redirect?page=changes>
Changes:
[msbukal] FhirIO: use .search() or .searchType instead of .setResourceType()
[nick.caballero] [BEAM-14363] Fixes WatermarkParameters builder for Kinesis
[noreply] Remove unnecessary decorator from RunInference interface (#17463)
[noreply] [BEAM-13590] Minor deprecated warning fix (#17453)
[noreply] [BEAM-12164]: fix the negative throughput issue (#17461)
[noreply] Updated goldens for the screen diff integration tests (#17467)
[noreply] fixes copy by value error for bytes.Buffer in Error (#17469)
[noreply] Merge pull request #17354 from [BEAM-14170] - Create a test that runs
[noreply] Merge pull request #17447 from [BEAM-14357] Fix
[noreply] [BEAM-14324, BEAM-14325] Staticcheck cleanup in test files (#17393)
[noreply] BEAM-14187 Fix NPE (#17454)
[noreply] [BEAM-11105] Stateful watermark estimation (#17374)
[noreply] [BEAM-14304] implement parquetio to read/write parquet files (#17347)
------------------------------------------
[...truncated 49.72 KB...]
08fa02ce37eb: Preparing
a037458de4e0: Preparing
bafdbe68e4ae: Preparing
a13c519c6361: Preparing
7b9a4c020e68: Waiting
964154449ba6: Waiting
3fe9d00d75d7: Waiting
7a1c1c9d079b: Waiting
44c9846e824b: Waiting
5e5f3000d6ff: Waiting
784049eb671f: Waiting
12e1940c6229: Waiting
08fa02ce37eb: Waiting
a13c519c6361: Waiting
34461a8d10d3: Waiting
bafdbe68e4ae: Waiting
a037458de4e0: Waiting
e93827457889: Waiting
3bc383470c05: Waiting
4a6eae1d28c6: Waiting
d89f02fa017d: Pushed
98c5441bf5ed: Pushed
48aaa9aaf2dd: Pushed
c2b2d9642fa7: Pushed
9cb62a2ef04b: Pushed
964154449ba6: Pushed
3fe9d00d75d7: Pushed
12e1940c6229: Pushed
7b9a4c020e68: Pushed
44c9846e824b: Pushed
5e5f3000d6ff: Pushed
3bc383470c05: Layer already exists
7a1c1c9d079b: Pushed
e93827457889: Layer already exists
08fa02ce37eb: Layer already exists
a037458de4e0: Layer already exists
a13c519c6361: Layer already exists
bafdbe68e4ae: Layer already exists
34461a8d10d3: Pushed
4a6eae1d28c6: Pushed
784049eb671f: Pushed
20220427123044: digest: sha256:663e4583e395c5e749b4eb8afd1c9c87ebcf9081e05a13c047a84b1c639c52fd size: 4729
> Task :sdks:java:testing:load-tests:run
Apr 27, 2022 12:32:46 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Apr 27, 2022 12:32:46 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 208 files. Enable logging at DEBUG level to see which files will be staged.
Apr 27, 2022 12:32:47 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Apr 27, 2022 12:32:47 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 27, 2022 12:32:49 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 208 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 27, 2022 12:32:50 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 208 files cached, 0 files newly uploaded in 0 seconds
Apr 27, 2022 12:32:50 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 27, 2022 12:32:50 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <115405 bytes, hash a52582fcb6e4e90b59c0310f4221259ea839fc486655d0e5411dd50e9750d274> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-pSWC_Lbk6QtZwDEPQiElnqg5_EhmVdDlQR3VDpdQ0nQ.pb
Apr 27, 2022 12:32:51 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Apr 27, 2022 12:32:51 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@57272109, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@59696551, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@648d0e6d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@79e66b2f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@17273273, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5f69e2b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@984169e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43f1bb92, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6d6bbd35, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5c5d6175, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7544ac86, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3b27b497, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@b1534d3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c74aa0d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6c841199, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6a818392, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@489091bd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@512d6e60, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1de9b505, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b122839]
Apr 27, 2022 12:32:52 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Apr 27, 2022 12:32:52 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Apr 27, 2022 12:32:52 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Apr 27, 2022 12:32:52 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Apr 27, 2022 12:32:52 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6a6f6c7e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6c5ddccd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1dbd580, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6c101cc1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7d0d91a1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7fb48179, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@201c3cda, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4c86da0c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5d97caa4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6732726, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@474821de, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3d64c581, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5ec5ea63, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4190bc8a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@47d023b7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5c83ae01, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2d64c100, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@69d45cca, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2fdf17dc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6e6d4780]
Apr 27, 2022 12:32:52 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Apr 27, 2022 12:32:52 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Apr 27, 2022 12:32:52 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Apr 27, 2022 12:32:52 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Apr 27, 2022 12:32:52 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Apr 27, 2022 12:32:52 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Apr 27, 2022 12:32:52 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Apr 27, 2022 12:32:52 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Apr 27, 2022 12:32:52 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Apr 27, 2022 12:32:52 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Apr 27, 2022 12:32:52 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Apr 27, 2022 12:32:52 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.39.0-SNAPSHOT
Apr 27, 2022 12:32:55 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-04-27_05_32_52-13783208352401644293?project=apache-beam-testing
Apr 27, 2022 12:32:55 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2022-04-27_05_32_52-13783208352401644293
Apr 27, 2022 12:32:55 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2022-04-27_05_32_52-13783208352401644293
Apr 27, 2022 12:33:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2022-04-27T12:33:00.309Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java170dataflow0v20streaming0cogbk01-jenkins-04-oxw8. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Apr 27, 2022 12:33:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-27T12:33:04.785Z: Worker configuration: e2-standard-2 in us-central1-f.
Apr 27, 2022 12:33:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-27T12:33:05.467Z: Expanding SplittableParDo operations into optimizable parts.
Apr 27, 2022 12:33:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-27T12:33:05.508Z: Expanding CollectionToSingleton operations into optimizable parts.
Apr 27, 2022 12:33:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-27T12:33:05.573Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 27, 2022 12:33:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-27T12:33:05.643Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Apr 27, 2022 12:33:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-27T12:33:05.688Z: Expanding GroupByKey operations into streaming Read/Write steps
Apr 27, 2022 12:33:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-27T12:33:05.742Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 27, 2022 12:33:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-27T12:33:05.818Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 27, 2022 12:33:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-27T12:33:05.842Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Apr 27, 2022 12:33:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-27T12:33:05.876Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Apr 27, 2022 12:33:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-27T12:33:05.911Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Apr 27, 2022 12:33:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-27T12:33:05.942Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Apr 27, 2022 12:33:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-27T12:33:06.005Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Apr 27, 2022 12:33:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-27T12:33:06.037Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Apr 27, 2022 12:33:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-27T12:33:06.069Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Apr 27, 2022 12:33:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-27T12:33:06.103Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Apr 27, 2022 12:33:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-27T12:33:06.136Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Apr 27, 2022 12:33:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-27T12:33:06.174Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Apr 27, 2022 12:33:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-27T12:33:06.229Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Apr 27, 2022 12:33:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-27T12:33:06.251Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Apr 27, 2022 12:33:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-27T12:33:06.277Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Apr 27, 2022 12:33:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-27T12:33:06.312Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Apr 27, 2022 12:33:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-27T12:33:06.346Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Apr 27, 2022 12:33:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-27T12:33:06.378Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Apr 27, 2022 12:33:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-27T12:33:06.414Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Apr 27, 2022 12:33:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-27T12:33:06.459Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Apr 27, 2022 12:33:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-27T12:33:06.491Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Apr 27, 2022 12:33:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-27T12:33:06.522Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Apr 27, 2022 12:33:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-27T12:33:06.555Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Apr 27, 2022 12:33:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-27T12:33:06.588Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Apr 27, 2022 12:33:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-27T12:33:06.790Z: Running job using Streaming Engine
Apr 27, 2022 12:33:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-27T12:33:07.056Z: Starting 5 ****s in us-central1-f...
Apr 27, 2022 12:33:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-27T12:33:14.872Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 27, 2022 12:33:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-27T12:33:29.714Z: Autoscaling: Raised the number of ****s to 2 so that the pipeline can catch up with its backlog and keep up with its input rate.
Apr 27, 2022 12:33:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-27T12:33:29.748Z: Resized **** pool to 2, though goal was 5. This could be a quota issue.
Apr 27, 2022 12:33:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-27T12:33:39.988Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Apr 27, 2022 12:34:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-27T12:34:32.448Z: Workers have started successfully.
Apr 27, 2022 4:01:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-27T16:01:05.631Z: Cancel request is committed for workflow job: 2022-04-27_05_32_52-13783208352401644293.
Apr 27, 2022 4:01:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-27T16:01:05.715Z: Cleaning up.
Apr 27, 2022 4:01:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-27T16:01:05.797Z: Stopping **** pool...
Apr 27, 2022 4:01:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-27T16:01:05.851Z: Stopping **** pool...
Apr 27, 2022 4:01:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-27T16:01:39.762Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Apr 27, 2022 4:01:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-27T16:01:39.810Z: Worker pool stopped.
Apr 27, 2022 4:01:47 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2022-04-27_05_32_52-13783208352401644293 finished with status CANCELLED.
Load test results for test (ID): f0c9bf3d-1dfb-4478-b631-3345fde1f7d2 and timestamp: 2022-04-27T12:32:46.828000000Z:
Metric: Value:
dataflow_v2_java17_runtime_sec 12344.836
dataflow_v2_java17_total_bytes_count 3.6730648E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)
> Task :sdks:java:testing:load-tests:run FAILED
> Task :runners:google-cloud-dataflow-java:cleanUpDockerJavaImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220427123044
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:663e4583e395c5e749b4eb8afd1c9c87ebcf9081e05a13c047a84b1c639c52fd
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220427123044]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:663e4583e395c5e749b4eb8afd1c9c87ebcf9081e05a13c047a84b1c639c52fd]
Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220427123044] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:663e4583e395c5e749b4eb8afd1c9c87ebcf9081e05a13c047a84b1c639c52fd])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:663e4583e395c5e749b4eb8afd1c9c87ebcf9081e05a13c047a84b1c639c52fd
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:663e4583e395c5e749b4eb8afd1c9c87ebcf9081e05a13c047a84b1c639c52fd
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:663e4583e395c5e749b4eb8afd1c9c87ebcf9081e05a13c047a84b1c639c52fd].
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.
BUILD FAILED in 3h 31m 28s
110 actionable tasks: 74 executed, 32 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/qx42rbv36bzjg
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17 #127
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17/127/display/redirect?page=changes>
Changes:
[Heejong Lee] [BEAM-14343] Allow expansion service override in ExternalPythonTransform
[Heejong Lee] update
[Heejong Lee] allows remote host
[Heejong Lee] improve compatibility with python rowcoder
[ahmedabualsaud] added tempLocation to test pipeline options
[ahmedabualsaud] using tempRoot for temp bucket location
[ahmedabualsaud] small fixes
[noreply] [BEAM-14320] Update programming-guide w/Java GroupByKey example (#17369)
[noreply] Minor: Fix release script for `current` symlinks (#17457)
[noreply] Minor: fix typo (#17452)
[noreply] Change return type for PytorchInferenceRunner (#17460)
[noreply] [BEAM-13608] JmsIO dynamic topics feature (#17163)
[Heejong Lee] add test
------------------------------------------
[...truncated 266.00 KB...]
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmi
Apr 26, 2022 12:41:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-04-26T12:41:38.717Z: generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
Apr 26, 2022 4:01:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-26T16:01:06.136Z: Cancel request is committed for workflow job: 2022-04-26_05_32_36-10559723092899471543.
Apr 26, 2022 4:01:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-26T16:01:06.236Z: Cleaning up.
Apr 26, 2022 4:01:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-26T16:01:06.348Z: Stopping **** pool...
Apr 26, 2022 4:01:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-26T16:01:06.402Z: Stopping **** pool...
Apr 26, 2022 4:01:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-26T16:01:41.609Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Apr 26, 2022 4:01:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-26T16:01:41.698Z: Worker pool stopped.
Apr 26, 2022 4:01:49 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2022-04-26_05_32_36-10559723092899471543 finished with status CANCELLED.
Load test results for test (ID): 4d63233f-5eeb-4551-9e29-9f8f1a886f3e and timestamp: 2022-04-26T12:32:31.049000000Z:
Metric: Value:
dataflow_v2_java17_runtime_sec 12381.962
dataflow_v2_java17_total_bytes_count 3.83593313E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)
> Task :sdks:java:testing:load-tests:run FAILED
> Task :runners:google-cloud-dataflow-java:cleanUpDockerJavaImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220426123035
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:dda484922e902119919c5fb891b4ba16ad2be7bb5db8e969575b5cef50f11c50
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220426123035]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:dda484922e902119919c5fb891b4ba16ad2be7bb5db8e969575b5cef50f11c50]
Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220426123035] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:dda484922e902119919c5fb891b4ba16ad2be7bb5db8e969575b5cef50f11c50])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:dda484922e902119919c5fb891b4ba16ad2be7bb5db8e969575b5cef50f11c50
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:dda484922e902119919c5fb891b4ba16ad2be7bb5db8e969575b5cef50f11c50
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:dda484922e902119919c5fb891b4ba16ad2be7bb5db8e969575b5cef50f11c50].
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.
BUILD FAILED in 3h 31m 35s
110 actionable tasks: 73 executed, 33 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/ck5n63lkyh5d6
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17 #126
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17/126/display/redirect>
Changes:
------------------------------------------
[...truncated 52.75 KB...]
cc9d3b2e0c73: Preparing
06b058a4b9a7: Preparing
3bc383470c05: Preparing
e93827457889: Preparing
08fa02ce37eb: Preparing
a037458de4e0: Preparing
bafdbe68e4ae: Preparing
a13c519c6361: Preparing
2ae9e2f61925: Waiting
a037458de4e0: Waiting
bafdbe68e4ae: Waiting
08fa02ce37eb: Waiting
cc9d3b2e0c73: Waiting
a13c519c6361: Waiting
e93827457889: Waiting
06b058a4b9a7: Waiting
3bc383470c05: Waiting
712d27e00e03: Waiting
47c9d7fb5d05: Waiting
9cc4406b0a68: Waiting
a82745df9946: Waiting
e9d68c658358: Waiting
f0134388448b: Waiting
45ff9c2b079e: Waiting
34000863c977: Pushed
c9e867e365e1: Pushed
4e79872db04a: Pushed
d9073639ad20: Pushed
47cf847cdf23: Pushed
9cc4406b0a68: Pushed
a82745df9946: Pushed
e9d68c658358: Pushed
47c9d7fb5d05: Pushed
712d27e00e03: Pushed
45ff9c2b079e: Pushed
3bc383470c05: Layer already exists
e93827457889: Layer already exists
08fa02ce37eb: Layer already exists
a037458de4e0: Layer already exists
f0134388448b: Pushed
bafdbe68e4ae: Layer already exists
a13c519c6361: Layer already exists
cc9d3b2e0c73: Pushed
06b058a4b9a7: Pushed
2ae9e2f61925: Pushed
20220425123042: digest: sha256:c8d7b6367a6d100d98235579c4752deeafcba963def5ca559ea35987a5fb17ad size: 4729
> Task :sdks:java:testing:load-tests:run
Apr 25, 2022 12:33:22 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Apr 25, 2022 12:33:22 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 208 files. Enable logging at DEBUG level to see which files will be staged.
Apr 25, 2022 12:33:23 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Apr 25, 2022 12:33:23 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Apr 25, 2022 12:33:26 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 208 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Apr 25, 2022 12:33:28 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 208 files cached, 0 files newly uploaded in 1 seconds
Apr 25, 2022 12:33:28 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 25, 2022 12:33:28 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <115405 bytes, hash b7aa88cc699ec5780882b5423e514ec2096d7b4ba193a228de2f0cdd1a28c105> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-t6qIzGmexXgIgrVCPlFOwglte0uhk6Io3i8M3RoowQU.pb
Apr 25, 2022 12:33:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Apr 25, 2022 12:33:30 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@57272109, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@59696551, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@648d0e6d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@79e66b2f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@17273273, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5f69e2b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@984169e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43f1bb92, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6d6bbd35, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5c5d6175, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7544ac86, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3b27b497, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@b1534d3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c74aa0d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6c841199, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6a818392, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@489091bd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@512d6e60, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1de9b505, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b122839]
Apr 25, 2022 12:33:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Apr 25, 2022 12:33:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Apr 25, 2022 12:33:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Apr 25, 2022 12:33:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Apr 25, 2022 12:33:30 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6a6f6c7e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6c5ddccd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1dbd580, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6c101cc1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7d0d91a1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7fb48179, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@201c3cda, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4c86da0c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5d97caa4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6732726, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@474821de, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3d64c581, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5ec5ea63, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4190bc8a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@47d023b7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5c83ae01, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2d64c100, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@69d45cca, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2fdf17dc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6e6d4780]
Apr 25, 2022 12:33:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Apr 25, 2022 12:33:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Apr 25, 2022 12:33:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Apr 25, 2022 12:33:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Apr 25, 2022 12:33:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Apr 25, 2022 12:33:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Apr 25, 2022 12:33:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Apr 25, 2022 12:33:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Apr 25, 2022 12:33:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Apr 25, 2022 12:33:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Apr 25, 2022 12:33:30 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Apr 25, 2022 12:33:30 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.39.0-SNAPSHOT
Apr 25, 2022 12:33:31 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-04-25_05_33_30-12043335608045540008?project=apache-beam-testing
Apr 25, 2022 12:33:31 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2022-04-25_05_33_30-12043335608045540008
Apr 25, 2022 12:33:31 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2022-04-25_05_33_30-12043335608045540008
Apr 25, 2022 12:33:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2022-04-25T12:33:36.436Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java170dataflow0v20streaming0cogbk01-jenkins-04-l9tt. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Apr 25, 2022 12:33:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-25T12:33:41.206Z: Worker configuration: e2-standard-2 in us-central1-f.
Apr 25, 2022 12:33:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-25T12:33:42.026Z: Expanding SplittableParDo operations into optimizable parts.
Apr 25, 2022 12:33:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-25T12:33:42.061Z: Expanding CollectionToSingleton operations into optimizable parts.
Apr 25, 2022 12:33:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-25T12:33:42.119Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 25, 2022 12:33:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-25T12:33:42.182Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Apr 25, 2022 12:33:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-25T12:33:42.205Z: Expanding GroupByKey operations into streaming Read/Write steps
Apr 25, 2022 12:33:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-25T12:33:42.273Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 25, 2022 12:33:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-25T12:33:42.372Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 25, 2022 12:33:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-25T12:33:42.439Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Apr 25, 2022 12:33:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-25T12:33:42.469Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Apr 25, 2022 12:33:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-25T12:33:42.522Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Apr 25, 2022 12:33:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-25T12:33:42.569Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Apr 25, 2022 12:33:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-25T12:33:42.591Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Apr 25, 2022 12:33:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-25T12:33:42.622Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Apr 25, 2022 12:33:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-25T12:33:42.646Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Apr 25, 2022 12:33:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-25T12:33:42.677Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Apr 25, 2022 12:33:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-25T12:33:42.712Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Apr 25, 2022 12:33:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-25T12:33:42.778Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Apr 25, 2022 12:33:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-25T12:33:42.800Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Apr 25, 2022 12:33:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-25T12:33:42.833Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Apr 25, 2022 12:33:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-25T12:33:42.865Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Apr 25, 2022 12:33:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-25T12:33:42.888Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Apr 25, 2022 12:33:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-25T12:33:42.922Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Apr 25, 2022 12:33:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-25T12:33:42.956Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Apr 25, 2022 12:33:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-25T12:33:42.977Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Apr 25, 2022 12:33:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-25T12:33:42.999Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Apr 25, 2022 12:33:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-25T12:33:43.025Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Apr 25, 2022 12:33:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-25T12:33:43.048Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Apr 25, 2022 12:33:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-25T12:33:43.082Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Apr 25, 2022 12:33:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-25T12:33:43.114Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Apr 25, 2022 12:33:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-25T12:33:43.273Z: Running job using Streaming Engine
Apr 25, 2022 12:33:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-25T12:33:43.560Z: Starting 5 ****s in us-central1-f...
Apr 25, 2022 12:34:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-25T12:34:05.733Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Apr 25, 2022 12:34:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-25T12:34:15.369Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 25, 2022 12:35:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-25T12:35:13.639Z: Workers have started successfully.
Apr 25, 2022 4:00:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-25T16:00:50.504Z: Cancel request is committed for workflow job: 2022-04-25_05_33_30-12043335608045540008.
Apr 25, 2022 4:00:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-25T16:00:55.276Z: Cleaning up.
Apr 25, 2022 4:00:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-25T16:00:55.348Z: Stopping **** pool...
Apr 25, 2022 4:00:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-25T16:00:55.393Z: Stopping **** pool...
Apr 25, 2022 4:01:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-25T16:01:32.367Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Apr 25, 2022 4:01:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-25T16:01:32.412Z: Worker pool stopped.
Apr 25, 2022 4:01:42 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2022-04-25_05_33_30-12043335608045540008 finished with status CANCELLED.
Load test results for test (ID): 1b9e34f8-3d87-47e9-a001-6cf2ce2c8668 and timestamp: 2022-04-25T12:33:23.163000000Z:
Metric: Value:
dataflow_v2_java17_runtime_sec 12314.251
dataflow_v2_java17_total_bytes_count 1.86143849E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)
> Task :sdks:java:testing:load-tests:run FAILED
> Task :runners:google-cloud-dataflow-java:cleanUpDockerJavaImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220425123042
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:c8d7b6367a6d100d98235579c4752deeafcba963def5ca559ea35987a5fb17ad
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220425123042]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:c8d7b6367a6d100d98235579c4752deeafcba963def5ca559ea35987a5fb17ad]
Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220425123042] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:c8d7b6367a6d100d98235579c4752deeafcba963def5ca559ea35987a5fb17ad])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:c8d7b6367a6d100d98235579c4752deeafcba963def5ca559ea35987a5fb17ad
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:c8d7b6367a6d100d98235579c4752deeafcba963def5ca559ea35987a5fb17ad
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:c8d7b6367a6d100d98235579c4752deeafcba963def5ca559ea35987a5fb17ad].
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.
BUILD FAILED in 3h 31m 22s
110 actionable tasks: 73 executed, 33 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/nhzlzapq2wnzq
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17 #125
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17/125/display/redirect?page=changes>
Changes:
[noreply] [BEAM-13953] added documentation for BQ Storage Write API (#17391)
------------------------------------------
[...truncated 556.30 KB...]
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmi
Apr 24, 2022 3:29:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-04-24T15:29:54.472Z: Staged package error_prone_annotations-2.10.0-oknU0l37htQebIL8M131gBifDJ_uq9xTIz_B5QYHJKE.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/error_prone_annotations-2.10.0-oknU0l37htQebIL8M131gBifDJ_uq9xTIz_B5QYHJKE.jar' is inaccessible.
Apr 24, 2022 3:29:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-04-24T15:29:54.736Z: Staged package gax-grpc-2.8.1-testlib-nH-xuZAVjRReb1TzhwGz0aBrSNkuxID3L6Ws8zqaZD8.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/gax-grpc-2.8.1-testlib-nH-xuZAVjRReb1TzhwGz0aBrSNkuxID3L6Ws8zqaZD8.jar' is inaccessible.
Apr 24, 2022 3:29:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2022-04-24T15:29:57.352Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions.
Apr 24, 2022 4:00:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-24T16:00:57.071Z: Cancel request is committed for workflow job: 2022-04-24_05_32_40-2758132320092160384.
Apr 24, 2022 4:00:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-24T16:00:57.116Z: Cleaning up.
Apr 24, 2022 4:00:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-24T16:00:57.190Z: Stopping **** pool...
Apr 24, 2022 4:00:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-24T16:00:57.279Z: Stopping **** pool...
Apr 24, 2022 4:01:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-24T16:01:30.830Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Apr 24, 2022 4:01:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-24T16:01:30.880Z: Worker pool stopped.
Apr 24, 2022 4:01:36 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2022-04-24_05_32_40-2758132320092160384 finished with status CANCELLED.
Load test results for test (ID): b4faf98d-4252-476d-97ed-e3c0021847bf and timestamp: 2022-04-24T12:32:34.311000000Z:
Metric: Value:
dataflow_v2_java17_runtime_sec 12240.272
dataflow_v2_java17_total_bytes_count 3.88454972E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)
> Task :sdks:java:testing:load-tests:run FAILED
> Task :runners:google-cloud-dataflow-java:cleanUpDockerJavaImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220424123036
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:e61bdcb503f1696876b5e7165dad0472c06f854503531b3a55567435696a11c6
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220424123036]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:e61bdcb503f1696876b5e7165dad0472c06f854503531b3a55567435696a11c6]
Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220424123036] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:e61bdcb503f1696876b5e7165dad0472c06f854503531b3a55567435696a11c6])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:e61bdcb503f1696876b5e7165dad0472c06f854503531b3a55567435696a11c6
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:e61bdcb503f1696876b5e7165dad0472c06f854503531b3a55567435696a11c6
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:e61bdcb503f1696876b5e7165dad0472c06f854503531b3a55567435696a11c6].
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.
BUILD FAILED in 3h 31m 23s
110 actionable tasks: 73 executed, 33 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/e5x46eyj3ggzi
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17 - Build # 124 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17 - Build # 124 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17/124/ to view the results.
Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17 #123
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17/123/display/redirect?page=changes>
Changes:
[mmack] [BEAM-14335] Spotless Spark sources
[mmack] [BEAM-14345] Force paranamer 2.8 for Spark Hadoop version tests to avoid
[kamil.bregula] Revert "[BEAM-14300] Fix Java precommit failure"
[kamil.bregula] Revert "Merge pull request #17223 from [BEAM-14215] Improve argument
[noreply] [BEAM-13657] Sunset python 3.6 (#17252)
[noreply] Removes unsupported Python 3.6 from the release validation script
[noreply] [BEAM-13984] Implement RunInference for PyTorch (#17196)
[noreply] [BEAM-13945] add json type support for java bigquery connector (#17209)
[noreply] [BEAM-14346] Fix incorrect error case index in ret2() (#17425)
[noreply] [BEAM-14342] Fix wrong default buffer type in fn_runner (#17420)
[noreply] Updates opencensus-api dependency to the latest version - 0.31.0
[noreply] [BEAM-14306] Add unit testing to pane coder (#17370)
[noreply] Updated the dep and golden for screen diff integration tests (#17442)
------------------------------------------
[...truncated 73.50 KB...]
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
Apr 22, 2022 4:01:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-22T16:01:06.907Z: Cancel request is committed for workflow job: 2022-04-22_05_32_45-12150457720570078207.
Apr 22, 2022 4:01:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-22T16:01:07.360Z: Cleaning up.
Apr 22, 2022 4:01:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-22T16:01:07.483Z: Stopping **** pool...
Apr 22, 2022 4:01:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-22T16:01:07.527Z: Stopping **** pool...
Apr 22, 2022 4:01:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-22T16:01:39.793Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Apr 22, 2022 4:01:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-22T16:01:39.919Z: Worker pool stopped.
Apr 22, 2022 4:01:46 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2022-04-22_05_32_45-12150457720570078207 finished with status CANCELLED.
Load test results for test (ID): 8efaf76e-f982-4e87-a1c5-795f0aefdff6 and timestamp: 2022-04-22T12:32:39.059000000Z:
Metric: Value:
dataflow_v2_java17_runtime_sec 12367.843
dataflow_v2_java17_total_bytes_count 2.30870961E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)
> Task :sdks:java:testing:load-tests:run FAILED
> Task :runners:google-cloud-dataflow-java:cleanUpDockerJavaImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220422123041
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:5a40815573ef05e81b82b884f5f2067779fb56ac6eb374813ef4a350e4a2f56b
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220422123041]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:5a40815573ef05e81b82b884f5f2067779fb56ac6eb374813ef4a350e4a2f56b]
Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220422123041] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:5a40815573ef05e81b82b884f5f2067779fb56ac6eb374813ef4a350e4a2f56b])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:5a40815573ef05e81b82b884f5f2067779fb56ac6eb374813ef4a350e4a2f56b
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:5a40815573ef05e81b82b884f5f2067779fb56ac6eb374813ef4a350e4a2f56b
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:5a40815573ef05e81b82b884f5f2067779fb56ac6eb374813ef4a350e4a2f56b].
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.
BUILD FAILED in 3h 31m 26s
110 actionable tasks: 74 executed, 32 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/vqmcwxbnxikfe
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17 #122
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17/122/display/redirect?page=changes>
Changes:
[vachan] Annotating Read API tests.
[bulat.safiullin] [BEAM-14247] [Website] add image
[bulat.safiullin] [BEAM-14247] [Website] center image
[mattcasters] BEAM-1857 : CHANGES.md entry for 2.38.0
[mmack] [BEAM-14323] Improve IDE integration of Spark cross version builds
[noreply] [BEAM-14112] Fixed ReadFromBigQuery with Interactive Beam (#17306)
[noreply] Update .asf.yaml (#17409)
[noreply] [BEAM-14336] Sickbay flight delays test - dataset seems to be missing
[noreply] [BEAM-14338] Update watermark unit tests to use time.Time.Equals()
[noreply] [BEAM-14328] Tweaks to "Differences from pandas" page (#17413)
[Andrew Pilloud] [BEAM-14253] Disable broken test pending Dataflow fix
[yiru] fix: BigQuery Storage Connector trace id population missing bracket
[noreply] [BEAM-14330] Temporarily disable the clusters auto-cleanup (#17400)
[noreply] Update Beam website to release 2.38.0 (#17378)
[noreply] [BEAM-14213] Add API and construction time validation for Batched DoFns
[noreply] Minor: Update release guide regarding archive.apache.org (#17419)
[noreply] [BEAM-14017] beam_PreCommit_CommunityMetrics_Cron test failing (#17396)
[noreply] BEAM-13582 Fixing broken links in the documentation (#17300)
------------------------------------------
[...truncated 296.04 KB...]
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmi
Apr 21, 2022 4:00:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-21T16:00:50.620Z: Cancel request is committed for workflow job: 2022-04-21_05_32_39-4975189586864708056.
Apr 21, 2022 4:00:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-21T16:00:50.757Z: Cleaning up.
Apr 21, 2022 4:00:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-21T16:00:50.818Z: Stopping **** pool...
Apr 21, 2022 4:00:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-21T16:00:50.876Z: Stopping **** pool...
Apr 21, 2022 4:01:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-21T16:01:27.729Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Apr 21, 2022 4:01:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-21T16:01:27.773Z: Worker pool stopped.
Apr 21, 2022 4:01:35 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2022-04-21_05_32_39-4975189586864708056 finished with status CANCELLED.
Load test results for test (ID): 69cefdbc-5566-4ae6-8d36-deadae2f54a9 and timestamp: 2022-04-21T12:32:33.827000000Z:
Metric: Value:
dataflow_v2_java17_runtime_sec 12360.759
dataflow_v2_java17_total_bytes_count 3.86006401E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)
> Task :sdks:java:testing:load-tests:run FAILED
> Task :runners:google-cloud-dataflow-java:cleanUpDockerJavaImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220421123040
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:d75db3a9f4e11ee1af2fac745b8daa23835fede6d9d30368df784c0733674d91
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220421123040]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:d75db3a9f4e11ee1af2fac745b8daa23835fede6d9d30368df784c0733674d91]
Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220421123040] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:d75db3a9f4e11ee1af2fac745b8daa23835fede6d9d30368df784c0733674d91])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:d75db3a9f4e11ee1af2fac745b8daa23835fede6d9d30368df784c0733674d91
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:d75db3a9f4e11ee1af2fac745b8daa23835fede6d9d30368df784c0733674d91
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:d75db3a9f4e11ee1af2fac745b8daa23835fede6d9d30368df784c0733674d91].
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.
BUILD FAILED in 3h 31m 16s
110 actionable tasks: 73 executed, 33 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/xbd7uwrkmoupa
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17 #121
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17/121/display/redirect?page=changes>
Changes:
[andyye333] Change func to PTransform
[noreply] Populate actual dataflow job id to bigquery write trace id (#17130)
[relax] mark static thread as a daemon thread
[noreply] [BEAM-13866] Add miscellaneous exec unit tests (#17363)
------------------------------------------
[...truncated 278.69 KB...]
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmi
Apr 20, 2022 12:39:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-04-20T12:39:53.912Z: generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
Apr 20, 2022 2:25:40 PM org.apache.beam.sdk.metrics.MetricsEnvironment getCurrentContainer
WARNING: Reporting metrics are not supported in the current execution environment.
Apr 20, 2022 4:00:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-20T16:00:41.522Z: Cancel request is committed for workflow job: 2022-04-20_05_33_06-426165222419621729.
Apr 20, 2022 4:00:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-20T16:00:46.465Z: Cleaning up.
Apr 20, 2022 4:00:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-20T16:00:46.544Z: Stopping **** pool...
Apr 20, 2022 4:00:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-20T16:00:46.600Z: Stopping **** pool...
Apr 20, 2022 4:01:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-20T16:01:38.405Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Apr 20, 2022 4:01:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-20T16:01:38.436Z: Worker pool stopped.
Apr 20, 2022 4:01:45 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2022-04-20_05_33_06-426165222419621729 finished with status CANCELLED.
Load test results for test (ID): b9981ed6-af2e-4594-8adb-e07903a06596 and timestamp: 2022-04-20T12:33:00.824000000Z:
Metric: Value:
dataflow_v2_java17_runtime_sec 12295.121
dataflow_v2_java17_total_bytes_count 4.18616716E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)
> Task :sdks:java:testing:load-tests:run FAILED
> Task :runners:google-cloud-dataflow-java:cleanUpDockerJavaImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220420123035
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ac7d89b321c2d902732fa25bb4c64744ce7de142c48e22272b4be7dc53db75fa
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220420123035]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ac7d89b321c2d902732fa25bb4c64744ce7de142c48e22272b4be7dc53db75fa]
Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220420123035] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ac7d89b321c2d902732fa25bb4c64744ce7de142c48e22272b4be7dc53db75fa])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ac7d89b321c2d902732fa25bb4c64744ce7de142c48e22272b4be7dc53db75fa
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ac7d89b321c2d902732fa25bb4c64744ce7de142c48e22272b4be7dc53db75fa
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ac7d89b321c2d902732fa25bb4c64744ce7de142c48e22272b4be7dc53db75fa].
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.
BUILD FAILED in 3h 31m 52s
110 actionable tasks: 74 executed, 32 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/ygqdlziw45zlg
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17 #120
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17/120/display/redirect?page=changes>
Changes:
[Heejong Lee] [BEAM-14251] add output_coder_override to ExpansionRequest
[Heejong Lee] remove null
[rarokni] [BEAM-14307] Fix Slow Side input pattern bug in sample
[Heejong Lee] better error msg
[Heejong Lee] update from comments
[noreply] [BEAM-14316] Introducing KafkaIO.Read implementation compatibility
[noreply] [BEAM-14290] Address staticcheck warnings in the reflectx package
[noreply] [BEAM-14302] Simply bools in fn.go, genx_test.go (#17356)
[noreply] Merge pull request #17382: [BEAM-12356] Close DatasetService leak as
------------------------------------------
[...truncated 603.30 KB...]
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmi
Apr 19, 2022 4:00:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-19T16:00:35.094Z: Cancel request is committed for workflow job: 2022-04-19_05_32_43-4783407724810226384.
Apr 19, 2022 4:00:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-19T16:00:35.218Z: Cleaning up.
Apr 19, 2022 4:00:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-19T16:00:35.281Z: Stopping **** pool...
Apr 19, 2022 4:00:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-19T16:00:35.348Z: Stopping **** pool...
Apr 19, 2022 4:01:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-19T16:01:20.787Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Apr 19, 2022 4:01:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-19T16:01:20.825Z: Worker pool stopped.
Apr 19, 2022 4:01:27 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2022-04-19_05_32_43-4783407724810226384 finished with status CANCELLED.
Load test results for test (ID): 11d10eda-55e5-4e22-88c7-2210a04f5830 and timestamp: 2022-04-19T12:32:38.652000000Z:
Metric: Value:
dataflow_v2_java17_runtime_sec 12334.25
dataflow_v2_java17_total_bytes_count 3.96843308E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)
> Task :sdks:java:testing:load-tests:run FAILED
> Task :runners:google-cloud-dataflow-java:cleanUpDockerJavaImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220419123039
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:398d8ea056f8445ad50eb460635608987789ba49af2871667a67805752ce3e14
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220419123039]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:398d8ea056f8445ad50eb460635608987789ba49af2871667a67805752ce3e14]
Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220419123039] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:398d8ea056f8445ad50eb460635608987789ba49af2871667a67805752ce3e14])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:398d8ea056f8445ad50eb460635608987789ba49af2871667a67805752ce3e14
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:398d8ea056f8445ad50eb460635608987789ba49af2871667a67805752ce3e14
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:398d8ea056f8445ad50eb460635608987789ba49af2871667a67805752ce3e14].
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.
BUILD FAILED in 3h 31m 9s
110 actionable tasks: 74 executed, 32 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/nbrwfottzid4k
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17 #119
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17/119/display/redirect>
Changes:
------------------------------------------
[...truncated 52.86 KB...]
Apr 18, 2022 12:32:36 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Apr 18, 2022 12:32:36 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <115405 bytes, hash 32f81f2b8cc26e63e0172c1bf398abf417be3a6ad4c55b00f6fac5fec2b06166> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-MvgfK4zCbmPgFywb85ir9Be-OmrUxVsA9vrF_sKwYWY.pb
Apr 18, 2022 12:32:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Apr 18, 2022 12:32:37 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@57272109, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@59696551, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@648d0e6d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@79e66b2f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@17273273, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5f69e2b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@984169e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43f1bb92, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6d6bbd35, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5c5d6175, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7544ac86, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3b27b497, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@b1534d3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c74aa0d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6c841199, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6a818392, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@489091bd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@512d6e60, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1de9b505, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b122839]
Apr 18, 2022 12:32:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Apr 18, 2022 12:32:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Apr 18, 2022 12:32:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Apr 18, 2022 12:32:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5
Apr 18, 2022 12:32:37 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6a6f6c7e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6c5ddccd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1dbd580, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6c101cc1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7d0d91a1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7fb48179, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@201c3cda, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4c86da0c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5d97caa4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6732726, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@474821de, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3d64c581, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5ec5ea63, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4190bc8a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@47d023b7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5c83ae01, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2d64c100, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@69d45cca, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2fdf17dc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6e6d4780]
Apr 18, 2022 12:32:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Apr 18, 2022 12:32:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Apr 18, 2022 12:32:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Apr 18, 2022 12:32:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Apr 18, 2022 12:32:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Apr 18, 2022 12:32:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Apr 18, 2022 12:32:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Apr 18, 2022 12:32:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Apr 18, 2022 12:32:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Apr 18, 2022 12:32:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Apr 18, 2022 12:32:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Apr 18, 2022 12:32:38 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.39.0-SNAPSHOT
Apr 18, 2022 12:32:38 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-04-18_05_32_38-2279234934055603378?project=apache-beam-testing
Apr 18, 2022 12:32:38 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2022-04-18_05_32_38-2279234934055603378
Apr 18, 2022 12:32:38 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2022-04-18_05_32_38-2279234934055603378
Apr 18, 2022 12:32:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2022-04-18T12:32:43.788Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java170dataflow0v20streaming0cogbk01-jenkins-04-gxql. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Apr 18, 2022 12:32:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-18T12:32:47.975Z: Worker configuration: e2-standard-2 in us-central1-b.
Apr 18, 2022 12:32:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-18T12:32:48.764Z: Expanding SplittableParDo operations into optimizable parts.
Apr 18, 2022 12:32:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-18T12:32:48.792Z: Expanding CollectionToSingleton operations into optimizable parts.
Apr 18, 2022 12:32:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-18T12:32:48.847Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 18, 2022 12:32:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-18T12:32:48.928Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Apr 18, 2022 12:32:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-18T12:32:48.957Z: Expanding GroupByKey operations into streaming Read/Write steps
Apr 18, 2022 12:32:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-18T12:32:49.014Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 18, 2022 12:32:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-18T12:32:49.139Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 18, 2022 12:32:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-18T12:32:49.164Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Apr 18, 2022 12:32:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-18T12:32:49.186Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Apr 18, 2022 12:32:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-18T12:32:49.217Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Apr 18, 2022 12:32:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-18T12:32:49.248Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Apr 18, 2022 12:32:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-18T12:32:49.282Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Apr 18, 2022 12:32:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-18T12:32:49.313Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Apr 18, 2022 12:32:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-18T12:32:49.336Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Apr 18, 2022 12:32:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-18T12:32:49.367Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Apr 18, 2022 12:32:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-18T12:32:49.389Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Apr 18, 2022 12:32:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-18T12:32:49.421Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign
Apr 18, 2022 12:32:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-18T12:32:49.446Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse
Apr 18, 2022 12:32:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-18T12:32:49.493Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Apr 18, 2022 12:32:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-18T12:32:49.528Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Apr 18, 2022 12:32:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-18T12:32:49.559Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Apr 18, 2022 12:32:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-18T12:32:49.591Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Apr 18, 2022 12:32:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-18T12:32:49.635Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Apr 18, 2022 12:32:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-18T12:32:49.666Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign
Apr 18, 2022 12:32:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-18T12:32:49.697Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream
Apr 18, 2022 12:32:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-18T12:32:49.720Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets
Apr 18, 2022 12:32:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-18T12:32:49.755Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Apr 18, 2022 12:32:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-18T12:32:49.789Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)
Apr 18, 2022 12:32:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-18T12:32:49.823Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Apr 18, 2022 12:32:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-18T12:32:49.990Z: Running job using Streaming Engine
Apr 18, 2022 12:32:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-18T12:32:50.240Z: Starting 5 ****s in us-central1-b...
Apr 18, 2022 12:32:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-18T12:32:54.782Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 18, 2022 12:33:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-18T12:33:20.535Z: Autoscaling: Raised the number of ****s to 2 so that the pipeline can catch up with its backlog and keep up with its input rate.
Apr 18, 2022 12:33:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-18T12:33:20.553Z: Resized **** pool to 2, though goal was 5. This could be a quota issue.
Apr 18, 2022 12:33:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-18T12:33:30.837Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Apr 18, 2022 12:34:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-18T12:34:20.974Z: Workers have started successfully.
Apr 18, 2022 12:37:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-04-18T12:37:47.439Z: generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
Apr 18, 2022 12:37:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-04-18T12:37:50.916Z: generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
Apr 18, 2022 4:00:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-18T16:00:42.714Z: Cancel request is committed for workflow job: 2022-04-18_05_32_38-2279234934055603378.
Apr 18, 2022 4:00:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-18T16:00:42.828Z: Cleaning up.
Apr 18, 2022 4:00:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-18T16:00:42.983Z: Stopping **** pool...
Apr 18, 2022 4:00:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-18T16:00:43.025Z: Stopping **** pool...
Apr 18, 2022 4:01:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-18T16:01:37.008Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Apr 18, 2022 4:01:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-18T16:01:37.038Z: Worker pool stopped.
Apr 18, 2022 4:01:42 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2022-04-18_05_32_38-2279234934055603378 finished with status CANCELLED.
Load test results for test (ID): 41fb1f1f-1d70-40a1-91aa-fc95be2b5cea and timestamp: 2022-04-18T12:32:32.627000000Z:
Metric: Value:
dataflow_v2_java17_runtime_sec 12297.749
dataflow_v2_java17_total_bytes_count 2.7185534E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)
> Task :sdks:java:testing:load-tests:run FAILED
> Task :runners:google-cloud-dataflow-java:cleanUpDockerJavaImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220418123034
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:a4df32eaf063e0e131f7d9440f668f5e0c40c242f28640b1d74abac4601a5ffc
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220418123034]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:a4df32eaf063e0e131f7d9440f668f5e0c40c242f28640b1d74abac4601a5ffc]
Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220418123034] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:a4df32eaf063e0e131f7d9440f668f5e0c40c242f28640b1d74abac4601a5ffc])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:a4df32eaf063e0e131f7d9440f668f5e0c40c242f28640b1d74abac4601a5ffc
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:a4df32eaf063e0e131f7d9440f668f5e0c40c242f28640b1d74abac4601a5ffc
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:a4df32eaf063e0e131f7d9440f668f5e0c40c242f28640b1d74abac4601a5ffc].
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.
BUILD FAILED in 3h 31m 28s
110 actionable tasks: 73 executed, 33 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/7scm7ayhwflta
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17 #118
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17/118/display/redirect>
Changes:
------------------------------------------
[...truncated 637.69 KB...]
dist_proc/dax/workflow/****/streaming/windmill_cache_access.cc:40
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/windmill_cache_access.cc:40
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/windmill_cache_access.cc:40
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/windmill_cache_access.cc:40
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/windmill_cache_access.cc:40
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/windmill_cache_access.cc:40
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/windmill_cache_access.cc:40
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/windmill_cache_access.cc:40
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/windmill_cache_access.cc:40
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/windmill_cache_access.cc:40
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/windmill_cache_access.cc:40
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/windmill_cache_access.cc:40
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
Apr 17, 2022 4:01:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-17T16:01:00.019Z: Cancel request is committed for workflow job: 2022-04-17_05_32_49-8133379480979873444.
Apr 17, 2022 4:01:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-17T16:01:00.101Z: Cleaning up.
Apr 17, 2022 4:01:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-17T16:01:00.175Z: Stopping **** pool...
Apr 17, 2022 4:01:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-17T16:01:00.235Z: Stopping **** pool...
Apr 17, 2022 4:01:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-17T16:01:55.255Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Apr 17, 2022 4:01:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-17T16:01:55.299Z: Worker pool stopped.
Apr 17, 2022 4:02:01 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2022-04-17_05_32_49-8133379480979873444 finished with status CANCELLED.
Load test results for test (ID): d70e5409-1870-4777-a37f-7c2c351e5b0d and timestamp: 2022-04-17T12:32:39.505000000Z:
Metric: Value:
dataflow_v2_java17_runtime_sec 12368.731
dataflow_v2_java17_total_bytes_count 3.97045431E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)
> Task :sdks:java:testing:load-tests:run FAILED
> Task :runners:google-cloud-dataflow-java:cleanUpDockerJavaImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220417123040
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:9dbf27da19729e3ddbc7bc5f595b3268e4fb3920d7b8b0f79f8617edd668c153
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220417123040]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:9dbf27da19729e3ddbc7bc5f595b3268e4fb3920d7b8b0f79f8617edd668c153]
Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220417123040] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:9dbf27da19729e3ddbc7bc5f595b3268e4fb3920d7b8b0f79f8617edd668c153])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:9dbf27da19729e3ddbc7bc5f595b3268e4fb3920d7b8b0f79f8617edd668c153
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:9dbf27da19729e3ddbc7bc5f595b3268e4fb3920d7b8b0f79f8617edd668c153
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:9dbf27da19729e3ddbc7bc5f595b3268e4fb3920d7b8b0f79f8617edd668c153].
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.
BUILD FAILED in 3h 31m 41s
110 actionable tasks: 73 executed, 33 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/7ysdlikbx7obw
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17 #117
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17/117/display/redirect?page=changes>
Changes:
[pandiana] BigQueryServicesImpl: reduce number of threads spawned by
[noreply] [BEAM-13204] Fix website bug where code tabs do not appear if the
------------------------------------------
[...truncated 182.64 KB...]
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmi
Apr 16, 2022 12:39:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-04-16T12:39:13.646Z: generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
Apr 16, 2022 4:00:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-16T16:00:42.923Z: Cancel request is committed for workflow job: 2022-04-16_05_32_38-3458726814305771999.
Apr 16, 2022 4:00:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-16T16:00:43.055Z: Cleaning up.
Apr 16, 2022 4:00:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-16T16:00:43.463Z: Stopping **** pool...
Apr 16, 2022 4:00:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-16T16:00:43.571Z: Stopping **** pool...
Apr 16, 2022 4:01:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-16T16:01:35.236Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Apr 16, 2022 4:01:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-16T16:01:35.273Z: Worker pool stopped.
Apr 16, 2022 4:01:43 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2022-04-16_05_32_38-3458726814305771999 finished with status CANCELLED.
Load test results for test (ID): 18408d1d-a895-4b4d-93e6-dea02a74d3f0 and timestamp: 2022-04-16T12:32:32.679000000Z:
Metric: Value:
dataflow_v2_java17_runtime_sec 12274.909
dataflow_v2_java17_total_bytes_count 3.06803042E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)
> Task :sdks:java:testing:load-tests:run FAILED
> Task :runners:google-cloud-dataflow-java:cleanUpDockerJavaImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220416123040
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:2f305dee5a665a5943e6b107852db2e393161be627e50c3c07a9468916d0b1cc
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220416123040]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:2f305dee5a665a5943e6b107852db2e393161be627e50c3c07a9468916d0b1cc]
Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220416123040] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:2f305dee5a665a5943e6b107852db2e393161be627e50c3c07a9468916d0b1cc])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:2f305dee5a665a5943e6b107852db2e393161be627e50c3c07a9468916d0b1cc
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:2f305dee5a665a5943e6b107852db2e393161be627e50c3c07a9468916d0b1cc
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:2f305dee5a665a5943e6b107852db2e393161be627e50c3c07a9468916d0b1cc].
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.
BUILD FAILED in 3h 31m 23s
110 actionable tasks: 73 executed, 33 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/esyedybeb576a
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17 #116
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17/116/display/redirect?page=changes>
Changes:
[Kenneth Knowles] Upgrade to Gradle 7.4
[Kenneth Knowles] Remove Python module dependency on Dataflow worker
[noreply] [BEAM-13925] Dont double assign committers if author or other reviewer
[noreply] [BEAM-13739] Remove deprecated shallow clone funcs (#17362)
[noreply] [BEAM-11104] Pipe Continuation to DataSource level (#17334)
[noreply] [BEAM-11105] Basic Watermark Estimation (Wall Clock Observing) (#17267)
[noreply] Respect output coder for TextIO. (#17367)
[noreply] Merge pull request #17200 from [BEAM-12164]: fix the autoscaling backlog
[noreply] [BEAM-17035] Call python3 directly when it is available. (#17366)
[noreply] Merge pull request #17375: [BEAM-8691] Declare newer
------------------------------------------
[...truncated 71.10 KB...]
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
Apr 15, 2022 12:41:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-04-15T12:41:18.300Z: generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
Apr 15, 2022 4:01:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-15T16:01:05.786Z: Cancel request is committed for workflow job: 2022-04-15_05_32_48-13724693753058666410.
Apr 15, 2022 4:01:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-15T16:01:05.883Z: Cleaning up.
Apr 15, 2022 4:01:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-15T16:01:05.980Z: Stopping **** pool...
Apr 15, 2022 4:01:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-15T16:01:06.071Z: Stopping **** pool...
Apr 15, 2022 4:02:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-15T16:02:00.436Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Apr 15, 2022 4:02:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-15T16:02:00.504Z: Worker pool stopped.
Apr 15, 2022 4:02:06 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2022-04-15_05_32_48-13724693753058666410 finished with status CANCELLED.
Load test results for test (ID): 0435a4fb-2a03-4fa6-af5e-87148d28d2fa and timestamp: 2022-04-15T12:32:43.470000000Z:
Metric: Value:
dataflow_v2_java17_runtime_sec 12355.25
dataflow_v2_java17_total_bytes_count 4.53040508E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)
> Task :sdks:java:testing:load-tests:run FAILED
> Task :runners:google-cloud-dataflow-java:cleanUpDockerJavaImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220415123038
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:53461e94a63c37c564d048ebadf5ba0e688f41cb7d8cb63c337a89a1b8f3f47c
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220415123038]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:53461e94a63c37c564d048ebadf5ba0e688f41cb7d8cb63c337a89a1b8f3f47c]
Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220415123038] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:53461e94a63c37c564d048ebadf5ba0e688f41cb7d8cb63c337a89a1b8f3f47c])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:837acb140e871f602a1bfdea8e8c28ed0b1e9a458ab48d2d986ff464eca7dc12
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:837acb140e871f602a1bfdea8e8c28ed0b1e9a458ab48d2d986ff464eca7dc12
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:837acb140e871f602a1bfdea8e8c28ed0b1e9a458ab48d2d986ff464eca7dc12].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:53461e94a63c37c564d048ebadf5ba0e688f41cb7d8cb63c337a89a1b8f3f47c
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:53461e94a63c37c564d048ebadf5ba0e688f41cb7d8cb63c337a89a1b8f3f47c
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:53461e94a63c37c564d048ebadf5ba0e688f41cb7d8cb63c337a89a1b8f3f47c].
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.
BUILD FAILED in 3h 31m 52s
110 actionable tasks: 74 executed, 32 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/k52jyknmsqc7g
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17 #115
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17/115/display/redirect?page=changes>
Changes:
[relax] handle changing schemas in Storage API sink
[noreply] Fix a couple style issues (#17361)
[noreply] [BEAM-14287] Clean up staticcheck warnings in graph/coder (#17337)
[noreply] Improvements to dataflow job service for non-Python jobs. (#17338)
[noreply] Bump minimist (#17290)
[noreply] Bump ansi-regex (#17291)
[noreply] Bump nanoid (#17292)
[noreply] Bump lodash (#17293)
[noreply] Bump url-parse (#17294)
[noreply] Bump moment (#17328)
[noreply] Merge pull request #15549 from [BEAM-11997] Changed RedisIO
------------------------------------------
[...truncated 553.44 KB...]
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/windmill_cache_access.cc:40
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/windmill_cache_access.cc:40
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/windmill_cache_access.cc:40
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==
Apr 14, 2022 4:01:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-14T16:01:12.736Z: Cancel request is committed for workflow job: 2022-04-14_05_32_42-8770949662358378302.
Apr 14, 2022 4:01:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-14T16:01:13.004Z: Cleaning up.
Apr 14, 2022 4:01:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-14T16:01:13.099Z: Stopping **** pool...
Apr 14, 2022 4:01:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-14T16:01:13.153Z: Stopping **** pool...
Apr 14, 2022 4:02:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-14T16:02:08.314Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Apr 14, 2022 4:02:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-14T16:02:08.352Z: Worker pool stopped.
Apr 14, 2022 4:02:16 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2022-04-14_05_32_42-8770949662358378302 finished with status CANCELLED.
Load test results for test (ID): 177c6c38-46ba-410b-9699-1564a041fe5f and timestamp: 2022-04-14T12:32:37.729000000Z:
Exception in thread "main" Metric: Value:
dataflow_v2_java17_runtime_sec 12359.0
dataflow_v2_java17_total_bytes_count 3.92879535E10
java.lang.RuntimeException: Invalid job state: CANCELLED.
at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)
> Task :sdks:java:testing:load-tests:run FAILED
> Task :runners:google-cloud-dataflow-java:cleanUpDockerJavaImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220414123041
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:c8c32183cc6707a7b84326de30eea9462c7a3984216ecd24f7da6731a7402c7b
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220414123041]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:c8c32183cc6707a7b84326de30eea9462c7a3984216ecd24f7da6731a7402c7b]
Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220414123041] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:c8c32183cc6707a7b84326de30eea9462c7a3984216ecd24f7da6731a7402c7b])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:c8c32183cc6707a7b84326de30eea9462c7a3984216ecd24f7da6731a7402c7b
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:c8c32183cc6707a7b84326de30eea9462c7a3984216ecd24f7da6731a7402c7b
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:c8c32183cc6707a7b84326de30eea9462c7a3984216ecd24f7da6731a7402c7b].
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.
BUILD FAILED in 3h 31m 55s
110 actionable tasks: 74 executed, 32 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/klpmailx5y7ua
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17 #114
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17/114/display/redirect?page=changes>
Changes:
[Valentyn Tymofieiev] Add remaining Dataflow test suites for Python 3.9.
[Heejong Lee] [BEAM-14232] Only resolve artifacts in expanded environments for Java
[noreply] Fix test ordering issue (#17350)
[buqian] Do not pass null to MoreObjects.firstNonNull as default value
[ningkang0957] [BEAM-14288] Fixed flaky test
[noreply] [BEAM-14277] Disables Spanner change streams tests (#17346)
[noreply] [BEAM-14219] Run cleanup script to remove stale prebuilt SDK container
[Heejong Lee] [BEAM-14300] Fix Java precommit failure
[noreply] [BEAM-14116] Rollback "Chunk commit requests dynamically (#17004)"
[noreply] [BEAM-13982] A base class for run inference (#16970)
[ningkang0957] Enumerates all possible expected strings when asserting
[noreply] [BEAM-13966] Add pivot(), a non-deferred column operation on categorical
------------------------------------------
[...truncated 102.69 KB...]
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17/ws/src/sdks/java/container/license_scripts/pull_licenses_java.py",> line 58, in pull_from_url
url_read = urlopen(url)
File "/usr/lib/python3.8/urllib/request.py", line 222, in urlopen
return opener.open(url, data, timeout)
File "/usr/lib/python3.8/urllib/request.py", line 531, in open
response = meth(req, response)
File "/usr/lib/python3.8/urllib/request.py", line 640, in http_response
response = self.parent.error(
File "/usr/lib/python3.8/urllib/request.py", line 563, in error
result = self._call_chain(*args)
File "/usr/lib/python3.8/urllib/request.py", line 502, in _call_chain
result = func(*args)
File "/usr/lib/python3.8/urllib/request.py", line 755, in http_error_302
return self.parent.open(new, timeout=req.timeout)
File "/usr/lib/python3.8/urllib/request.py", line 531, in open
response = meth(req, response)
File "/usr/lib/python3.8/urllib/request.py", line 640, in http_response
response = self.parent.error(
File "/usr/lib/python3.8/urllib/request.py", line 563, in error
result = self._call_chain(*args)
File "/usr/lib/python3.8/urllib/request.py", line 502, in _call_chain
result = func(*args)
File "/usr/lib/python3.8/urllib/request.py", line 755, in http_error_302
return self.parent.open(new, timeout=req.timeout)
File "/usr/lib/python3.8/urllib/request.py", line 531, in open
response = meth(req, response)
File "/usr/lib/python3.8/urllib/request.py", line 640, in http_response
response = self.parent.error(
File "/usr/lib/python3.8/urllib/request.py", line 569, in error
return self._call_chain(*args)
File "/usr/lib/python3.8/urllib/request.py", line 502, in _call_chain
result = func(*args)
File "/usr/lib/python3.8/urllib/request.py", line 649, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 502: Bad Gateway
ERROR:root:Invalid url for slf4j-api-1.7.30: http://www.opensource.org/licenses/mit-license.php after 9 retries.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17/ws/src/sdks/java/container/license_scripts/pull_licenses_java.py",> line 58, in pull_from_url
url_read = urlopen(url)
File "/usr/lib/python3.8/urllib/request.py", line 222, in urlopen
return opener.open(url, data, timeout)
File "/usr/lib/python3.8/urllib/request.py", line 531, in open
response = meth(req, response)
File "/usr/lib/python3.8/urllib/request.py", line 640, in http_response
response = self.parent.error(
File "/usr/lib/python3.8/urllib/request.py", line 563, in error
result = self._call_chain(*args)
File "/usr/lib/python3.8/urllib/request.py", line 502, in _call_chain
result = func(*args)
File "/usr/lib/python3.8/urllib/request.py", line 755, in http_error_302
return self.parent.open(new, timeout=req.timeout)
File "/usr/lib/python3.8/urllib/request.py", line 531, in open
response = meth(req, response)
File "/usr/lib/python3.8/urllib/request.py", line 640, in http_response
response = self.parent.error(
File "/usr/lib/python3.8/urllib/request.py", line 563, in error
result = self._call_chain(*args)
File "/usr/lib/python3.8/urllib/request.py", line 502, in _call_chain
result = func(*args)
File "/usr/lib/python3.8/urllib/request.py", line 755, in http_error_302
return self.parent.open(new, timeout=req.timeout)
File "/usr/lib/python3.8/urllib/request.py", line 531, in open
response = meth(req, response)
File "/usr/lib/python3.8/urllib/request.py", line 640, in http_response
response = self.parent.error(
File "/usr/lib/python3.8/urllib/request.py", line 569, in error
return self._call_chain(*args)
File "/usr/lib/python3.8/urllib/request.py", line 502, in _call_chain
result = func(*args)
File "/usr/lib/python3.8/urllib/request.py", line 649, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 502: Bad Gateway
ERROR:root:Invalid url for slf4j-simple-1.7.30: http://www.opensource.org/licenses/mit-license.php after 9 retries.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17/ws/src/sdks/java/container/license_scripts/pull_licenses_java.py",> line 58, in pull_from_url
url_read = urlopen(url)
File "/usr/lib/python3.8/urllib/request.py", line 222, in urlopen
return opener.open(url, data, timeout)
File "/usr/lib/python3.8/urllib/request.py", line 531, in open
response = meth(req, response)
File "/usr/lib/python3.8/urllib/request.py", line 640, in http_response
response = self.parent.error(
File "/usr/lib/python3.8/urllib/request.py", line 563, in error
result = self._call_chain(*args)
File "/usr/lib/python3.8/urllib/request.py", line 502, in _call_chain
result = func(*args)
File "/usr/lib/python3.8/urllib/request.py", line 755, in http_error_302
return self.parent.open(new, timeout=req.timeout)
File "/usr/lib/python3.8/urllib/request.py", line 531, in open
response = meth(req, response)
File "/usr/lib/python3.8/urllib/request.py", line 640, in http_response
response = self.parent.error(
File "/usr/lib/python3.8/urllib/request.py", line 569, in error
return self._call_chain(*args)
File "/usr/lib/python3.8/urllib/request.py", line 502, in _call_chain
result = func(*args)
File "/usr/lib/python3.8/urllib/request.py", line 649, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 502: Bad Gateway
ERROR:root:Invalid url for classgraph-4.8.104: http://opensource.org/licenses/MIT. Retrying...
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17/ws/src/sdks/java/container/license_scripts/pull_licenses_java.py",> line 58, in pull_from_url
url_read = urlopen(url)
File "/usr/lib/python3.8/urllib/request.py", line 222, in urlopen
return opener.open(url, data, timeout)
File "/usr/lib/python3.8/urllib/request.py", line 531, in open
response = meth(req, response)
File "/usr/lib/python3.8/urllib/request.py", line 640, in http_response
response = self.parent.error(
File "/usr/lib/python3.8/urllib/request.py", line 563, in error
result = self._call_chain(*args)
File "/usr/lib/python3.8/urllib/request.py", line 502, in _call_chain
result = func(*args)
File "/usr/lib/python3.8/urllib/request.py", line 755, in http_error_302
return self.parent.open(new, timeout=req.timeout)
File "/usr/lib/python3.8/urllib/request.py", line 531, in open
response = meth(req, response)
File "/usr/lib/python3.8/urllib/request.py", line 640, in http_response
response = self.parent.error(
File "/usr/lib/python3.8/urllib/request.py", line 569, in error
return self._call_chain(*args)
File "/usr/lib/python3.8/urllib/request.py", line 502, in _call_chain
result = func(*args)
File "/usr/lib/python3.8/urllib/request.py", line 649, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 502: Bad Gateway
ERROR:root:Invalid url for kafka-1.16.3: http://opensource.org/licenses/MIT after 9 retries.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17/ws/src/sdks/java/container/license_scripts/pull_licenses_java.py",> line 58, in pull_from_url
url_read = urlopen(url)
File "/usr/lib/python3.8/urllib/request.py", line 222, in urlopen
return opener.open(url, data, timeout)
File "/usr/lib/python3.8/urllib/request.py", line 531, in open
response = meth(req, response)
File "/usr/lib/python3.8/urllib/request.py", line 640, in http_response
response = self.parent.error(
File "/usr/lib/python3.8/urllib/request.py", line 563, in error
result = self._call_chain(*args)
File "/usr/lib/python3.8/urllib/request.py", line 502, in _call_chain
result = func(*args)
File "/usr/lib/python3.8/urllib/request.py", line 755, in http_error_302
return self.parent.open(new, timeout=req.timeout)
File "/usr/lib/python3.8/urllib/request.py", line 531, in open
response = meth(req, response)
File "/usr/lib/python3.8/urllib/request.py", line 640, in http_response
response = self.parent.error(
File "/usr/lib/python3.8/urllib/request.py", line 569, in error
return self._call_chain(*args)
File "/usr/lib/python3.8/urllib/request.py", line 502, in _call_chain
result = func(*args)
File "/usr/lib/python3.8/urllib/request.py", line 649, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 502: Bad Gateway
ERROR:root:Invalid url for classgraph-4.8.104: http://opensource.org/licenses/MIT. Retrying...
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17/ws/src/sdks/java/container/license_scripts/pull_licenses_java.py",> line 58, in pull_from_url
url_read = urlopen(url)
File "/usr/lib/python3.8/urllib/request.py", line 222, in urlopen
return opener.open(url, data, timeout)
File "/usr/lib/python3.8/urllib/request.py", line 531, in open
response = meth(req, response)
File "/usr/lib/python3.8/urllib/request.py", line 640, in http_response
response = self.parent.error(
File "/usr/lib/python3.8/urllib/request.py", line 563, in error
result = self._call_chain(*args)
File "/usr/lib/python3.8/urllib/request.py", line 502, in _call_chain
result = func(*args)
File "/usr/lib/python3.8/urllib/request.py", line 755, in http_error_302
return self.parent.open(new, timeout=req.timeout)
File "/usr/lib/python3.8/urllib/request.py", line 531, in open
response = meth(req, response)
File "/usr/lib/python3.8/urllib/request.py", line 640, in http_response
response = self.parent.error(
File "/usr/lib/python3.8/urllib/request.py", line 569, in error
return self._call_chain(*args)
File "/usr/lib/python3.8/urllib/request.py", line 502, in _call_chain
result = func(*args)
File "/usr/lib/python3.8/urllib/request.py", line 649, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 502: Bad Gateway
ERROR:root:Invalid url for classgraph-4.8.104: http://opensource.org/licenses/MIT. Retrying...
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17/ws/src/sdks/java/container/license_scripts/pull_licenses_java.py",> line 58, in pull_from_url
url_read = urlopen(url)
File "/usr/lib/python3.8/urllib/request.py", line 222, in urlopen
return opener.open(url, data, timeout)
File "/usr/lib/python3.8/urllib/request.py", line 531, in open
response = meth(req, response)
File "/usr/lib/python3.8/urllib/request.py", line 640, in http_response
response = self.parent.error(
File "/usr/lib/python3.8/urllib/request.py", line 563, in error
result = self._call_chain(*args)
File "/usr/lib/python3.8/urllib/request.py", line 502, in _call_chain
result = func(*args)
File "/usr/lib/python3.8/urllib/request.py", line 755, in http_error_302
return self.parent.open(new, timeout=req.timeout)
File "/usr/lib/python3.8/urllib/request.py", line 531, in open
response = meth(req, response)
File "/usr/lib/python3.8/urllib/request.py", line 640, in http_response
response = self.parent.error(
File "/usr/lib/python3.8/urllib/request.py", line 569, in error
return self._call_chain(*args)
File "/usr/lib/python3.8/urllib/request.py", line 502, in _call_chain
result = func(*args)
File "/usr/lib/python3.8/urllib/request.py", line 649, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 502: Bad Gateway
ERROR:root:Invalid url for classgraph-4.8.104: http://opensource.org/licenses/MIT after 9 retries.
ERROR:root:['slf4j-api-1.7.30', 'slf4j-simple-1.7.30', 'kafka-1.16.3', 'classgraph-4.8.104']
ERROR:root:**************************************** Licenses were not able to be pulled automatically for some dependencies. Please search source code of the dependencies on the internet and add "license" and "notice" (if available) field to <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17/ws/src/sdks/java/container/license_scripts/dep_urls_java.yaml> for each missing license. Dependency List: [classgraph-4.8.104,kafka-1.16.3,slf4j-api-1.7.30,slf4j-simple-1.7.30]
INFO:root:pull_licenses_java.py failed. It took 373.621474 seconds with 16 threads.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17/ws/src/sdks/java/container/license_scripts/pull_licenses_java.py",> line 321, in <module>
raise RuntimeError('{n} error(s) occurred.'.format(n=len(error_msg)),
RuntimeError: ('1 error(s) occurred.', ['**************************************** Licenses were not able to be pulled automatically for some dependencies. Please search source code of the dependencies on the internet and add "license" and "notice" (if available) field to <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17/ws/src/sdks/java/container/license_scripts/dep_urls_java.yaml> for each missing license. Dependency List: [classgraph-4.8.104,kafka-1.16.3,slf4j-api-1.7.30,slf4j-simple-1.7.30]'])
> Task :sdks:java:container:pullLicenses FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:container:pullLicenses'.
> Process 'command './license_scripts/license_script.sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.
BUILD FAILED in 6m 51s
104 actionable tasks: 67 executed, 33 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/7apywnizdf2lq
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17 #113
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17/113/display/redirect?page=changes>
Changes:
[kamil.bregula] [BEAM-14215] Improve argument validation in SnowflakeIO
[benjamin.gonzalez] [BEAM-14013] Add PreCommit Kotlin examples Jenkins Job
[Andrew Pilloud] [BEAM-13151] Support multiple layers of AutoValue nesting
[Heejong Lee] [BEAM-14233] Merge requirements from expanded response for Java External
[benjamin.gonzalez] [BEAM-14013] Add spark, direct, flink runners as triggers for Kotlin
[noreply] [BEAM-13898] Add tests to the pubsubx package. (#17324)
[noreply] [BEAM-14285] Clean up Staticcheck Warnings in io packages (#17336)
[noreply] [BEAM-14187] Fix concurrency issue in IsmReaderImpl (#17201)
[noreply] [BEAM-14288] Skip flaking test
[noreply] Simplify specifying additional dependencies in Go SDK in XLang IOs
[noreply] [BEAM-14240] Clean staticcheck warnings in runner packages (#17340)
[Daniel Oliveira] [BEAM-13538] Workaround to fix go-licenses crash.
------------------------------------------
[...truncated 556.31 KB...]
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/windmill_cache_access.cc:40
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/windmill_cache_access.cc:40
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/windmill_cache_access.cc:40
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/windmill_cache_access.cc:40
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/windmill_cache_access.cc:40
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/windmill_cache_access.cc:40
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/windmill_cache_access.cc:40
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/windmill_cache_access.cc:40
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_pro
Apr 12, 2022 4:00:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-12T16:00:40.629Z: Cancel request is committed for workflow job: 2022-04-12_05_32_38-16584136735451216641.
Apr 12, 2022 4:00:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-12T16:00:40.690Z: Cleaning up.
Apr 12, 2022 4:00:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-12T16:00:40.763Z: Stopping **** pool...
Apr 12, 2022 4:00:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-12T16:00:40.832Z: Stopping **** pool...
Apr 12, 2022 4:01:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-12T16:01:33.548Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Apr 12, 2022 4:01:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-12T16:01:33.651Z: Worker pool stopped.
Apr 12, 2022 4:01:40 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2022-04-12_05_32_38-16584136735451216641 finished with status CANCELLED.
Load test results for test (ID): f37135c9-cf31-48db-840a-2d9fcc7df2c6 and timestamp: 2022-04-12T12:32:32.859000000Z:
Metric: Value:
dataflow_v2_java17_runtime_sec 12307.586
dataflow_v2_java17_total_bytes_count 2.85776675E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)
> Task :sdks:java:testing:load-tests:run FAILED
> Task :runners:google-cloud-dataflow-java:cleanUpDockerJavaImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220412123038
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:0297880347919b8404a0e4f2e0710b1d062f0edb1384b21986d0e776bbc6aa3f
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220412123038]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:0297880347919b8404a0e4f2e0710b1d062f0edb1384b21986d0e776bbc6aa3f]
Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220412123038] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:0297880347919b8404a0e4f2e0710b1d062f0edb1384b21986d0e776bbc6aa3f])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:0297880347919b8404a0e4f2e0710b1d062f0edb1384b21986d0e776bbc6aa3f
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:0297880347919b8404a0e4f2e0710b1d062f0edb1384b21986d0e776bbc6aa3f
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:0297880347919b8404a0e4f2e0710b1d062f0edb1384b21986d0e776bbc6aa3f].
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.
BUILD FAILED in 3h 31m 22s
110 actionable tasks: 73 executed, 33 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/2huuoa7qg7lim
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17 #112
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17/112/display/redirect>
Changes:
------------------------------------------
[...truncated 853.62 KB...]
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmi
Apr 11, 2022 4:01:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-11T16:01:39.528Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Apr 11, 2022 4:01:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-11T16:01:39.598Z: Worker pool stopped.
Apr 11, 2022 4:01:47 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2022-04-11_05_32_40-13881262804693704859 finished with status CANCELLED.
Load test results for test (ID): 3555057e-2d48-4bc3-ac06-f1e42cd3fd98 and timestamp: 2022-04-11T12:32:33.996000000Z:
Metric: Value:
dataflow_v2_java17_runtime_sec 12364.825
dataflow_v2_java17_total_bytes_count 2.65637155E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)
> Task :sdks:java:testing:load-tests:run FAILED
> Task :runners:google-cloud-dataflow-java:cleanUpDockerJavaImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220411123036
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:0a4f249317af477039f94a21c8d09d2748f9a67a49e53e178a857fa36329751d
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220411123036]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:0a4f249317af477039f94a21c8d09d2748f9a67a49e53e178a857fa36329751d]
Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220411123036] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:0a4f249317af477039f94a21c8d09d2748f9a67a49e53e178a857fa36329751d])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:0a4f249317af477039f94a21c8d09d2748f9a67a49e53e178a857fa36329751d
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:0a4f249317af477039f94a21c8d09d2748f9a67a49e53e178a857fa36329751d
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:0a4f249317af477039f94a21c8d09d2748f9a67a49e53e178a857fa36329751d].
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.
BUILD FAILED in 3h 31m 30s
110 actionable tasks: 73 executed, 33 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/jxqpntg3gnezk
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17 #111
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17/111/display/redirect?page=changes>
Changes:
[chamikaramj] Re-raise exceptions swallowed in several Python I/O connectors
[noreply] Merge pull request #16928: [BEAM-11971] Re add reverted timer
------------------------------------------
[...truncated 85.92 KB...]
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/windmill_cache_access.cc:40
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/windmill_cache_access.cc:40
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/windmill_cache_access.cc:40
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/windmill_cache_access.cc:40
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/windmill_cache_access.cc:40
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/windmill_cache_access.cc:40
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/windmill_cache_access.cc:40
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/windmill_cache_access.cc:40
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/windmill_cache_access.cc:40
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/windmill_cache_access.cc:40
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/windmill_cache_access.cc:40
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/windmill_cache_access.cc:40
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
Apr 10, 2022 12:38:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-04-10T12:38:51.118Z: generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
Apr 10, 2022 4:00:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-10T16:00:41.808Z: Cancel request is committed for workflow job: 2022-04-10_05_32_58-12731610135292703172.
Apr 10, 2022 4:00:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-10T16:00:42.282Z: Cleaning up.
Apr 10, 2022 4:00:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-10T16:00:42.603Z: Stopping **** pool...
Apr 10, 2022 4:00:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-10T16:00:42.647Z: Stopping **** pool...
Apr 10, 2022 4:01:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-10T16:01:37.779Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Apr 10, 2022 4:01:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-10T16:01:37.815Z: Worker pool stopped.
Apr 10, 2022 4:01:43 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2022-04-10_05_32_58-12731610135292703172 finished with status CANCELLED.
Load test results for test (ID): e587a666-2285-4158-aa53-0f1d12ccb15d and timestamp: 2022-04-10T12:32:53.321000000Z:
Metric: Value:
dataflow_v2_java17_runtime_sec 12330.282
dataflow_v2_java17_total_bytes_count 4.13022115E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)
> Task :sdks:java:testing:load-tests:run FAILED
> Task :runners:google-cloud-dataflow-java:cleanUpDockerJavaImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220410123035
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:72a303835871089201e801a436c8ee2eb92fc6c2dda32997e97e694aa7e1d69b
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220410123035]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:72a303835871089201e801a436c8ee2eb92fc6c2dda32997e97e694aa7e1d69b]
Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220410123035] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:72a303835871089201e801a436c8ee2eb92fc6c2dda32997e97e694aa7e1d69b])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:72a303835871089201e801a436c8ee2eb92fc6c2dda32997e97e694aa7e1d69b
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:72a303835871089201e801a436c8ee2eb92fc6c2dda32997e97e694aa7e1d69b
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:72a303835871089201e801a436c8ee2eb92fc6c2dda32997e97e694aa7e1d69b].
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.
BUILD FAILED in 3h 31m 27s
110 actionable tasks: 73 executed, 33 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/xlrh2zgdmm7ii
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17 #110
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17/110/display/redirect?page=changes>
Changes:
[benjamin.gonzalez] [BEAM-11714] Change spotBugs jenkins config
[Robert Bradshaw] Cleanup docs on Shared.
[Kyle Weaver] Nit: correct description for precommit cron jobs.
[benjamin.gonzalez] [BEAM-11714] Add dummy class for testing
[benjamin.gonzalez] [BEAM-11714] Remove dummy class used for testing
[benjamin.gonzalez] [BEAM-11714] Spotbugs print toJenkins UI precommit_Java17
[noreply] [BEAM-13767] Remove eclipse plugin as it generates a lot of unused tasks
[noreply] [BEAM-10708] Updated beam_sql error message (#17314)
[noreply] [BEAM-14281] add as_deterministic_coder to nullable coder (#17322)
[noreply] Improvements to Beam/Spark quickstart. (#17129)
[chamikaramj] Disable BigQueryIOStorageWriteIT for Runner v2 test suite
------------------------------------------
[...truncated 376.46 KB...]
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
Apr 09, 2022 4:00:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-09T16:00:34.827Z: Cancel request is committed for workflow job: 2022-04-09_05_32_53-12302577311782903590.
Apr 09, 2022 4:00:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-09T16:00:34.904Z: Cleaning up.
Apr 09, 2022 4:00:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-09T16:00:34.982Z: Stopping **** pool...
Apr 09, 2022 4:00:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-09T16:00:35.025Z: Stopping **** pool...
Apr 09, 2022 4:01:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-09T16:01:35.116Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Apr 09, 2022 4:01:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-09T16:01:35.184Z: Worker pool stopped.
Apr 09, 2022 4:01:41 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2022-04-09_05_32_53-12302577311782903590 finished with status CANCELLED.
Load test results for test (ID): 08c559da-265c-438e-8b4d-f608d8c83deb and timestamp: 2022-04-09T12:32:40.587000000Z:
Metric: Value:
dataflow_v2_java17_runtime_sec 12320.833
dataflow_v2_java17_total_bytes_count 4.00156759E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)
> Task :sdks:java:testing:load-tests:run FAILED
> Task :runners:google-cloud-dataflow-java:cleanUpDockerJavaImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220409123043
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:df5bb150e9ffb293214f67ec0c73b3ecc1e326f7adf9a974bcfecfcfeec7d625
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220409123043]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:df5bb150e9ffb293214f67ec0c73b3ecc1e326f7adf9a974bcfecfcfeec7d625]
Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220409123043] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:df5bb150e9ffb293214f67ec0c73b3ecc1e326f7adf9a974bcfecfcfeec7d625])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:df5bb150e9ffb293214f67ec0c73b3ecc1e326f7adf9a974bcfecfcfeec7d625
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:df5bb150e9ffb293214f67ec0c73b3ecc1e326f7adf9a974bcfecfcfeec7d625
ERROR: (gcloud.container.images.delete) Not found: response: {'docker-distribution-api-version': 'registry/2.0', 'content-type': 'application/json', 'date': 'Sat, 09 Apr 2022 16:01:48 GMT', 'server': 'Docker Registry', 'cache-control': 'private', 'x-xss-protection': '0', 'x-frame-options': 'SAMEORIGIN', 'transfer-encoding': 'chunked', 'status': '404', 'content-length': '168', '-content-encoding': 'gzip'}
Failed to compute blob liveness for manifest: 'sha256:df5bb150e9ffb293214f67ec0c73b3ecc1e326f7adf9a974bcfecfcfeec7d625': None
> Task :runners:google-cloud-dataflow-java:cleanUpDockerJavaImages FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17/ws/src/runners/google-cloud-dataflow-java/build.gradle'> line: 297
* What went wrong:
Execution failed for task ':runners:google-cloud-dataflow-java:cleanUpDockerJavaImages'.
> Process 'command './scripts/cleanup_untagged_gcr_images.sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.
BUILD FAILED in 3h 31m 22s
110 actionable tasks: 74 executed, 32 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/wmmfpbdzoqmja
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17 - Build # 109 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17 - Build # 109 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17/109/ to view the results.
Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17 #108
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17/108/display/redirect?page=changes>
Changes:
[noreply] Avoid pr-bot state desync (#17299)
[noreply] [BEAM-14259] Clean up staticcheck warnings in the exec package (#17285)
[noreply] Minor: Prefer registered schema in SQL docs (#17298)
[Kyle Weaver] [BEAM-14262] Update plugins for Dockerized Jenkins.
[Kyle Weaver] Add ansicolor and ws-cleanup plugins.
[noreply] [Playground] add meta tags (#17207)
[noreply] fixes golint and deprecated issues in recent Go SDK import (#17304)
[noreply] [BEAM-14266] Replace deprecated ptypes package uses (#17302)
[noreply] [BEAM-11936] Fix rawtypes warnings in SnowflakeIO (#17257)
[noreply] Merge pull request #17262: [BEAM-14244] Use the supplied output
------------------------------------------
[...truncated 90.68 KB...]
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
Apr 07, 2022 12:42:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-04-07T12:42:00.955Z: generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
Apr 07, 2022 4:00:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-07T16:00:46.424Z: Cancel request is committed for workflow job: 2022-04-07_05_32_40-9479556150086241544.
Apr 07, 2022 4:00:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-07T16:00:46.508Z: Cleaning up.
Apr 07, 2022 4:00:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-07T16:00:46.593Z: Stopping **** pool...
Apr 07, 2022 4:00:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-07T16:00:46.653Z: Stopping **** pool...
Apr 07, 2022 4:01:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-07T16:01:43.819Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Apr 07, 2022 4:01:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-07T16:01:43.862Z: Worker pool stopped.
Apr 07, 2022 4:01:49 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2022-04-07_05_32_40-9479556150086241544 finished with status CANCELLED.
Load test results for test (ID): 36061655-0445-4a2e-8ace-0a89d2cb57c9 and timestamp: 2022-04-07T12:32:34.775000000Z:
Metric: Value:
dataflow_v2_java17_runtime_sec 12340.917
dataflow_v2_java17_total_bytes_count 2.70217214E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)
> Task :sdks:java:testing:load-tests:run FAILED
> Task :runners:google-cloud-dataflow-java:cleanUpDockerJavaImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220407123036
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ef93a3b6b99df0cbecc4e8d9dd63fda639570d96e4ee4cb9d7a37ad26e93235c
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220407123036]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ef93a3b6b99df0cbecc4e8d9dd63fda639570d96e4ee4cb9d7a37ad26e93235c]
Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220407123036] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ef93a3b6b99df0cbecc4e8d9dd63fda639570d96e4ee4cb9d7a37ad26e93235c])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ef93a3b6b99df0cbecc4e8d9dd63fda639570d96e4ee4cb9d7a37ad26e93235c
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ef93a3b6b99df0cbecc4e8d9dd63fda639570d96e4ee4cb9d7a37ad26e93235c
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ef93a3b6b99df0cbecc4e8d9dd63fda639570d96e4ee4cb9d7a37ad26e93235c].
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.
BUILD FAILED in 3h 31m 34s
110 actionable tasks: 73 executed, 33 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/cos6vkydokaz4
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17 #107
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17/107/display/redirect?page=changes>
Changes:
[bingyeli] update query
[Robert Bradshaw] [BEAM-14250] Fix coder registration for types defined in __main__.
[johnjcasey] [BEAM-14256] update SpEL dependency to 5.3.18.RELEASE
[johnjcasey] [BEAM-14256] remove .RELEASE
[dannymccormick] Fix dependency issue causing failures
[Kyle Weaver] [BEAM-9649] Add region option to Mongo Dataflow test.
[noreply] Allow get_coder(None).
[noreply] [BEAM-13015] Disable retries for fnapi grpc channels which otherwise
[noreply] [BEAM-13952] Sickbay
[noreply] BEAM-14235 parquetio module does not parse PEP-440 compliant Pyarrow
[noreply] [Website] Contribution guide page indent bug fix (#17287)
[noreply] [BEAM-10976] Document go sdk bundle finalization (#17048)
[noreply] [BEAM-13829] Expose status API from Go SDK Harness (#16957)
------------------------------------------
[...truncated 636.76 KB...]
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmi
Apr 06, 2022 4:02:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-06T16:02:07.953Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Apr 06, 2022 4:02:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-06T16:02:08.003Z: Worker pool stopped.
Apr 06, 2022 4:02:18 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2022-04-06_05_34_14-17492783050096433347 finished with status CANCELLED.
Load test results for test (ID): 7c5381e5-9149-4612-9a16-feaa4e40361b and timestamp: 2022-04-06T12:34:08.158000000Z:
Metric: Value:
dataflow_v2_java17_runtime_sec 12270.523
dataflow_v2_java17_total_bytes_count 2.59884868E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)
> Task :sdks:java:testing:load-tests:run FAILED
> Task :runners:google-cloud-dataflow-java:cleanUpDockerJavaImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220406123036
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:5233ac8e4bfedcb4cff524cb83131c0b1e4fba7448433c636ca8b5ddbd966942
ERROR: (gcloud.container.images.untag) Image could not be found: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220406123036]
> Task :runners:google-cloud-dataflow-java:cleanUpDockerJavaImages FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17/ws/src/runners/google-cloud-dataflow-java/build.gradle'> line: 294
* What went wrong:
Execution failed for task ':runners:google-cloud-dataflow-java:cleanUpDockerJavaImages'.
> Process 'command 'gcloud'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.
BUILD FAILED in 3h 31m 57s
110 actionable tasks: 74 executed, 32 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/3pof7unckot2g
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17 #106
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17/106/display/redirect?page=changes>
Changes:
[benjamin.gonzalez] [BEAM-8970] Add docs to run wordcount example on portable Spark Runner
[Kiley Sok] Update python container version
[benjamin.gonzalez] [BEAM-8970] Add period to end of sentence
[Kyle Weaver] Add self-descriptive message for expected errors.
[noreply] Add --dataflowServiceOptions=enable_prime to useUnifiedWorker conditions
[noreply] [BEAM-10529] nullable xlang coder (#16923)
[noreply] Fix go fmt break in core/typex/special.go (#17266)
[noreply] [BEAM-5436] Add doc page on Go cross compilation. (#17256)
[noreply] Pr-bot Don't count all reviews as approvals (#17269)
[noreply] Fix postcommits (#17263)
[noreply] [BEAM-14241] Address staticcheck warnings in boot.go (#17264)
[noreply] [BEAM-14157] GrpcWindmillServer: Use stream specific boolean to do
[noreply] [BEAM-10582] Allow (and test) pyarrow 7 (#17229)
[noreply] [BEAM-13519] Solve race issues when the server responds with an error
------------------------------------------
[...truncated 754.22 KB...]
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/windmill_cache_access.cc:40
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/windmill_cache_access.cc:40
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/windmill_cache_access.cc:40
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/windmill_cache_access.cc:40
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/windmill_cache_access.cc:40
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/d
Apr 05, 2022 4:00:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-05T16:00:57.319Z: Cancel request is committed for workflow job: 2022-04-05_05_32_51-3666642527379357457.
Apr 05, 2022 4:00:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-05T16:00:57.444Z: Cleaning up.
Apr 05, 2022 4:00:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-05T16:00:57.512Z: Stopping **** pool...
Apr 05, 2022 4:00:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-05T16:00:57.572Z: Stopping **** pool...
Apr 05, 2022 4:01:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-05T16:01:56.238Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Apr 05, 2022 4:01:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-05T16:01:56.273Z: Worker pool stopped.
Apr 05, 2022 4:02:02 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2022-04-05_05_32_51-3666642527379357457 finished with status CANCELLED.
Load test results for test (ID): 61edf798-1b2b-4693-ac07-01cede3f0333 and timestamp: 2022-04-05T12:32:46.389000000Z:
Metric: Value:
dataflow_v2_java17_runtime_sec 12341.89
dataflow_v2_java17_total_bytes_count 3.73281742E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)
> Task :sdks:java:testing:load-tests:run FAILED
> Task :runners:google-cloud-dataflow-java:cleanUpDockerJavaImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220405123037
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ef2e32b5e5b62e6653188e891ea43a7f16a95758d2a2a60ca5a5bf9a17408f92
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220405123037]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ef2e32b5e5b62e6653188e891ea43a7f16a95758d2a2a60ca5a5bf9a17408f92]
Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220405123037] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ef2e32b5e5b62e6653188e891ea43a7f16a95758d2a2a60ca5a5bf9a17408f92])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ef2e32b5e5b62e6653188e891ea43a7f16a95758d2a2a60ca5a5bf9a17408f92
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ef2e32b5e5b62e6653188e891ea43a7f16a95758d2a2a60ca5a5bf9a17408f92
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ef2e32b5e5b62e6653188e891ea43a7f16a95758d2a2a60ca5a5bf9a17408f92].
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.
BUILD FAILED in 3h 32m 4s
110 actionable tasks: 73 executed, 33 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/6e3fmf37nc6ek
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17 #105
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17/105/display/redirect>
Changes:
------------------------------------------
[...truncated 458.36 KB...]
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/windmill_cache_access.cc:40
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/windmill_cache_access.cc:40
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/windmill_cache_access.cc:40
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/windmill_cache_access.cc:40
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/windmill_cache_access.cc:40
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/windmill_cache_access.cc:40
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on the backend. The work has already completed or will be retried. This is expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/strea
Apr 04, 2022 4:03:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-04T16:03:21.050Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Apr 04, 2022 4:03:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-04T16:03:21.112Z: Worker pool stopped.
Apr 04, 2022 4:03:28 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2022-04-04_05_32_41-386720858990461328 finished with status CANCELLED.
Load test results for test (ID): 2383320e-7504-40a1-ad91-cb6e77e8b3e2 and timestamp: 2022-04-04T12:32:35.986000000Z:
Metric: Value:
dataflow_v2_java17_runtime_sec 12339.642
dataflow_v2_java17_total_bytes_count 4.13967225E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED.
at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)
> Task :sdks:java:testing:load-tests:run FAILED
> Task :runners:google-cloud-dataflow-java:cleanUpDockerJavaImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220404123037
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:45c5397c5fa9a9de944159d473d9f56fc2f54a4148a3c72764fddfe40e2d0a3e
Deleted: sha256:f38cc01db745182ff7a12a33f7d29a94450e7955475e328dbfd5d6679df2632e
Deleted: sha256:073336a4ab9c6ac10620325143a6a21cd777bd14844d213ea25473db6ab36516
Deleted: sha256:4fe965c2eae1d1cc36d550e0ca852eeb3d6bc4eaf65fb308a84da3841e726169
Deleted: sha256:283e8a8e965c8042cbbeb09d5447041b9a412c1c466192098f579946d3ae0275
Deleted: sha256:4c7d9132b334603c78570b9394031051f9a2da5bb8ddd534c8af8976eb736425
Deleted: sha256:393cb90935a2dfd2ccfd37144d81ca63e2b5336f326e77cc39c300a0dc219b99
Deleted: sha256:ec5d3fea0ae0c7b9fd52a3ccb56eed6cb0b005933c4b1b5bf5cfb343e808d5c9
Deleted: sha256:4b43eebb0f4fa7225f7e64ce89bf2af024b9d52e0d305232224111caa9d3be1b
Deleted: sha256:ce5d7e50f946084f9c3275530cbb6925429e2680d70801ab7d4c8c4f7eb75167
Deleted: sha256:e2d9238788a94ae0b3b31fd7050972e7296b293881ee1e1a83462373ce3cc522
Deleted: sha256:d6c833b54a19c4bc44f94a20259bfa11e72935f683b20f362135af2b055dd32d
Deleted: sha256:ab584f8cc7c000a513240b6ccb35f88a83a2e66acf5fd862185bf354bd83d941
Deleted: sha256:3a7378f2f2a5bfd4a21b203280a311ec06441366a9872beb37e22300f62dde04
Deleted: sha256:0673b74f8cc304eacb3a5ccd2061ac5ce9e7e47b1cf287163a4f88e360f65b47
Deleted: sha256:de2eb27e40434e484c7d44348ba38c33d8bab9b69e8579d1b427d162850794ad
Deleted: sha256:d684a28793d0c737a5dd9630ffd9b736c611776f7343bdde4a0d33b6033256aa
Deleted: sha256:7a2e75438ed419d51bd23de46fccfd3510b05f98ee8445ef09d8060990bbf5e7
Deleted: sha256:f4e8a583db4780006d08aaf5430a075314cb384a1a670cabfae26badce7332b9
Deleted: sha256:9d847d2693a2ff84a4ab706e3aebaabad342902028512191fb08eeb3848178fc
Deleted: sha256:77fb90446d06ecf3c9fb2668059da8ac083ac72061a1940e6fe7f44b202deb06
Deleted: sha256:3fb64211c2f47dc0cf9aa4fe9f87a4b967c7c01f5fa60313cb84bbd104debc9e
Deleted: sha256:f7f6b7b8fe430f0221ce5cbdffbad04f6f7ea5707cc57a58b2e633909e341f48
Deleted: sha256:c45fc619af0b548f520a2fbcc8feb4db24c79b21f4880bfd490cdc6891cc1ead
Deleted: sha256:4e401894ffb4918a306baeca124fcf46f8e3cbf56a2bcbc326dfac281f154f9a
Deleted: sha256:c37c8a4d65d03507aaa3c88508270d1f290ed91b7a46e33f05bb4b6715b7b610
Deleted: sha256:5508bea2f0be5b66f54591f22053e73883d51fdf4834de963e3e8dab94534052
Deleted: sha256:6778903f746a88bf72c1443c0d8634a82a3634ac73c94fac6102f1e388bfa57e
Deleted: sha256:be70fcf12cfee61004019b2438f91712281f81c250e89035b68b8169726a0fe9
Deleted: sha256:8cbb3ead3217b35e9217e8041c7ad34e59350c1b49da667855d2fc04b9969efe
Deleted: sha256:e0ae11ea9dbfa3398c7555362a8726f00e02b7e79a32cdab5d3d49fa9d902449
Deleted: sha256:4e6578a82049418c5e65286a1896da15b1689fa95f7bda334942339922bf5f53
Deleted: sha256:f1b27c2e719be61ea47e42183321baab4a64a048db130414c3f337f34ea9ca7e
Deleted: sha256:702d6242cc7552eb938a97b9064a875d4db995eb546216e74945a8844bf3388a
Deleted: sha256:cc61c28c05d3668e15b2c408ab0a939a955f7aabdd21f01c60fcec93456b9c08
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220404123037]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:45c5397c5fa9a9de944159d473d9f56fc2f54a4148a3c72764fddfe40e2d0a3e]
Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220404123037] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:45c5397c5fa9a9de944159d473d9f56fc2f54a4148a3c72764fddfe40e2d0a3e])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:45c5397c5fa9a9de944159d473d9f56fc2f54a4148a3c72764fddfe40e2d0a3e
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:45c5397c5fa9a9de944159d473d9f56fc2f54a4148a3c72764fddfe40e2d0a3e
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:45c5397c5fa9a9de944159d473d9f56fc2f54a4148a3c72764fddfe40e2d0a3e].
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.
BUILD FAILED in 3h 33m 12s
110 actionable tasks: 73 executed, 33 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/6aumrwiig2kai
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17 - Build # 104 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17 - Build # 104 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17/104/ to view the results.
beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17 - Build # 103 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17 - Build # 103 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17/103/ to view the results.
beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17 - Build # 102 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17 - Build # 102 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17/102/ to view the results.