You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@beam.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2022/03/20 13:07:29 UTC
Build failed in Jenkins: beam_LoadTests_Java_ParDo_Dataflow_V2_Streaming_Java17 #90
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_ParDo_Dataflow_V2_Streaming_Java17/90/display/redirect?page=changes>
Changes:
[noreply] [BEAM-14122] Upgrade pip-licenses dependency (#17132)
------------------------------------------
[...truncated 286.97 KB...]
> Task :sdks:java:expansion-service:compileJava UP-TO-DATE
> Task :sdks:java:expansion-service:classes UP-TO-DATE
> Task :sdks:java:expansion-service:jar UP-TO-DATE
> Task :sdks:java:io:kafka:compileJava UP-TO-DATE
> Task :sdks:java:io:kafka:classes UP-TO-DATE
> Task :sdks:java:io:kafka:jar UP-TO-DATE
> Task :sdks:java:container:java17:copyDockerfileDependencies UP-TO-DATE
> Task :sdks:java:io:google-cloud-platform:compileJava UP-TO-DATE
> Task :sdks:java:io:google-cloud-platform:classes UP-TO-DATE
> Task :sdks:java:io:google-cloud-platform:jar UP-TO-DATE
> Task :sdks:java:testing:load-tests:compileJava UP-TO-DATE
> Task :sdks:java:testing:load-tests:classes UP-TO-DATE
> Task :sdks:java:testing:load-tests:jar UP-TO-DATE
> Task :runners:google-cloud-dataflow-java:compileJava UP-TO-DATE
> Task :runners:google-cloud-dataflow-java:classes UP-TO-DATE
> Task :runners:google-cloud-dataflow-java:jar UP-TO-DATE
> Task :sdks:java:container:goBuild
/home/jenkins/go/bin/go1.16.12 build -o ./build/target/linux_amd64/boot boot.go
> Task :sdks:java:container:java17:copySdkHarnessLauncher
Execution optimizations have been disabled for task ':sdks:java:container:java17:copySdkHarnessLauncher' to ensure correctness due to the following reasons:
- Gradle detected a problem with the following location: '<https://ci-beam.apache.org/job/beam_LoadTests_Java_ParDo_Dataflow_V2_Streaming_Java17/ws/src/sdks/java/container/build/target'.> Reason: Task ':sdks:java:container:java17:copySdkHarnessLauncher' uses this output of task ':sdks:java:container:downloadCloudProfilerAgent' without declaring an explicit or implicit dependency. This can lead to incorrect results being produced, depending on what order the tasks are executed. Please refer to https://docs.gradle.org/7.3.2/userguide/validation_problems.html#implicit_dependency for more details about this problem.
- Gradle detected a problem with the following location: '<https://ci-beam.apache.org/job/beam_LoadTests_Java_ParDo_Dataflow_V2_Streaming_Java17/ws/src/sdks/java/container/build/target'.> Reason: Task ':sdks:java:container:java17:copySdkHarnessLauncher' uses this output of task ':sdks:java:container:pullLicenses' without declaring an explicit or implicit dependency. This can lead to incorrect results being produced, depending on what order the tasks are executed. Please refer to https://docs.gradle.org/7.3.2/userguide/validation_problems.html#implicit_dependency for more details about this problem.
> Task :sdks:java:container:java17:dockerPrepare
> Task :sdks:java:container:java17:docker
> Task :runners:google-cloud-dataflow-java:buildAndPushDockerJavaContainer
WARNING: `gcloud docker` will not be supported for Docker client versions above 18.03.
As an alternative, use `gcloud auth configure-docker` to configure `docker` to
use `gcloud` as a credential helper, then use `docker` as you would for non-GCR
registries, e.g. `docker pull gcr.io/project-id/my-image`. Add
`--verbosity=error` to silence this warning: `gcloud docker
--verbosity=error -- pull gcr.io/project-id/my-image`.
See: https://cloud.google.com/container-registry/docs/support/deprecation-notices#gcloud-docker
The push refers to repository [us.gcr.io/apache-beam-testing/java-postcommit-it/java]
6db9b26143d5: Preparing
cf2f036f1441: Preparing
f4cb6f0bf7a3: Preparing
6f6612a5111f: Preparing
4d119a33f964: Preparing
e433cf8199de: Preparing
287b505c5f48: Preparing
cb9d3722e205: Preparing
b648214b4e91: Preparing
2ea9565e850d: Preparing
9c53ea4aa0bb: Preparing
45ccd9372db1: Preparing
8fd528b23689: Preparing
de337572acf9: Preparing
383ff12ef634: Preparing
327e42081bbe: Preparing
6e632f416458: Preparing
e019be289189: Preparing
c9a63110150b: Preparing
8fd528b23689: Waiting
2ea9565e850d: Waiting
287b505c5f48: Waiting
b648214b4e91: Waiting
de337572acf9: Waiting
6e632f416458: Waiting
c9a63110150b: Waiting
e019be289189: Waiting
cb9d3722e205: Waiting
327e42081bbe: Waiting
383ff12ef634: Waiting
9c53ea4aa0bb: Waiting
e433cf8199de: Waiting
45ccd9372db1: Waiting
f4cb6f0bf7a3: Pushed
cf2f036f1441: Pushed
4d119a33f964: Pushed
6db9b26143d5: Pushed
6f6612a5111f: Pushed
e433cf8199de: Pushed
cb9d3722e205: Pushed
287b505c5f48: Pushed
b648214b4e91: Pushed
2ea9565e850d: Pushed
de337572acf9: Layer already exists
383ff12ef634: Layer already exists
327e42081bbe: Layer already exists
45ccd9372db1: Pushed
6e632f416458: Layer already exists
e019be289189: Layer already exists
c9a63110150b: Layer already exists
8fd528b23689: Pushed
9c53ea4aa0bb: Pushed
20220320125858: digest: sha256:10620c3e761c05cecf2b69718acbe4fec023259ea0cd8299bd137b2e23badcc8 size: 4314
> Task :sdks:java:testing:load-tests:run
Mar 20, 2022 12:59:27 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Mar 20, 2022 12:59:28 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 207 files. Enable logging at DEBUG level to see which files will be staged.
Mar 20, 2022 12:59:29 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: ParDo(TimeMonitor)
Mar 20, 2022 12:59:29 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Mar 20, 2022 12:59:31 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 207 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Mar 20, 2022 12:59:32 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 207 files cached, 0 files newly uploaded in 0 seconds
Mar 20, 2022 12:59:32 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Mar 20, 2022 12:59:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <90966 bytes, hash 3325351d4a6c815c5914782d3201f98993947c1ff202a6f2060e2ebb71a159e8> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-MyU1HUpsgVxZFHgtMgH5iZOUfB_yAqbyBg4uu3GhWeg.pb
Mar 20, 2022 12:59:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Mar 20, 2022 12:59:34 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@e57e5d6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6dfa915a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@78054f54, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@cb7fa71, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3dffc764, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4b6e1c0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@561b61ed, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@654c7d2d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26cb5207, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@15400fff, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@18d910b3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1e7ab390, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@625d9132, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77774571, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@277b8fa4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6cd64ee8, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@620c8641, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2f1d0bbc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5460b754, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@a9f023e]
Mar 20, 2022 12:59:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Mar 20, 2022 12:59:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s3
Mar 20, 2022 12:59:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(ByteMonitor) as step s4
Mar 20, 2022 12:59:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Step: 0 as step s5
Mar 20, 2022 12:59:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor)2 as step s6
Mar 20, 2022 12:59:34 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.38.0-SNAPSHOT
Mar 20, 2022 12:59:34 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-03-20_05_59_34-16397546146545212736?project=apache-beam-testing
Mar 20, 2022 12:59:34 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2022-03-20_05_59_34-16397546146545212736
Mar 20, 2022 12:59:34 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2022-03-20_05_59_34-16397546146545212736
Mar 20, 2022 12:59:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2022-03-20T12:59:40.144Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java170dataflow0v20streaming0pardo04-jenkins-03-23ww. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Mar 20, 2022 12:59:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-20T12:59:44.329Z: Worker configuration: e2-standard-2 in us-central1-b.
Mar 20, 2022 12:59:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-20T12:59:45.030Z: Expanding SplittableParDo operations into optimizable parts.
Mar 20, 2022 12:59:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-20T12:59:45.064Z: Expanding CollectionToSingleton operations into optimizable parts.
Mar 20, 2022 12:59:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-20T12:59:45.124Z: Expanding CoGroupByKey operations into optimizable parts.
Mar 20, 2022 12:59:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-20T12:59:45.163Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Mar 20, 2022 12:59:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-20T12:59:45.193Z: Expanding GroupByKey operations into streaming Read/Write steps
Mar 20, 2022 12:59:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-20T12:59:45.255Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Mar 20, 2022 12:59:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-20T12:59:45.343Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Mar 20, 2022 12:59:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-20T12:59:45.381Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Mar 20, 2022 12:59:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-20T12:59:45.405Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Mar 20, 2022 12:59:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-20T12:59:45.437Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Mar 20, 2022 12:59:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-20T12:59:45.465Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Mar 20, 2022 12:59:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-20T12:59:45.500Z: Fusing consumer ParDo(TimeMonitor)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Mar 20, 2022 12:59:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-20T12:59:45.528Z: Fusing consumer ParDo(ByteMonitor)/ParMultiDo(ByteMonitor) into ParDo(TimeMonitor)/ParMultiDo(TimeMonitor)
Mar 20, 2022 12:59:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-20T12:59:45.564Z: Fusing consumer Step: 0/ParMultiDo(CounterOperation) into ParDo(ByteMonitor)/ParMultiDo(ByteMonitor)
Mar 20, 2022 12:59:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-20T12:59:45.598Z: Fusing consumer ParDo(TimeMonitor)2/ParMultiDo(TimeMonitor) into Step: 0/ParMultiDo(CounterOperation)
Mar 20, 2022 12:59:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-20T12:59:45.700Z: Running job using Streaming Engine
Mar 20, 2022 12:59:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-20T12:59:45.919Z: Starting 5 ****s in us-central1-b...
Mar 20, 2022 1:00:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-20T13:00:16.623Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Mar 20, 2022 1:00:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-20T13:00:32.169Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Mar 20, 2022 1:01:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-20T13:01:29.777Z: Workers have started successfully.
Mar 20, 2022 1:01:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-20T13:01:29.810Z: Workers have started successfully.
Mar 20, 2022 1:04:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-20T13:04:52.174Z: Cleaning up.
Mar 20, 2022 1:04:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-20T13:04:52.253Z: Stopping **** pool...
Mar 20, 2022 1:04:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-20T13:04:52.303Z: Stopping **** pool...
Mar 20, 2022 1:07:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-20T13:07:13.975Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Mar 20, 2022 1:07:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-20T13:07:14.015Z: Worker pool stopped.
Mar 20, 2022 1:07:18 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2022-03-20_05_59_34-16397546146545212736 finished with status DONE.
Load test results for test (ID): 12dabf58-3326-404a-8fa2-bddf030a8330 and timestamp: 2022-03-20T12:59:28.981000000Z:
Metric: Value:
dataflow_v2_java17_runtime_sec 95.692
dataflow_v2_java17_total_bytes_count 4.7598959E9
> Task :runners:google-cloud-dataflow-java:cleanUpDockerJavaImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220320125858
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:10620c3e761c05cecf2b69718acbe4fec023259ea0cd8299bd137b2e23badcc8
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220320125858]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:10620c3e761c05cecf2b69718acbe4fec023259ea0cd8299bd137b2e23badcc8]
Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220320125858] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:10620c3e761c05cecf2b69718acbe4fec023259ea0cd8299bd137b2e23badcc8])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:10620c3e761c05cecf2b69718acbe4fec023259ea0cd8299bd137b2e23badcc8
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:10620c3e761c05cecf2b69718acbe4fec023259ea0cd8299bd137b2e23badcc8
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:10620c3e761c05cecf2b69718acbe4fec023259ea0cd8299bd137b2e23badcc8].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:394e6ea13a36e13654c68c657c9d6243beb766376883f3d983b5751470a2a5fc
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:394e6ea13a36e13654c68c657c9d6243beb766376883f3d983b5751470a2a5fc
ERROR: (gcloud.container.images.delete) Not found: response: {'docker-distribution-api-version': 'registry/2.0', 'content-type': 'application/json', 'date': 'Sun, 20 Mar 2022 13:07:27 GMT', 'server': 'Docker Registry', 'cache-control': 'private', 'x-xss-protection': '0', 'x-frame-options': 'SAMEORIGIN', 'transfer-encoding': 'chunked', 'status': '404', 'content-length': '168', '-content-encoding': 'gzip'}
Failed to compute blob liveness for manifest: 'sha256:394e6ea13a36e13654c68c657c9d6243beb766376883f3d983b5751470a2a5fc': None
> Task :runners:google-cloud-dataflow-java:cleanUpDockerJavaImages FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Java_ParDo_Dataflow_V2_Streaming_Java17/ws/src/runners/google-cloud-dataflow-java/build.gradle'> line: 297
* What went wrong:
Execution failed for task ':runners:google-cloud-dataflow-java:cleanUpDockerJavaImages'.
> Process 'command './scripts/cleanup_untagged_gcr_images.sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.
BUILD FAILED in 8m 29s
104 actionable tasks: 8 executed, 96 up-to-date
Publishing build scan...
https://gradle.com/s/y5le6zzwlsqa2
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Jenkins build is back to normal : beam_LoadTests_Java_ParDo_Dataflow_V2_Streaming_Java17 #93
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_ParDo_Dataflow_V2_Streaming_Java17/93/display/redirect?page=changes>
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Java_ParDo_Dataflow_V2_Streaming_Java17 #92
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_ParDo_Dataflow_V2_Streaming_Java17/92/display/redirect?page=changes>
Changes:
[Kyle Weaver] [BEAM-14124] Add display data to BQ storage reads.
[mmack] [adhoc] Move aws IT tests to testing package according to best practices
[noreply] fixes static checks and go lint issues (#17138)
[Kyle Weaver] Don't print in task configuration.
[noreply] [BEAM-14136] Clean up staticcheck and linter warnings in the Go SDK
[noreply] Merge pull request #17063 from [BEAM-12164] Fix flaky tests
[noreply] Revert "[BEAM-14112] Avoid storing a generator in _CustomBigQuerySource
[Kyle Weaver] [BEAM-4106] Remove filesToStage from Flink pipeline option list.
[noreply] [BEAM-14071] Enabling Flink on Dataproc for Interactive Beam (#17044)
[noreply] Minor: Bypass schema registry in schemas_test.py (#17108)
------------------------------------------
[...truncated 95.94 KB...]
result = self._call_chain(self.handle_open, protocol, protocol +
File "/usr/lib/python3.8/urllib/request.py", line 502, in _call_chain
result = func(*args)
File "/usr/lib/python3.8/urllib/request.py", line 1383, in http_open
return self.do_open(http.client.HTTPConnection, req)
File "/usr/lib/python3.8/urllib/request.py", line 1357, in do_open
raise URLError(err)
urllib.error.URLError: <urlopen error [Errno 101] Network is unreachable>
ERROR:root:Invalid url for checkstyle-8.23: http://www.gnu.org/licenses/old-licenses/lgpl-2.1.txt. Retrying...
Traceback (most recent call last):
File "/usr/lib/python3.8/urllib/request.py", line 1354, in do_open
h.request(req.get_method(), req.selector, req.data, headers,
File "/usr/lib/python3.8/http/client.py", line 1256, in request
self._send_request(method, url, body, headers, encode_chunked)
File "/usr/lib/python3.8/http/client.py", line 1302, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "/usr/lib/python3.8/http/client.py", line 1251, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/usr/lib/python3.8/http/client.py", line 1011, in _send_output
self.send(msg)
File "/usr/lib/python3.8/http/client.py", line 951, in send
self.connect()
File "/usr/lib/python3.8/http/client.py", line 922, in connect
self.sock = self._create_connection(
File "/usr/lib/python3.8/socket.py", line 808, in create_connection
raise err
File "/usr/lib/python3.8/socket.py", line 796, in create_connection
sock.connect(sa)
OSError: [Errno 101] Network is unreachable
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_LoadTests_Java_ParDo_Dataflow_V2_Streaming_Java17/ws/src/sdks/java/container/license_scripts/pull_licenses_java.py",> line 58, in pull_from_url
url_read = urlopen(url)
File "/usr/lib/python3.8/urllib/request.py", line 222, in urlopen
return opener.open(url, data, timeout)
File "/usr/lib/python3.8/urllib/request.py", line 525, in open
response = self._open(req, data)
File "/usr/lib/python3.8/urllib/request.py", line 542, in _open
result = self._call_chain(self.handle_open, protocol, protocol +
File "/usr/lib/python3.8/urllib/request.py", line 502, in _call_chain
result = func(*args)
File "/usr/lib/python3.8/urllib/request.py", line 1383, in http_open
return self.do_open(http.client.HTTPConnection, req)
File "/usr/lib/python3.8/urllib/request.py", line 1357, in do_open
raise URLError(err)
urllib.error.URLError: <urlopen error [Errno 101] Network is unreachable>
ERROR:root:Invalid url for checkstyle-8.23: http://www.gnu.org/licenses/old-licenses/lgpl-2.1.txt. Retrying...
Traceback (most recent call last):
File "/usr/lib/python3.8/urllib/request.py", line 1354, in do_open
h.request(req.get_method(), req.selector, req.data, headers,
File "/usr/lib/python3.8/http/client.py", line 1256, in request
self._send_request(method, url, body, headers, encode_chunked)
File "/usr/lib/python3.8/http/client.py", line 1302, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "/usr/lib/python3.8/http/client.py", line 1251, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/usr/lib/python3.8/http/client.py", line 1011, in _send_output
self.send(msg)
File "/usr/lib/python3.8/http/client.py", line 951, in send
self.connect()
File "/usr/lib/python3.8/http/client.py", line 922, in connect
self.sock = self._create_connection(
File "/usr/lib/python3.8/socket.py", line 808, in create_connection
raise err
File "/usr/lib/python3.8/socket.py", line 796, in create_connection
sock.connect(sa)
OSError: [Errno 101] Network is unreachable
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_LoadTests_Java_ParDo_Dataflow_V2_Streaming_Java17/ws/src/sdks/java/container/license_scripts/pull_licenses_java.py",> line 58, in pull_from_url
url_read = urlopen(url)
File "/usr/lib/python3.8/urllib/request.py", line 222, in urlopen
return opener.open(url, data, timeout)
File "/usr/lib/python3.8/urllib/request.py", line 525, in open
response = self._open(req, data)
File "/usr/lib/python3.8/urllib/request.py", line 542, in _open
result = self._call_chain(self.handle_open, protocol, protocol +
File "/usr/lib/python3.8/urllib/request.py", line 502, in _call_chain
result = func(*args)
File "/usr/lib/python3.8/urllib/request.py", line 1383, in http_open
return self.do_open(http.client.HTTPConnection, req)
File "/usr/lib/python3.8/urllib/request.py", line 1357, in do_open
raise URLError(err)
urllib.error.URLError: <urlopen error [Errno 101] Network is unreachable>
ERROR:root:Invalid url for checkstyle-8.23: http://www.gnu.org/licenses/old-licenses/lgpl-2.1.txt. Retrying...
Traceback (most recent call last):
File "/usr/lib/python3.8/urllib/request.py", line 1354, in do_open
h.request(req.get_method(), req.selector, req.data, headers,
File "/usr/lib/python3.8/http/client.py", line 1256, in request
self._send_request(method, url, body, headers, encode_chunked)
File "/usr/lib/python3.8/http/client.py", line 1302, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "/usr/lib/python3.8/http/client.py", line 1251, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/usr/lib/python3.8/http/client.py", line 1011, in _send_output
self.send(msg)
File "/usr/lib/python3.8/http/client.py", line 951, in send
self.connect()
File "/usr/lib/python3.8/http/client.py", line 922, in connect
self.sock = self._create_connection(
File "/usr/lib/python3.8/socket.py", line 808, in create_connection
raise err
File "/usr/lib/python3.8/socket.py", line 796, in create_connection
sock.connect(sa)
OSError: [Errno 101] Network is unreachable
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_LoadTests_Java_ParDo_Dataflow_V2_Streaming_Java17/ws/src/sdks/java/container/license_scripts/pull_licenses_java.py",> line 58, in pull_from_url
url_read = urlopen(url)
File "/usr/lib/python3.8/urllib/request.py", line 222, in urlopen
return opener.open(url, data, timeout)
File "/usr/lib/python3.8/urllib/request.py", line 525, in open
response = self._open(req, data)
File "/usr/lib/python3.8/urllib/request.py", line 542, in _open
result = self._call_chain(self.handle_open, protocol, protocol +
File "/usr/lib/python3.8/urllib/request.py", line 502, in _call_chain
result = func(*args)
File "/usr/lib/python3.8/urllib/request.py", line 1383, in http_open
return self.do_open(http.client.HTTPConnection, req)
File "/usr/lib/python3.8/urllib/request.py", line 1357, in do_open
raise URLError(err)
urllib.error.URLError: <urlopen error [Errno 101] Network is unreachable>
ERROR:root:Invalid url for checkstyle-8.23: http://www.gnu.org/licenses/old-licenses/lgpl-2.1.txt. Retrying...
Traceback (most recent call last):
File "/usr/lib/python3.8/urllib/request.py", line 1354, in do_open
h.request(req.get_method(), req.selector, req.data, headers,
File "/usr/lib/python3.8/http/client.py", line 1256, in request
self._send_request(method, url, body, headers, encode_chunked)
File "/usr/lib/python3.8/http/client.py", line 1302, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "/usr/lib/python3.8/http/client.py", line 1251, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/usr/lib/python3.8/http/client.py", line 1011, in _send_output
self.send(msg)
File "/usr/lib/python3.8/http/client.py", line 951, in send
self.connect()
File "/usr/lib/python3.8/http/client.py", line 922, in connect
self.sock = self._create_connection(
File "/usr/lib/python3.8/socket.py", line 808, in create_connection
raise err
File "/usr/lib/python3.8/socket.py", line 796, in create_connection
sock.connect(sa)
OSError: [Errno 101] Network is unreachable
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_LoadTests_Java_ParDo_Dataflow_V2_Streaming_Java17/ws/src/sdks/java/container/license_scripts/pull_licenses_java.py",> line 58, in pull_from_url
url_read = urlopen(url)
File "/usr/lib/python3.8/urllib/request.py", line 222, in urlopen
return opener.open(url, data, timeout)
File "/usr/lib/python3.8/urllib/request.py", line 525, in open
response = self._open(req, data)
File "/usr/lib/python3.8/urllib/request.py", line 542, in _open
result = self._call_chain(self.handle_open, protocol, protocol +
File "/usr/lib/python3.8/urllib/request.py", line 502, in _call_chain
result = func(*args)
File "/usr/lib/python3.8/urllib/request.py", line 1383, in http_open
return self.do_open(http.client.HTTPConnection, req)
File "/usr/lib/python3.8/urllib/request.py", line 1357, in do_open
raise URLError(err)
urllib.error.URLError: <urlopen error [Errno 101] Network is unreachable>
ERROR:root:Invalid url for checkstyle-8.23: http://www.gnu.org/licenses/old-licenses/lgpl-2.1.txt. Retrying...
Traceback (most recent call last):
File "/usr/lib/python3.8/urllib/request.py", line 1354, in do_open
h.request(req.get_method(), req.selector, req.data, headers,
File "/usr/lib/python3.8/http/client.py", line 1256, in request
self._send_request(method, url, body, headers, encode_chunked)
File "/usr/lib/python3.8/http/client.py", line 1302, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "/usr/lib/python3.8/http/client.py", line 1251, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/usr/lib/python3.8/http/client.py", line 1011, in _send_output
self.send(msg)
File "/usr/lib/python3.8/http/client.py", line 951, in send
self.connect()
File "/usr/lib/python3.8/http/client.py", line 922, in connect
self.sock = self._create_connection(
File "/usr/lib/python3.8/socket.py", line 808, in create_connection
raise err
File "/usr/lib/python3.8/socket.py", line 796, in create_connection
sock.connect(sa)
OSError: [Errno 101] Network is unreachable
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_LoadTests_Java_ParDo_Dataflow_V2_Streaming_Java17/ws/src/sdks/java/container/license_scripts/pull_licenses_java.py",> line 58, in pull_from_url
url_read = urlopen(url)
File "/usr/lib/python3.8/urllib/request.py", line 222, in urlopen
return opener.open(url, data, timeout)
File "/usr/lib/python3.8/urllib/request.py", line 525, in open
response = self._open(req, data)
File "/usr/lib/python3.8/urllib/request.py", line 542, in _open
result = self._call_chain(self.handle_open, protocol, protocol +
File "/usr/lib/python3.8/urllib/request.py", line 502, in _call_chain
result = func(*args)
File "/usr/lib/python3.8/urllib/request.py", line 1383, in http_open
return self.do_open(http.client.HTTPConnection, req)
File "/usr/lib/python3.8/urllib/request.py", line 1357, in do_open
raise URLError(err)
urllib.error.URLError: <urlopen error [Errno 101] Network is unreachable>
ERROR:root:Invalid url for checkstyle-8.23: http://www.gnu.org/licenses/old-licenses/lgpl-2.1.txt after 9 retries.
ERROR:root:['spotbugs-annotations-4.0.6', 'jFormatString-3.0.0', 'checkstyle-8.23']
ERROR:root:**************************************** Licenses were not able to be pulled automatically for some dependencies. Please search source code of the dependencies on the internet and add "license" and "notice" (if available) field to <https://ci-beam.apache.org/job/beam_LoadTests_Java_ParDo_Dataflow_V2_Streaming_Java17/ws/src/sdks/java/container/license_scripts/dep_urls_java.yaml> for each missing license. Dependency List: [checkstyle-8.23,jFormatString-3.0.0,spotbugs-annotations-4.0.6]
INFO:root:pull_licenses_java.py failed. It took 841.24793 seconds with 16 threads.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_LoadTests_Java_ParDo_Dataflow_V2_Streaming_Java17/ws/src/sdks/java/container/license_scripts/pull_licenses_java.py",> line 321, in <module>
raise RuntimeError('{n} error(s) occurred.'.format(n=len(error_msg)),
RuntimeError: ('1 error(s) occurred.', ['**************************************** Licenses were not able to be pulled automatically for some dependencies. Please search source code of the dependencies on the internet and add "license" and "notice" (if available) field to <https://ci-beam.apache.org/job/beam_LoadTests_Java_ParDo_Dataflow_V2_Streaming_Java17/ws/src/sdks/java/container/license_scripts/dep_urls_java.yaml> for each missing license. Dependency List: [checkstyle-8.23,jFormatString-3.0.0,spotbugs-annotations-4.0.6]'])
> Task :sdks:java:container:pullLicenses FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:container:pullLicenses'.
> Process 'command './license_scripts/license_script.sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.
BUILD FAILED in 14m 41s
103 actionable tasks: 67 executed, 32 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/6j2zqyncxfe6c
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Java_ParDo_Dataflow_V2_Streaming_Java17 #91
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_ParDo_Dataflow_V2_Streaming_Java17/91/display/redirect>
Changes:
------------------------------------------
[...truncated 227.96 KB...]
> Task :sdks:java:harness:classes UP-TO-DATE
> Task :sdks:java:harness:jar UP-TO-DATE
> Task :runners:java-fn-execution:compileJava UP-TO-DATE
> Task :runners:java-fn-execution:classes UP-TO-DATE
> Task :runners:java-fn-execution:jar UP-TO-DATE
> Task :sdks:java:harness:shadowJar UP-TO-DATE
> Task :sdks:java:expansion-service:compileJava UP-TO-DATE
> Task :sdks:java:expansion-service:classes UP-TO-DATE
> Task :sdks:java:expansion-service:jar UP-TO-DATE
> Task :sdks:java:io:kafka:compileJava UP-TO-DATE
> Task :sdks:java:io:kafka:classes UP-TO-DATE
> Task :sdks:java:io:kafka:jar UP-TO-DATE
> Task :sdks:java:container:java17:copyDockerfileDependencies UP-TO-DATE
> Task :sdks:java:io:google-cloud-platform:compileJava UP-TO-DATE
> Task :sdks:java:io:google-cloud-platform:classes UP-TO-DATE
> Task :sdks:java:io:google-cloud-platform:jar UP-TO-DATE
> Task :sdks:java:testing:load-tests:compileJava UP-TO-DATE
> Task :sdks:java:testing:load-tests:classes UP-TO-DATE
> Task :sdks:java:testing:load-tests:jar UP-TO-DATE
> Task :runners:google-cloud-dataflow-java:compileJava UP-TO-DATE
> Task :runners:google-cloud-dataflow-java:classes UP-TO-DATE
> Task :runners:google-cloud-dataflow-java:jar UP-TO-DATE
> Task :sdks:java:container:goBuild
/home/jenkins/go/bin/go1.16.12 build -o ./build/target/linux_amd64/boot boot.go
> Task :sdks:java:container:java17:copySdkHarnessLauncher
Execution optimizations have been disabled for task ':sdks:java:container:java17:copySdkHarnessLauncher' to ensure correctness due to the following reasons:
- Gradle detected a problem with the following location: '<https://ci-beam.apache.org/job/beam_LoadTests_Java_ParDo_Dataflow_V2_Streaming_Java17/ws/src/sdks/java/container/build/target'.> Reason: Task ':sdks:java:container:java17:copySdkHarnessLauncher' uses this output of task ':sdks:java:container:downloadCloudProfilerAgent' without declaring an explicit or implicit dependency. This can lead to incorrect results being produced, depending on what order the tasks are executed. Please refer to https://docs.gradle.org/7.3.2/userguide/validation_problems.html#implicit_dependency for more details about this problem.
- Gradle detected a problem with the following location: '<https://ci-beam.apache.org/job/beam_LoadTests_Java_ParDo_Dataflow_V2_Streaming_Java17/ws/src/sdks/java/container/build/target'.> Reason: Task ':sdks:java:container:java17:copySdkHarnessLauncher' uses this output of task ':sdks:java:container:pullLicenses' without declaring an explicit or implicit dependency. This can lead to incorrect results being produced, depending on what order the tasks are executed. Please refer to https://docs.gradle.org/7.3.2/userguide/validation_problems.html#implicit_dependency for more details about this problem.
> Task :sdks:java:container:java17:dockerPrepare
> Task :sdks:java:container:java17:docker
> Task :runners:google-cloud-dataflow-java:buildAndPushDockerJavaContainer
WARNING: `gcloud docker` will not be supported for Docker client versions above 18.03.
As an alternative, use `gcloud auth configure-docker` to configure `docker` to
use `gcloud` as a credential helper, then use `docker` as you would for non-GCR
registries, e.g. `docker pull gcr.io/project-id/my-image`. Add
`--verbosity=error` to silence this warning: `gcloud docker
--verbosity=error -- pull gcr.io/project-id/my-image`.
See: https://cloud.google.com/container-registry/docs/support/deprecation-notices#gcloud-docker
The push refers to repository [us.gcr.io/apache-beam-testing/java-postcommit-it/java]
4500764986f6: Preparing
46ac930f1c8f: Preparing
6e9557895629: Preparing
b9fb74ece4d8: Preparing
c64631c863e5: Preparing
da7a25550266: Preparing
1e77e7a0c09a: Preparing
d041b0b534c3: Preparing
58e20008cc33: Preparing
e09d9bc87aee: Preparing
8bfaa3dc8059: Preparing
92eba12d2d5c: Preparing
f7db564e114d: Preparing
de337572acf9: Preparing
383ff12ef634: Preparing
327e42081bbe: Preparing
6e632f416458: Preparing
e019be289189: Preparing
c9a63110150b: Preparing
d041b0b534c3: Waiting
58e20008cc33: Waiting
92eba12d2d5c: Waiting
f7db564e114d: Waiting
327e42081bbe: Waiting
da7a25550266: Waiting
e09d9bc87aee: Waiting
383ff12ef634: Waiting
de337572acf9: Waiting
1e77e7a0c09a: Waiting
c9a63110150b: Waiting
6e632f416458: Waiting
6e9557895629: Pushed
46ac930f1c8f: Pushed
c64631c863e5: Pushed
4500764986f6: Pushed
b9fb74ece4d8: Pushed
da7a25550266: Pushed
d041b0b534c3: Pushed
58e20008cc33: Pushed
e09d9bc87aee: Pushed
de337572acf9: Layer already exists
383ff12ef634: Layer already exists
327e42081bbe: Layer already exists
6e632f416458: Layer already exists
1e77e7a0c09a: Pushed
e019be289189: Layer already exists
c9a63110150b: Layer already exists
92eba12d2d5c: Pushed
f7db564e114d: Pushed
8bfaa3dc8059: Pushed
20220321124822: digest: sha256:10351619e15f98aa87a41b3079536e6540cb152e1c6faf209ec1c0561e60a0ce size: 4314
> Task :sdks:java:testing:load-tests:run
Mar 21, 2022 12:48:54 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Mar 21, 2022 12:48:54 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 207 files. Enable logging at DEBUG level to see which files will be staged.
Mar 21, 2022 12:48:55 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: ParDo(TimeMonitor)
Mar 21, 2022 12:48:55 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Mar 21, 2022 12:48:57 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 207 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Mar 21, 2022 12:48:58 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 207 files cached, 0 files newly uploaded in 0 seconds
Mar 21, 2022 12:48:58 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Mar 21, 2022 12:48:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <90966 bytes, hash 80cbf95dd6ff9588c6bf14fda5f454f5560ddeee7d1936d4e3745c1c2d4feb48> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-gMv5Xdb_lYjGvxT9pfRU9VYN3u59GTbU43RcHC1P60g.pb
Mar 21, 2022 12:48:59 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1
Mar 21, 2022 12:48:59 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6dfa915a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@78054f54, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@cb7fa71, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3dffc764, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4b6e1c0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@561b61ed, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@654c7d2d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26cb5207, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@15400fff, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@18d910b3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1e7ab390, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@625d9132, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77774571, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@277b8fa4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6cd64ee8, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@620c8641, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2f1d0bbc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5460b754, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@a9f023e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@c27a3a2]
Mar 21, 2022 12:48:59 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Mar 21, 2022 12:48:59 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s3
Mar 21, 2022 12:48:59 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(ByteMonitor) as step s4
Mar 21, 2022 12:48:59 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Step: 0 as step s5
Mar 21, 2022 12:48:59 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor)2 as step s6
Mar 21, 2022 12:48:59 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.38.0-SNAPSHOT
Mar 21, 2022 12:49:00 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-03-21_05_49_00-6896949720821721160?project=apache-beam-testing
Mar 21, 2022 12:49:00 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2022-03-21_05_49_00-6896949720821721160
Mar 21, 2022 12:49:00 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2022-03-21_05_49_00-6896949720821721160
Mar 21, 2022 12:49:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2022-03-21T12:49:04.607Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java170dataflow0v20streaming0pardo03-jenkins-03-25p4. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Mar 21, 2022 12:49:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-21T12:49:09.709Z: Worker configuration: e2-standard-2 in us-central1-b.
Mar 21, 2022 12:49:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-21T12:49:10.350Z: Expanding SplittableParDo operations into optimizable parts.
Mar 21, 2022 12:49:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-21T12:49:10.382Z: Expanding CollectionToSingleton operations into optimizable parts.
Mar 21, 2022 12:49:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-21T12:49:10.449Z: Expanding CoGroupByKey operations into optimizable parts.
Mar 21, 2022 12:49:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-21T12:49:10.485Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Mar 21, 2022 12:49:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-21T12:49:10.514Z: Expanding GroupByKey operations into streaming Read/Write steps
Mar 21, 2022 12:49:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-21T12:49:10.549Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Mar 21, 2022 12:49:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-21T12:49:10.613Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Mar 21, 2022 12:49:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-21T12:49:10.644Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Mar 21, 2022 12:49:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-21T12:49:10.669Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Mar 21, 2022 12:49:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-21T12:49:10.696Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Mar 21, 2022 12:49:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-21T12:49:10.722Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Mar 21, 2022 12:49:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-21T12:49:10.756Z: Fusing consumer ParDo(TimeMonitor)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds)
Mar 21, 2022 12:49:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-21T12:49:10.780Z: Fusing consumer ParDo(ByteMonitor)/ParMultiDo(ByteMonitor) into ParDo(TimeMonitor)/ParMultiDo(TimeMonitor)
Mar 21, 2022 12:49:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-21T12:49:10.805Z: Fusing consumer Step: 0/ParMultiDo(CounterOperation) into ParDo(ByteMonitor)/ParMultiDo(ByteMonitor)
Mar 21, 2022 12:49:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-21T12:49:10.840Z: Fusing consumer ParDo(TimeMonitor)2/ParMultiDo(TimeMonitor) into Step: 0/ParMultiDo(CounterOperation)
Mar 21, 2022 12:49:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-21T12:49:10.935Z: Running job using Streaming Engine
Mar 21, 2022 12:49:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-21T12:49:11.111Z: Starting 5 ****s in us-central1-b...
Mar 21, 2022 12:49:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-21T12:49:30.852Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Mar 21, 2022 12:49:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-21T12:49:57.064Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Mar 21, 2022 12:50:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-21T12:50:54.213Z: Workers have started successfully.
Mar 21, 2022 12:50:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-21T12:50:54.235Z: Workers have started successfully.
Mar 21, 2022 12:53:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-21T12:53:15.865Z: Cleaning up.
Mar 21, 2022 12:53:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-21T12:53:15.981Z: Stopping **** pool...
Mar 21, 2022 12:53:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-21T12:53:16.035Z: Stopping **** pool...
Mar 21, 2022 12:55:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-21T12:55:35.273Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
Mar 21, 2022 12:55:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-21T12:55:35.318Z: Worker pool stopped.
Mar 21, 2022 12:55:41 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2022-03-21_05_49_00-6896949720821721160 finished with status DONE.
Load test results for test (ID): 6d909a53-dfb2-4526-8f14-ed7b88d59a01 and timestamp: 2022-03-21T12:48:55.084000000Z:
Metric: Value:
dataflow_v2_java17_runtime_sec 57.891
dataflow_v2_java17_total_bytes_count 4.2306662E9
> Task :runners:google-cloud-dataflow-java:cleanUpDockerJavaImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220321124822
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:10351619e15f98aa87a41b3079536e6540cb152e1c6faf209ec1c0561e60a0ce
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220321124822]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:10351619e15f98aa87a41b3079536e6540cb152e1c6faf209ec1c0561e60a0ce]
Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220321124822] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:10351619e15f98aa87a41b3079536e6540cb152e1c6faf209ec1c0561e60a0ce])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:2f40a57c518bb3b823fd06aa508ac5c5338efc84c860529bc267ef594bd40d66
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:2f40a57c518bb3b823fd06aa508ac5c5338efc84c860529bc267ef594bd40d66
ERROR: (gcloud.container.images.delete) Not found: response: {'docker-distribution-api-version': 'registry/2.0', 'content-type': 'application/json', 'date': 'Mon, 21 Mar 2022 12:55:48 GMT', 'server': 'Docker Registry', 'cache-control': 'private', 'x-xss-protection': '0', 'x-frame-options': 'SAMEORIGIN', 'transfer-encoding': 'chunked', 'status': '404', 'content-length': '168', '-content-encoding': 'gzip'}
Failed to compute blob liveness for manifest: 'sha256:2f40a57c518bb3b823fd06aa508ac5c5338efc84c860529bc267ef594bd40d66': None
> Task :runners:google-cloud-dataflow-java:cleanUpDockerJavaImages FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Java_ParDo_Dataflow_V2_Streaming_Java17/ws/src/runners/google-cloud-dataflow-java/build.gradle'> line: 297
* What went wrong:
Execution failed for task ':runners:google-cloud-dataflow-java:cleanUpDockerJavaImages'.
> Process 'command './scripts/cleanup_untagged_gcr_images.sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.
BUILD FAILED in 7m 27s
104 actionable tasks: 8 executed, 96 up-to-date
Publishing build scan...
https://gradle.com/s/ndbymbnu2hhws
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org