You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@beam.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2021/09/12 12:44:03 UTC
Build failed in Jenkins:
beam_LoadTests_Java_ParDo_Dataflow_V2_Batch_Java11 #87
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_ParDo_Dataflow_V2_Batch_Java11/87/display/redirect>
Changes:
------------------------------------------
[...truncated 39.38 KB...]
> Task :sdks:java:io:kafka:jar
> Task :sdks:java:io:google-cloud-platform:compileJava FROM-CACHE
> Task :sdks:java:io:google-cloud-platform:classes UP-TO-DATE
> Task :sdks:java:io:google-cloud-platform:jar
> Task :sdks:java:container:java11:copyDockerfileDependencies
> Task :sdks:java:testing:load-tests:compileJava FROM-CACHE
> Task :sdks:java:testing:load-tests:classes UP-TO-DATE
> Task :sdks:java:testing:load-tests:jar
> Task :runners:google-cloud-dataflow-java:compileJava FROM-CACHE
> Task :runners:google-cloud-dataflow-java:classes
> Task :runners:google-cloud-dataflow-java:jar
> Task :sdks:java:container:generateLicenseReport
> Task :sdks:java:container:pullLicenses
Copying already-fetched licenses from <https://ci-beam.apache.org/job/beam_LoadTests_Java_ParDo_Dataflow_V2_Batch_Java11/ws/src/sdks/java/container/build/reports/dependency-license> to <https://ci-beam.apache.org/job/beam_LoadTests_Java_ParDo_Dataflow_V2_Batch_Java11/ws/src/sdks/java/container/build/target/java_third_party_licenses>
Already using interpreter /usr/bin/python3
Using base prefix '/usr'
New python executable in <https://ci-beam.apache.org/job/beam_LoadTests_Java_ParDo_Dataflow_V2_Batch_Java11/ws/src/sdks/java/container/build/virtualenv/bin/python3>
Also creating executable in <https://ci-beam.apache.org/job/beam_LoadTests_Java_ParDo_Dataflow_V2_Batch_Java11/ws/src/sdks/java/container/build/virtualenv/bin/python>
Installing setuptools, pip, wheel...
done.
DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality.
Collecting beautifulsoup4<5.0,>=4.9.0
Using cached beautifulsoup4-4.10.0-py3-none-any.whl (97 kB)
Collecting future<1.0.0,>=0.16.0
Using cached future-0.18.2-py3-none-any.whl
Collecting pyyaml<6.0.0,>=3.12
Using cached PyYAML-5.3.1-cp35-cp35m-linux_x86_64.whl
Collecting tenacity<6.0,>=5.0.2
Using cached tenacity-5.1.5-py2.py3-none-any.whl (34 kB)
Collecting soupsieve>1.2
Using cached soupsieve-2.1-py3-none-any.whl (32 kB)
Collecting six>=1.9.0
Using cached six-1.16.0-py2.py3-none-any.whl (11 kB)
Installing collected packages: soupsieve, six, tenacity, pyyaml, future, beautifulsoup4
Successfully installed beautifulsoup4-4.10.0 future-0.18.2 pyyaml-5.3.1 six-1.16.0 soupsieve-2.1 tenacity-5.1.5
Executing <https://ci-beam.apache.org/job/beam_LoadTests_Java_ParDo_Dataflow_V2_Batch_Java11/ws/src/sdks/java/container/build/virtualenv/bin/python> <https://ci-beam.apache.org/job/beam_LoadTests_Java_ParDo_Dataflow_V2_Batch_Java11/ws/src/sdks/java/container/license_scripts/pull_licenses_java.py> --license_index=<https://ci-beam.apache.org/job/beam_LoadTests_Java_ParDo_Dataflow_V2_Batch_Java11/ws/src/sdks/java/container/build/reports/dependency-license/index.json> --output_dir=<https://ci-beam.apache.org/job/beam_LoadTests_Java_ParDo_Dataflow_V2_Batch_Java11/ws/src/sdks/java/container/build/target/java_third_party_licenses> --dep_url_yaml=<https://ci-beam.apache.org/job/beam_LoadTests_Java_ParDo_Dataflow_V2_Batch_Java11/ws/src/sdks/java/container/license_scripts/dep_urls_java.yaml>
INFO:root:Pulling license for 207 dependencies using 16 threads.
INFO:root:pull_licenses_java.py succeed. It took 1.905533 seconds with 16 threads.
Copying licenses from <https://ci-beam.apache.org/job/beam_LoadTests_Java_ParDo_Dataflow_V2_Batch_Java11/ws/src/sdks/java/container/build/target/java_third_party_licenses> to <https://ci-beam.apache.org/job/beam_LoadTests_Java_ParDo_Dataflow_V2_Batch_Java11/ws/src/sdks/java/container/build/target/third_party_licenses.>
Finished license_scripts.sh
> Task :sdks:java:container:java11:copyJavaThirdPartyLicenses
> Task :release:go-licenses:java:dockerRun
+ go-licenses save github.com/apache/beam/sdks/java/container --save_path=/output/licenses
+ go-licenses csv github.com/apache/beam/sdks/java/container
+ tee /output/licenses/list.csv
google.golang.org/protobuf,https://go.googlesource.com/protobuf/+/refs/heads/master/LICENSE,BSD-3-Clause
google.golang.org/genproto/googleapis/rpc/status,https://github.com/googleapis/go-genproto/blob/master/LICENSE,Apache-2.0
golang.org/x/sys,https://go.googlesource.com/sys/+/refs/heads/master/LICENSE,BSD-3-Clause
golang.org/x/text,https://go.googlesource.com/text/+/refs/heads/master/LICENSE,BSD-3-Clause
google.golang.org/grpc,https://github.com/grpc/grpc-go/blob/master/LICENSE,Apache-2.0
golang.org/x/net,https://go.googlesource.com/net/+/refs/heads/master/LICENSE,BSD-3-Clause
github.com/golang/protobuf,https://github.com/golang/protobuf/blob/master/LICENSE,BSD-3-Clause
github.com/apache/beam/sdks/java/container,https://github.com/apache/beam/blob/master/LICENSE,Apache-2.0
github.com/apache/beam/sdks/go/pkg/beam,https://github.com/apache/beam/blob/master/sdks/go/README.md,Apache-2.0
+ chmod -R a+w /output/licenses
> Task :release:go-licenses:java:createLicenses
> Task :sdks:java:container:java11:copyGolangLicenses
> Task :sdks:java:container:java11:dockerPrepare
> Task :sdks:java:container:java11:docker
> Task :runners:google-cloud-dataflow-java:buildAndPushDockerContainer
WARNING: `gcloud docker` will not be supported for Docker client versions above 18.03.
As an alternative, use `gcloud auth configure-docker` to configure `docker` to
use `gcloud` as a credential helper, then use `docker` as you would for non-GCR
registries, e.g. `docker pull gcr.io/project-id/my-image`. Add
`--verbosity=error` to silence this warning: `gcloud docker
--verbosity=error -- pull gcr.io/project-id/my-image`.
See: https://cloud.google.com/container-registry/docs/support/deprecation-notices#gcloud-docker
The push refers to repository [us.gcr.io/apache-beam-testing/java-postcommit-it/java]
28c7366abdcc: Preparing
303ab604f3a3: Preparing
1b66c9189d58: Preparing
f41f33606117: Preparing
88c79fb12062: Preparing
8bb04b46f468: Preparing
5024d0e28aa6: Preparing
da141e0c849c: Preparing
9b2f798b25e7: Preparing
87a33da5d002: Preparing
2b787f9564c3: Preparing
eb48dee73fce: Preparing
3891808a925b: Preparing
d402f4f1b906: Preparing
00ef5416d927: Preparing
8555e663f65b: Preparing
d00da3cd7763: Preparing
4e61e63529c2: Preparing
799760671c38: Preparing
8bb04b46f468: Waiting
8555e663f65b: Waiting
eb48dee73fce: Waiting
5024d0e28aa6: Waiting
d00da3cd7763: Waiting
da141e0c849c: Waiting
3891808a925b: Waiting
00ef5416d927: Waiting
d402f4f1b906: Waiting
799760671c38: Waiting
9b2f798b25e7: Waiting
2b787f9564c3: Waiting
4e61e63529c2: Waiting
87a33da5d002: Waiting
303ab604f3a3: Pushed
88c79fb12062: Pushed
1b66c9189d58: Pushed
28c7366abdcc: Pushed
8bb04b46f468: Pushed
f41f33606117: Pushed
da141e0c849c: Pushed
9b2f798b25e7: Pushed
3891808a925b: Layer already exists
d402f4f1b906: Layer already exists
00ef5416d927: Layer already exists
8555e663f65b: Layer already exists
eb48dee73fce: Pushed
d00da3cd7763: Layer already exists
2b787f9564c3: Pushed
4e61e63529c2: Layer already exists
799760671c38: Layer already exists
5024d0e28aa6: Pushed
87a33da5d002: Pushed
20210912124136: digest: sha256:e5dee5e27a1fc9969388ca35d24342b4b92c0dc70f1a0c6e825af55fc6a1636a size: 4311
> Task :sdks:java:testing:load-tests:run
Sep 12, 2021 12:43:34 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Sep 12, 2021 12:43:35 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 195 files. Enable logging at DEBUG level to see which files will be staged.
Sep 12, 2021 12:43:35 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: ParDo(TimeMonitor)
Sep 12, 2021 12:43:35 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Sep 12, 2021 12:43:38 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 195 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Sep 12, 2021 12:43:38 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 195 files cached, 0 files newly uploaded in 0 seconds
Sep 12, 2021 12:43:38 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Sep 12, 2021 12:43:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <98992 bytes, hash 0ef43be1b20ce74b69f221c0dc69c971b7e32ea65cd8e4279ac4839f39365f2c> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-DvQ74bIM50tp8iHA3GnJcbfjLqZc2OQnmsSDnzk2Xyw.pb
Sep 12, 2021 12:43:40 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input as step s1
Sep 12, 2021 12:43:40 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s2
Sep 12, 2021 12:43:40 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(ByteMonitor) as step s3
Sep 12, 2021 12:43:40 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Step: 0 as step s4
Sep 12, 2021 12:43:40 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Step: 1 as step s5
Sep 12, 2021 12:43:40 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Step: 2 as step s6
Sep 12, 2021 12:43:40 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Step: 3 as step s7
Sep 12, 2021 12:43:40 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Step: 4 as step s8
Sep 12, 2021 12:43:40 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Step: 5 as step s9
Sep 12, 2021 12:43:40 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Step: 6 as step s10
Sep 12, 2021 12:43:40 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Step: 7 as step s11
Sep 12, 2021 12:43:40 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Step: 8 as step s12
Sep 12, 2021 12:43:40 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Step: 9 as step s13
Sep 12, 2021 12:43:40 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor)2 as step s14
Sep 12, 2021 12:43:40 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.34.0-SNAPSHOT
Sep 12, 2021 12:43:41 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-09-12_05_43_40-5859544672243416199?project=apache-beam-testing
Sep 12, 2021 12:43:41 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-09-12_05_43_40-5859544672243416199
Sep 12, 2021 12:43:41 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-09-12_05_43_40-5859544672243416199
Sep 12, 2021 12:43:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-09-12T12:43:45.421Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20batch0pardo01-jenkins-091212-696h. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Sep 12, 2021 12:43:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-12T12:43:48.955Z: Worker configuration: e2-standard-2 in us-central1-a.
Sep 12, 2021 12:43:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-12T12:43:49.717Z: Workflow failed. Causes: Project apache-beam-testing has insufficient quota(s) to execute this workflow with 1 instances in region us-central1. Quota summary (required/available): 1/24379 instances, 2/0 CPUs, 25/186621 disk GB, 0/2397 SSD disk GB, 1/278 instance groups, 1/281 managed instance groups, 1/507 instance templates, 1/608 in-use IP addresses.
Please see https://cloud.google.com/compute/docs/resource-quotas about requesting more quota.
Sep 12, 2021 12:43:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-12T12:43:49.753Z: Cleaning up.
Sep 12, 2021 12:43:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-12T12:43:49.834Z: Worker pool stopped.
Sep 12, 2021 12:43:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-12T12:43:51.015Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Sep 12, 2021 12:43:56 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-09-12_05_43_40-5859544672243416199 failed with status FAILED.
Sep 12, 2021 12:43:56 PM org.apache.beam.sdk.testutils.metrics.MetricsReader getCounterMetric
SEVERE: Failed to get metric totalBytes.count, from namespace pardo
Load test results for test (ID): 31345bf9-1d96-47ce-aff4-f6cf08fa3358 and timestamp: 2021-09-12T12:43:35.491000000Z:
Metric: Value:
dataflow_v2_java11_runtime_sec 0.0
dataflow_v2_java11_total_bytes_count -1.0
Exception in thread "main" java.lang.RuntimeException: Invalid job state: FAILED.
at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
at org.apache.beam.sdk.loadtests.ParDoLoadTest.run(ParDoLoadTest.java:53)
at org.apache.beam.sdk.loadtests.ParDoLoadTest.main(ParDoLoadTest.java:103)
> Task :sdks:java:testing:load-tests:run FAILED
> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210912124136
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:e5dee5e27a1fc9969388ca35d24342b4b92c0dc70f1a0c6e825af55fc6a1636a
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210912124136]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:e5dee5e27a1fc9969388ca35d24342b4b92c0dc70f1a0c6e825af55fc6a1636a]
Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210912124136] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:e5dee5e27a1fc9969388ca35d24342b4b92c0dc70f1a0c6e825af55fc6a1636a])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:e5dee5e27a1fc9969388ca35d24342b4b92c0dc70f1a0c6e825af55fc6a1636a
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:e5dee5e27a1fc9969388ca35d24342b4b92c0dc70f1a0c6e825af55fc6a1636a
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:e5dee5e27a1fc9969388ca35d24342b4b92c0dc70f1a0c6e825af55fc6a1636a].
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 2m 39s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date
Publishing build scan...
https://gradle.com/s/l27hxene4mwpm
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Jenkins build is back to normal :
beam_LoadTests_Java_ParDo_Dataflow_V2_Batch_Java11 #89
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_ParDo_Dataflow_V2_Batch_Java11/89/display/redirect?page=changes>
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_LoadTests_Java_ParDo_Dataflow_V2_Batch_Java11 #88
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_ParDo_Dataflow_V2_Batch_Java11/88/display/redirect>
Changes:
------------------------------------------
[...truncated 48.13 KB...]
1405e3c8f682: Preparing
9a81091d2fe7: Preparing
63d0e788e3fd: Preparing
afb5e70b8646: Preparing
3891808a925b: Preparing
d402f4f1b906: Preparing
00ef5416d927: Preparing
8555e663f65b: Preparing
d00da3cd7763: Preparing
4e61e63529c2: Preparing
799760671c38: Preparing
63d0e788e3fd: Waiting
8555e663f65b: Waiting
799760671c38: Waiting
3891808a925b: Waiting
4e61e63529c2: Waiting
afb5e70b8646: Waiting
d00da3cd7763: Waiting
00ef5416d927: Waiting
1179ec4a4c1f: Waiting
9a81091d2fe7: Waiting
1d20c80bbc4b: Waiting
1405e3c8f682: Waiting
b5740fd0c1f4: Waiting
d402f4f1b906: Waiting
1301e1a12f23: Pushed
bce378a3168e: Pushed
f8bbcccfeefa: Pushed
0bc338689b44: Pushed
1179ec4a4c1f: Pushed
91a941c06272: Pushed
1d20c80bbc4b: Pushed
1405e3c8f682: Pushed
3891808a925b: Layer already exists
d402f4f1b906: Layer already exists
00ef5416d927: Layer already exists
63d0e788e3fd: Pushed
b5740fd0c1f4: Pushed
8555e663f65b: Layer already exists
d00da3cd7763: Layer already exists
4e61e63529c2: Layer already exists
799760671c38: Layer already exists
afb5e70b8646: Pushed
9a81091d2fe7: Pushed
20210913124134: digest: sha256:0a5dabfc545165599d583278dce76db9ff0d7efef7c7be23a36ba60cdcc033fa size: 4311
> Task :sdks:java:testing:load-tests:run
Sep 13, 2021 12:43:26 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Sep 13, 2021 12:43:27 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 195 files. Enable logging at DEBUG level to see which files will be staged.
Sep 13, 2021 12:43:27 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: ParDo(TimeMonitor)
Sep 13, 2021 12:43:27 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Sep 13, 2021 12:43:30 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 195 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Sep 13, 2021 12:43:31 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 195 files cached, 0 files newly uploaded in 0 seconds
Sep 13, 2021 12:43:31 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Sep 13, 2021 12:43:31 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <98992 bytes, hash 0bc4b159cf05f3d51e2a72bd46fccbc97a28ae73538a0e3dd399aa42133d7645> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-C8SxWc8F89UeKnK9RvzLyXoornNTig4905mqQhM9dkU.pb
Sep 13, 2021 12:43:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input as step s1
Sep 13, 2021 12:43:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s2
Sep 13, 2021 12:43:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(ByteMonitor) as step s3
Sep 13, 2021 12:43:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Step: 0 as step s4
Sep 13, 2021 12:43:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Step: 1 as step s5
Sep 13, 2021 12:43:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Step: 2 as step s6
Sep 13, 2021 12:43:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Step: 3 as step s7
Sep 13, 2021 12:43:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Step: 4 as step s8
Sep 13, 2021 12:43:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Step: 5 as step s9
Sep 13, 2021 12:43:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Step: 6 as step s10
Sep 13, 2021 12:43:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Step: 7 as step s11
Sep 13, 2021 12:43:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Step: 8 as step s12
Sep 13, 2021 12:43:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Step: 9 as step s13
Sep 13, 2021 12:43:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor)2 as step s14
Sep 13, 2021 12:43:33 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.34.0-SNAPSHOT
Sep 13, 2021 12:43:33 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-09-13_05_43_33-77183844169192276?project=apache-beam-testing
Sep 13, 2021 12:43:33 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-09-13_05_43_33-77183844169192276
Sep 13, 2021 12:43:33 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-09-13_05_43_33-77183844169192276
Sep 13, 2021 12:43:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-09-13T12:43:37.785Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20batch0pardo01-jenkins-091312-khfv. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Sep 13, 2021 12:43:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-13T12:43:41.193Z: Worker configuration: e2-standard-2 in us-central1-a.
Sep 13, 2021 12:43:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-13T12:43:42.218Z: Expanding SplittableParDo operations into optimizable parts.
Sep 13, 2021 12:43:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-13T12:43:42.254Z: Expanding CollectionToSingleton operations into optimizable parts.
Sep 13, 2021 12:43:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-13T12:43:42.324Z: Expanding CoGroupByKey operations into optimizable parts.
Sep 13, 2021 12:43:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-13T12:43:42.358Z: Expanding GroupByKey operations into optimizable parts.
Sep 13, 2021 12:43:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-13T12:43:42.440Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Sep 13, 2021 12:43:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-13T12:43:42.463Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse
Sep 13, 2021 12:43:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-13T12:43:42.486Z: Fusing consumer Read-input-ParDo-BoundedSourceAsSDFWrapper--ParMultiDo-BoundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Sep 13, 2021 12:43:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-13T12:43:42.512Z: Fusing consumer Read-input-ParDo-BoundedSourceAsSDFWrapper--ParMultiDo-BoundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-BoundedSourceAsSDFWrapper--ParMultiDo-BoundedSourceAsSDFWrapper-/PairWithRestriction
Sep 13, 2021 12:43:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-13T12:43:42.546Z: Fusing consumer ParDo(TimeMonitor)/ParMultiDo(TimeMonitor) into Read-input-ParDo-BoundedSourceAsSDFWrapper--ParMultiDo-BoundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Sep 13, 2021 12:43:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-13T12:43:42.579Z: Fusing consumer ParDo(ByteMonitor)/ParMultiDo(ByteMonitor) into ParDo(TimeMonitor)/ParMultiDo(TimeMonitor)
Sep 13, 2021 12:43:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-13T12:43:42.613Z: Fusing consumer Step: 0/ParMultiDo(CounterOperation) into ParDo(ByteMonitor)/ParMultiDo(ByteMonitor)
Sep 13, 2021 12:43:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-13T12:43:42.648Z: Fusing consumer Step: 1/ParMultiDo(CounterOperation) into Step: 0/ParMultiDo(CounterOperation)
Sep 13, 2021 12:43:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-13T12:43:42.680Z: Fusing consumer Step: 2/ParMultiDo(CounterOperation) into Step: 1/ParMultiDo(CounterOperation)
Sep 13, 2021 12:43:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-13T12:43:42.710Z: Fusing consumer Step: 3/ParMultiDo(CounterOperation) into Step: 2/ParMultiDo(CounterOperation)
Sep 13, 2021 12:43:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-13T12:43:42.735Z: Fusing consumer Step: 4/ParMultiDo(CounterOperation) into Step: 3/ParMultiDo(CounterOperation)
Sep 13, 2021 12:43:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-13T12:43:42.770Z: Fusing consumer Step: 5/ParMultiDo(CounterOperation) into Step: 4/ParMultiDo(CounterOperation)
Sep 13, 2021 12:43:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-13T12:43:42.804Z: Fusing consumer Step: 6/ParMultiDo(CounterOperation) into Step: 5/ParMultiDo(CounterOperation)
Sep 13, 2021 12:43:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-13T12:43:42.837Z: Fusing consumer Step: 7/ParMultiDo(CounterOperation) into Step: 6/ParMultiDo(CounterOperation)
Sep 13, 2021 12:43:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-13T12:43:42.862Z: Fusing consumer Step: 8/ParMultiDo(CounterOperation) into Step: 7/ParMultiDo(CounterOperation)
Sep 13, 2021 12:43:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-13T12:43:42.897Z: Fusing consumer Step: 9/ParMultiDo(CounterOperation) into Step: 8/ParMultiDo(CounterOperation)
Sep 13, 2021 12:43:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-13T12:43:42.931Z: Fusing consumer ParDo(TimeMonitor)2/ParMultiDo(TimeMonitor) into Step: 9/ParMultiDo(CounterOperation)
Sep 13, 2021 12:43:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-13T12:43:43.266Z: Executing operation Read input/Impulse+Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)+Read-input-ParDo-BoundedSourceAsSDFWrapper--ParMultiDo-BoundedSourceAsSDFWrapper-/PairWithRestriction+Read-input-ParDo-BoundedSourceAsSDFWrapper--ParMultiDo-BoundedSourceAsSDFWrapper-/SplitWithSizing
Sep 13, 2021 12:43:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-13T12:43:43.361Z: Starting 5 ****s in us-central1-a...
Sep 13, 2021 12:44:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-13T12:44:14.356Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Sep 13, 2021 12:44:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-09-13T12:44:14.630Z: Autoscaling: Startup of the **** pool in zone us-central1-a reached 1 ****s, but the goal was 5 ****s. The service will retry. QUOTA_EXCEEDED: Quota 'CPUS' exceeded. Limit: 2500.0 in region us-central1.
Sep 13, 2021 12:44:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-13T12:44:27.048Z: Startup of the **** pool in zone us-central1-a failed to bring up any of the desired 5 ****s. QUOTA_EXCEEDED: Quota 'CPUS' exceeded. Limit: 2500.0 in region us-central1.
Sep 13, 2021 12:44:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-13T12:44:27.083Z: Workflow failed.
Sep 13, 2021 12:44:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-09-13T12:44:27.181Z: Unable to delete temp files: "gs://temp-storage-for-perf-tests/loadtests/dax-tmp-2021-09-13_05_43_33-77183844169192276-S01-0-ffd6e2b01bc7a324/tmp-ffd6e2b01bc7a5f3@DAX.sdfmeta."
Sep 13, 2021 12:44:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-09-13T12:44:27.217Z: S01:Read input/Impulse+Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)+Read-input-ParDo-BoundedSourceAsSDFWrapper--ParMultiDo-BoundedSourceAsSDFWrapper-/PairWithRestriction+Read-input-ParDo-BoundedSourceAsSDFWrapper--ParMultiDo-BoundedSourceAsSDFWrapper-/SplitWithSizing failed.
Sep 13, 2021 12:44:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-13T12:44:27.252Z: Finished operation Read input/Impulse+Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)+Read-input-ParDo-BoundedSourceAsSDFWrapper--ParMultiDo-BoundedSourceAsSDFWrapper-/PairWithRestriction+Read-input-ParDo-BoundedSourceAsSDFWrapper--ParMultiDo-BoundedSourceAsSDFWrapper-/SplitWithSizing
Sep 13, 2021 12:44:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-13T12:44:27.313Z: Cleaning up.
Sep 13, 2021 12:44:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-13T12:44:27.392Z: Stopping **** pool...
Sep 13, 2021 12:44:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-13T12:44:49.887Z: Worker pool stopped.
Sep 13, 2021 12:45:25 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-09-13_05_43_33-77183844169192276 failed with status FAILED.
Sep 13, 2021 12:45:25 PM org.apache.beam.sdk.testutils.metrics.MetricsReader getCounterMetric
SEVERE: Failed to get metric totalBytes.count, from namespace pardo
Load test results for test (ID): c3493c04-96da-4749-8043-60c2cf032340 and timestamp: 2021-09-13T12:43:27.476000000Z:
Metric: Value:
dataflow_v2_java11_runtime_sec 0.0
dataflow_v2_java11_total_bytes_count -1.0
Exception in thread "main" java.lang.RuntimeException: Invalid job state: FAILED.
at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
at org.apache.beam.sdk.loadtests.ParDoLoadTest.run(ParDoLoadTest.java:53)
at org.apache.beam.sdk.loadtests.ParDoLoadTest.main(ParDoLoadTest.java:103)
> Task :sdks:java:testing:load-tests:run FAILED
> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210913124134
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:0a5dabfc545165599d583278dce76db9ff0d7efef7c7be23a36ba60cdcc033fa
Deleted: sha256:f0c70a3baf36bcc98549b9bac33cbcae7c3de22791f0f3f04eaeb517da42c2ff
Deleted: sha256:1fb506ff01999ea74a8c4bbde902382a09b296d81b57c1063685a218c90fa542
Deleted: sha256:0698dd6615e4c0a9e9ebdc46bd2cd662a5932777ed66edd0bf0a8172b7ae4d44
Deleted: sha256:912df7a82926bf8e988a8651f644dc83776b594716f59f065359779180c98d3f
Deleted: sha256:9c17eaed81846e87fc8167faab75a6ba783a1250efe31fd5555f46dbefa85642
Deleted: sha256:f20869b32b19ca994cbe3c20cbf27ec407b0193ba2cbbb49d42eb89abc34735d
Deleted: sha256:0c0f565b1056eb1d2167ea343e28fcd7ab2faf7cb20f7cb015e4f558e4c4059e
Deleted: sha256:56ecafd2e468504b755c7e3bbd621d18b25748ded4c6e8408ff647c117e44419
Deleted: sha256:9ebc0e0990307c1bbc0518f95172ef627a6684cbee247771c8c509f4bacb1237
Deleted: sha256:952f0a94b297138bcab45cea3f64b7dca98dd0600a081bf61b963e753f64dc0e
Deleted: sha256:0dda026e22aba9efa73e86c1278030e17b31d9fa5daa15bbf0c4d22a349fc6c4
Deleted: sha256:80b5a7a0d8fedc93b2f4eb5987bc4b0f7996c813a2f080d8224f3e59f42413cc
Deleted: sha256:0637bd5107712023ed4510fc576c250e26647c374511f9af0a22dbb51cea0a94
Deleted: sha256:e753f0e405188ea33f63643ed3993a57787a3cd4b757cc2783441918d63f267f
Deleted: sha256:4aa583cd9bd5e1fd93e5c8e9e09f6dd5bf798312789af2c37392e87b78a3e10a
Deleted: sha256:af4f05077a17152641b512eaa40607d1b525890fb6f77a7b4d33c7642c605c8f
Deleted: sha256:4bd7398ab4eeed4590cad3808aa285d8b1003fa53acb136ddb4e8da63d0f0dc4
Deleted: sha256:28b372bbbe4ed78f5a44c445b1f469fc0a52691af64786f461e81caf6e132639
Deleted: sha256:398c3d13fb218db40c75ef3b35c87af3eafbfa0fefc82cf2701a1be7b56695f5
Deleted: sha256:a6733661be1b5971ea26508a23d0325842ceffac3e577561d2428440be6d868e
Deleted: sha256:d61e66202996695e4dd5e32ea62496851df9cedec41a393a96fc8848530d327a
Deleted: sha256:bc9ed45e868ef9aaff98e2eb2e0911e45a59d7ad9a6811bfdf0d3413410a7519
Deleted: sha256:1dccff1a98a9e3bd2d0c34f07f8ad716c0aaeb0d9c3dc3c18e276480736a5357
Deleted: sha256:cc3a6eba1a03448ed34aba34d19d6fd35046840ea9dade7219a0e5da1875ba55
Deleted: sha256:5729cafd091196c6f9a3c5165f32c4ab01290bfab823ae09f37775fd5c4455cc
Deleted: sha256:6f4c226ba089c6196a83fc2f71d0837d53b89933dbb9640fd289b84ce473d84f
Deleted: sha256:6ffd3fc53a8dd04024738802b124f5e0b5a59cdbb62ea68b64a699b33c6f30b2
Deleted: sha256:b2d8b1ca28e6cf516d003aae51d1e484af26c133ca39b36bf3922c6f5bd55f5d
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210913124134]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:0a5dabfc545165599d583278dce76db9ff0d7efef7c7be23a36ba60cdcc033fa]
Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210913124134] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:0a5dabfc545165599d583278dce76db9ff0d7efef7c7be23a36ba60cdcc033fa])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:0a5dabfc545165599d583278dce76db9ff0d7efef7c7be23a36ba60cdcc033fa
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:0a5dabfc545165599d583278dce76db9ff0d7efef7c7be23a36ba60cdcc033fa
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:0a5dabfc545165599d583278dce76db9ff0d7efef7c7be23a36ba60cdcc033fa].
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 9s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date
Publishing build scan...
https://gradle.com/s/7s7g5pudth2d6
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org