You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@beam.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2021/09/12 15:01:39 UTC
Build failed in Jenkins:
beam_LoadTests_Java_CoGBK_Dataflow_V2_Batch_Java11 #87
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Batch_Java11/87/display/redirect>
Changes:
------------------------------------------
[...truncated 96.09 KB...]
> Task :model:fn-execution:generateProto UP-TO-DATE
> Task :model:job-management:compileJava UP-TO-DATE
> Task :model:job-management:classes UP-TO-DATE
> Task :model:job-management:shadowJar UP-TO-DATE
> Task :model:fn-execution:compileJava UP-TO-DATE
> Task :model:fn-execution:classes UP-TO-DATE
> Task :model:fn-execution:shadowJar UP-TO-DATE
> Task :sdks:java:container:java11:copySdkHarnessLauncher
> Task :sdks:java:core:compileJava UP-TO-DATE
> Task :sdks:java:core:classes UP-TO-DATE
> Task :sdks:java:core:shadowJar UP-TO-DATE
> Task :sdks:java:extensions:protobuf:extractIncludeProto UP-TO-DATE
> Task :sdks:java:extensions:protobuf:generateProto NO-SOURCE
> Task :sdks:java:extensions:arrow:compileJava UP-TO-DATE
> Task :sdks:java:extensions:arrow:classes UP-TO-DATE
> Task :sdks:java:extensions:arrow:jar UP-TO-DATE
> Task :sdks:java:io:synthetic:compileJava UP-TO-DATE
> Task :sdks:java:io:synthetic:classes UP-TO-DATE
> Task :sdks:java:io:synthetic:jar UP-TO-DATE
> Task :sdks:java:extensions:protobuf:compileJava UP-TO-DATE
> Task :sdks:java:extensions:protobuf:classes UP-TO-DATE
> Task :sdks:java:io:kinesis:compileJava UP-TO-DATE
> Task :sdks:java:extensions:protobuf:jar UP-TO-DATE
> Task :sdks:java:io:kinesis:classes UP-TO-DATE
> Task :sdks:java:testing:test-utils:compileJava UP-TO-DATE
> Task :sdks:java:testing:test-utils:classes UP-TO-DATE
> Task :sdks:java:io:kinesis:jar UP-TO-DATE
> Task :sdks:java:testing:test-utils:jar UP-TO-DATE
> Task :sdks:java:fn-execution:compileJava UP-TO-DATE
> Task :sdks:java:fn-execution:classes UP-TO-DATE
> Task :sdks:java:fn-execution:jar UP-TO-DATE
> Task :runners:core-construction-java:compileJava UP-TO-DATE
> Task :runners:core-construction-java:classes UP-TO-DATE
> Task :runners:core-construction-java:jar UP-TO-DATE
> Task :runners:core-java:compileJava UP-TO-DATE
> Task :runners:core-java:classes UP-TO-DATE
> Task :runners:core-java:jar UP-TO-DATE
> Task :sdks:java:extensions:google-cloud-platform-core:compileJava UP-TO-DATE
> Task :sdks:java:extensions:google-cloud-platform-core:classes UP-TO-DATE
> Task :sdks:java:extensions:google-cloud-platform-core:jar UP-TO-DATE
> Task :sdks:java:harness:compileJava UP-TO-DATE
> Task :sdks:java:harness:classes UP-TO-DATE
> Task :sdks:java:harness:jar UP-TO-DATE
> Task :sdks:java:harness:shadowJar UP-TO-DATE
> Task :runners:java-fn-execution:compileJava UP-TO-DATE
> Task :runners:java-fn-execution:classes UP-TO-DATE
> Task :runners:java-fn-execution:jar UP-TO-DATE
> Task :sdks:java:expansion-service:compileJava UP-TO-DATE
> Task :sdks:java:expansion-service:classes UP-TO-DATE
> Task :sdks:java:expansion-service:jar UP-TO-DATE
> Task :sdks:java:io:kafka:compileJava UP-TO-DATE
> Task :sdks:java:io:kafka:classes UP-TO-DATE
> Task :sdks:java:io:kafka:jar UP-TO-DATE
> Task :sdks:java:container:java11:copyDockerfileDependencies UP-TO-DATE
> Task :sdks:java:io:google-cloud-platform:compileJava UP-TO-DATE
> Task :sdks:java:io:google-cloud-platform:classes UP-TO-DATE
> Task :sdks:java:io:google-cloud-platform:jar UP-TO-DATE
> Task :sdks:java:testing:load-tests:compileJava UP-TO-DATE
> Task :sdks:java:testing:load-tests:classes UP-TO-DATE
> Task :sdks:java:testing:load-tests:jar UP-TO-DATE
> Task :runners:google-cloud-dataflow-java:compileJava UP-TO-DATE
> Task :runners:google-cloud-dataflow-java:classes UP-TO-DATE
> Task :runners:google-cloud-dataflow-java:jar UP-TO-DATE
> Task :sdks:java:container:java11:dockerPrepare
> Task :sdks:java:container:java11:docker
> Task :runners:google-cloud-dataflow-java:buildAndPushDockerContainer
WARNING: `gcloud docker` will not be supported for Docker client versions above 18.03.
As an alternative, use `gcloud auth configure-docker` to configure `docker` to
use `gcloud` as a credential helper, then use `docker` as you would for non-GCR
registries, e.g. `docker pull gcr.io/project-id/my-image`. Add
`--verbosity=error` to silence this warning: `gcloud docker
--verbosity=error -- pull gcr.io/project-id/my-image`.
See: https://cloud.google.com/container-registry/docs/support/deprecation-notices#gcloud-docker
The push refers to repository [us.gcr.io/apache-beam-testing/java-postcommit-it/java]
6e24b7988ade: Preparing
b80b8aeaf46c: Preparing
d415eeb55498: Preparing
0f4ddefb278e: Preparing
fc6e9732d9ad: Preparing
1ea626c1103b: Preparing
f90fdabe51fa: Preparing
8f679cea6659: Preparing
645e2f3ecc4e: Preparing
111a5ed0ffdf: Preparing
a69cf31340b3: Preparing
798acf192be1: Preparing
3891808a925b: Preparing
d402f4f1b906: Preparing
00ef5416d927: Preparing
8555e663f65b: Preparing
d00da3cd7763: Preparing
4e61e63529c2: Preparing
799760671c38: Preparing
798acf192be1: Waiting
8555e663f65b: Waiting
d00da3cd7763: Waiting
4e61e63529c2: Waiting
d402f4f1b906: Waiting
3891808a925b: Waiting
1ea626c1103b: Waiting
8f679cea6659: Waiting
00ef5416d927: Waiting
f90fdabe51fa: Waiting
645e2f3ecc4e: Waiting
111a5ed0ffdf: Waiting
a69cf31340b3: Waiting
799760671c38: Waiting
b80b8aeaf46c: Pushed
d415eeb55498: Pushed
fc6e9732d9ad: Pushed
6e24b7988ade: Pushed
1ea626c1103b: Pushed
0f4ddefb278e: Pushed
8f679cea6659: Pushed
645e2f3ecc4e: Pushed
3891808a925b: Layer already exists
d402f4f1b906: Layer already exists
00ef5416d927: Layer already exists
8555e663f65b: Layer already exists
d00da3cd7763: Layer already exists
4e61e63529c2: Layer already exists
799760671c38: Layer already exists
a69cf31340b3: Pushed
f90fdabe51fa: Pushed
798acf192be1: Pushed
111a5ed0ffdf: Pushed
20210912150035: digest: sha256:7cfeee9f194e147a3baf3f381d3ef6319ad0a9767dda3e4dc7b5dde6f4b195d8 size: 4311
> Task :sdks:java:testing:load-tests:run
Sep 12, 2021 3:01:07 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Sep 12, 2021 3:01:08 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 195 files. Enable logging at DEBUG level to see which files will be staged.
Sep 12, 2021 3:01:09 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Sep 12, 2021 3:01:09 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Sep 12, 2021 3:01:11 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 195 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Sep 12, 2021 3:01:11 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 195 files cached, 0 files newly uploaded in 0 seconds
Sep 12, 2021 3:01:11 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Sep 12, 2021 3:01:11 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <101510 bytes, hash e9923899dae5b4613b3fa300d69ba0f4d9f55a062bbdb33ef62e467135736f8d> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-6ZI4mdrltGE7P6MA1pug9Nn1WgYrvbM-9i5GcTVzb40.pb
Sep 12, 2021 3:01:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input as step s1
Sep 12, 2021 3:01:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s2
Sep 12, 2021 3:01:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s3
Sep 12, 2021 3:01:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input as step s4
Sep 12, 2021 3:01:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s5
Sep 12, 2021 3:01:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s6
Sep 12, 2021 3:01:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s7
Sep 12, 2021 3:01:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s8
Sep 12, 2021 3:01:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s9
Sep 12, 2021 3:01:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s10
Sep 12, 2021 3:01:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s11
Sep 12, 2021 3:01:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s12
Sep 12, 2021 3:01:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s13
Sep 12, 2021 3:01:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s14
Sep 12, 2021 3:01:13 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.34.0-SNAPSHOT
Sep 12, 2021 3:01:14 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-09-12_08_01_13-9167744044806674124?project=apache-beam-testing
Sep 12, 2021 3:01:14 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-09-12_08_01_13-9167744044806674124
Sep 12, 2021 3:01:14 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-09-12_08_01_13-9167744044806674124
Sep 12, 2021 3:01:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-09-12T15:01:19.331Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20batch0cogbk02-jenkins-091215-97o9. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Sep 12, 2021 3:01:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-12T15:01:25.608Z: Worker configuration: e2-standard-2 in us-central1-c.
Sep 12, 2021 3:01:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-12T15:01:26.309Z: Workflow failed. Causes: The quota check has failed., Requested quota metric CPUs is currently unavailable.
Sep 12, 2021 3:01:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-12T15:01:26.338Z: Cleaning up.
Sep 12, 2021 3:01:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-12T15:01:26.398Z: Worker pool stopped.
Sep 12, 2021 3:01:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-12T15:01:27.377Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Sep 12, 2021 3:01:34 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-09-12_08_01_13-9167744044806674124 failed with status FAILED.
Sep 12, 2021 3:01:34 PM org.apache.beam.sdk.testutils.metrics.MetricsReader getCounterMetric
SEVERE: Failed to get metric totalBytes.count, from namespace co_gbk
Load test results for test (ID): faf338ac-9409-40cc-9285-391e3c8372a8 and timestamp: 2021-09-12T15:01:08.702000000Z:
Metric: Value:
dataflow_v2_java11_runtime_sec 0.0
dataflow_v2_java11_total_bytes_count -1.0
Exception in thread "main" java.lang.RuntimeException: Invalid job state: FAILED.
at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)
> Task :sdks:java:testing:load-tests:run FAILED
> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210912150035
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:7cfeee9f194e147a3baf3f381d3ef6319ad0a9767dda3e4dc7b5dde6f4b195d8
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210912150035]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:7cfeee9f194e147a3baf3f381d3ef6319ad0a9767dda3e4dc7b5dde6f4b195d8]
Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210912150035] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:7cfeee9f194e147a3baf3f381d3ef6319ad0a9767dda3e4dc7b5dde6f4b195d8])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:7cfeee9f194e147a3baf3f381d3ef6319ad0a9767dda3e4dc7b5dde6f4b195d8
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:7cfeee9f194e147a3baf3f381d3ef6319ad0a9767dda3e4dc7b5dde6f4b195d8
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:7cfeee9f194e147a3baf3f381d3ef6319ad0a9767dda3e4dc7b5dde6f4b195d8].
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1m 4s
96 actionable tasks: 9 executed, 87 up-to-date
Publishing build scan...
https://gradle.com/s/xqxgq7qfhsffi
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Jenkins build is back to normal :
beam_LoadTests_Java_CoGBK_Dataflow_V2_Batch_Java11 #89
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Batch_Java11/89/display/redirect?page=changes>
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_LoadTests_Java_CoGBK_Dataflow_V2_Batch_Java11 #88
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Batch_Java11/88/display/redirect>
Changes:
------------------------------------------
[...truncated 38.25 KB...]
> Task :sdks:java:expansion-service:compileJava FROM-CACHE
> Task :sdks:java:expansion-service:classes UP-TO-DATE
> Task :sdks:java:expansion-service:jar
> Task :sdks:java:io:kafka:compileJava FROM-CACHE
> Task :sdks:java:io:kafka:classes UP-TO-DATE
> Task :sdks:java:io:kafka:jar
> Task :sdks:java:io:google-cloud-platform:compileJava FROM-CACHE
> Task :sdks:java:io:google-cloud-platform:classes UP-TO-DATE
> Task :sdks:java:io:google-cloud-platform:jar
> Task :sdks:java:testing:load-tests:compileJava FROM-CACHE
> Task :sdks:java:testing:load-tests:classes UP-TO-DATE
> Task :sdks:java:testing:load-tests:jar
> Task :runners:google-cloud-dataflow-java:compileJava FROM-CACHE
> Task :runners:google-cloud-dataflow-java:classes
> Task :sdks:java:container:java11:copyDockerfileDependencies
> Task :runners:google-cloud-dataflow-java:jar
> Task :sdks:java:container:pullLicenses
Copying already-fetched licenses from <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Batch_Java11/ws/src/sdks/java/container/build/reports/dependency-license> to <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Batch_Java11/ws/src/sdks/java/container/build/target/java_third_party_licenses>
Already using interpreter /usr/bin/python3
Using base prefix '/usr'
New python executable in <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Batch_Java11/ws/src/sdks/java/container/build/virtualenv/bin/python3>
Also creating executable in <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Batch_Java11/ws/src/sdks/java/container/build/virtualenv/bin/python>
Installing setuptools, pip, wheel...
done.
DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality.
Collecting beautifulsoup4<5.0,>=4.9.0
Using cached beautifulsoup4-4.10.0-py3-none-any.whl (97 kB)
Collecting future<1.0.0,>=0.16.0
Using cached future-0.18.2-py3-none-any.whl
Collecting pyyaml<6.0.0,>=3.12
Using cached PyYAML-5.3.1-cp35-cp35m-linux_x86_64.whl
Collecting tenacity<6.0,>=5.0.2
Using cached tenacity-5.1.5-py2.py3-none-any.whl (34 kB)
Collecting soupsieve>1.2
Using cached soupsieve-2.1-py3-none-any.whl (32 kB)
Collecting six>=1.9.0
Using cached six-1.16.0-py2.py3-none-any.whl (11 kB)
Installing collected packages: soupsieve, six, tenacity, pyyaml, future, beautifulsoup4
Successfully installed beautifulsoup4-4.10.0 future-0.18.2 pyyaml-5.3.1 six-1.16.0 soupsieve-2.1 tenacity-5.1.5
Executing <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Batch_Java11/ws/src/sdks/java/container/build/virtualenv/bin/python> <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Batch_Java11/ws/src/sdks/java/container/license_scripts/pull_licenses_java.py> --license_index=<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Batch_Java11/ws/src/sdks/java/container/build/reports/dependency-license/index.json> --output_dir=<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Batch_Java11/ws/src/sdks/java/container/build/target/java_third_party_licenses> --dep_url_yaml=<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Batch_Java11/ws/src/sdks/java/container/license_scripts/dep_urls_java.yaml>
INFO:root:Pulling license for 207 dependencies using 16 threads.
INFO:root:pull_licenses_java.py succeed. It took 2.054025 seconds with 16 threads.
Copying licenses from <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Batch_Java11/ws/src/sdks/java/container/build/target/java_third_party_licenses> to <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Batch_Java11/ws/src/sdks/java/container/build/target/third_party_licenses.>
Finished license_scripts.sh
> Task :sdks:java:container:java11:copyJavaThirdPartyLicenses
> Task :release:go-licenses:java:dockerRun
+ go-licenses save github.com/apache/beam/sdks/java/container --save_path=/output/licenses
+ go-licenses csv github.com/apache/beam/sdks/java/container
+ tee /output/licenses/list.csv
golang.org/x/sys,https://go.googlesource.com/sys/+/refs/heads/master/LICENSE,BSD-3-Clause
golang.org/x/text,https://go.googlesource.com/text/+/refs/heads/master/LICENSE,BSD-3-Clause
github.com/apache/beam/sdks/java/container,https://github.com/apache/beam/blob/master/LICENSE,Apache-2.0
google.golang.org/grpc,https://github.com/grpc/grpc-go/blob/master/LICENSE,Apache-2.0
golang.org/x/net,https://go.googlesource.com/net/+/refs/heads/master/LICENSE,BSD-3-Clause
github.com/golang/protobuf,https://github.com/golang/protobuf/blob/master/LICENSE,BSD-3-Clause
github.com/apache/beam/sdks/go/pkg/beam,https://github.com/apache/beam/blob/master/sdks/go/README.md,Apache-2.0
google.golang.org/protobuf,https://go.googlesource.com/protobuf/+/refs/heads/master/LICENSE,BSD-3-Clause
google.golang.org/genproto/googleapis/rpc/status,https://github.com/googleapis/go-genproto/blob/master/LICENSE,Apache-2.0
+ chmod -R a+w /output/licenses
> Task :release:go-licenses:java:createLicenses
> Task :sdks:java:container:java11:copyGolangLicenses
> Task :sdks:java:container:java11:dockerPrepare
> Task :sdks:java:container:java11:docker
> Task :runners:google-cloud-dataflow-java:buildAndPushDockerContainer
WARNING: `gcloud docker` will not be supported for Docker client versions above 18.03.
As an alternative, use `gcloud auth configure-docker` to configure `docker` to
use `gcloud` as a credential helper, then use `docker` as you would for non-GCR
registries, e.g. `docker pull gcr.io/project-id/my-image`. Add
`--verbosity=error` to silence this warning: `gcloud docker
--verbosity=error -- pull gcr.io/project-id/my-image`.
See: https://cloud.google.com/container-registry/docs/support/deprecation-notices#gcloud-docker
The push refers to repository [us.gcr.io/apache-beam-testing/java-postcommit-it/java]
e351cc385439: Preparing
06fe54a7b0e4: Preparing
7ee5a6b7b088: Preparing
c35422b1a0bc: Preparing
52558eb498dc: Preparing
5703d200b252: Preparing
14fce565cb97: Preparing
dc94c2a9b231: Preparing
aba4c4a66159: Preparing
ffde52e9fe8d: Preparing
1bd4cac83540: Preparing
bcb357b5a840: Preparing
3891808a925b: Preparing
d402f4f1b906: Preparing
00ef5416d927: Preparing
8555e663f65b: Preparing
d00da3cd7763: Preparing
4e61e63529c2: Preparing
799760671c38: Preparing
ffde52e9fe8d: Waiting
1bd4cac83540: Waiting
8555e663f65b: Waiting
bcb357b5a840: Waiting
799760671c38: Waiting
d00da3cd7763: Waiting
3891808a925b: Waiting
d402f4f1b906: Waiting
00ef5416d927: Waiting
4e61e63529c2: Waiting
5703d200b252: Waiting
aba4c4a66159: Waiting
7ee5a6b7b088: Pushed
06fe54a7b0e4: Pushed
52558eb498dc: Pushed
e351cc385439: Pushed
c35422b1a0bc: Pushed
5703d200b252: Pushed
dc94c2a9b231: Pushed
aba4c4a66159: Pushed
3891808a925b: Layer already exists
d402f4f1b906: Layer already exists
00ef5416d927: Layer already exists
8555e663f65b: Layer already exists
d00da3cd7763: Layer already exists
1bd4cac83540: Pushed
4e61e63529c2: Layer already exists
799760671c38: Layer already exists
14fce565cb97: Pushed
bcb357b5a840: Pushed
ffde52e9fe8d: Pushed
20210913145232: digest: sha256:f871d9a17870a733716321ec7c56cf6602dfe355308a04fe572a8a7bb56b5ecb size: 4311
> Task :sdks:java:testing:load-tests:run
Sep 13, 2021 2:54:18 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Sep 13, 2021 2:54:18 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 195 files. Enable logging at DEBUG level to see which files will be staged.
Sep 13, 2021 2:54:19 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Sep 13, 2021 2:54:19 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Sep 13, 2021 2:54:21 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 195 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Sep 13, 2021 2:54:22 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 195 files cached, 0 files newly uploaded in 0 seconds
Sep 13, 2021 2:54:22 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/
Sep 13, 2021 2:54:22 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <101510 bytes, hash 46e682cd7600288a93033b33392a410039b49a9aa9b954d5d0bb9833740ca2bb> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-RuaCzXYAKIqTAzszOSpBADm0mpqpuVTV0LuYM3QMors.pb
Sep 13, 2021 2:54:23 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input as step s1
Sep 13, 2021 2:54:23 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s2
Sep 13, 2021 2:54:23 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s3
Sep 13, 2021 2:54:23 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input as step s4
Sep 13, 2021 2:54:23 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s5
Sep 13, 2021 2:54:23 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s6
Sep 13, 2021 2:54:23 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s7
Sep 13, 2021 2:54:23 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s8
Sep 13, 2021 2:54:23 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s9
Sep 13, 2021 2:54:23 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s10
Sep 13, 2021 2:54:23 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s11
Sep 13, 2021 2:54:23 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s12
Sep 13, 2021 2:54:23 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s13
Sep 13, 2021 2:54:23 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s14
Sep 13, 2021 2:54:23 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.34.0-SNAPSHOT
Sep 13, 2021 2:54:25 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-09-13_07_54_24-17861552043418126175?project=apache-beam-testing
Sep 13, 2021 2:54:25 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-09-13_07_54_24-17861552043418126175
Sep 13, 2021 2:54:25 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-09-13_07_54_24-17861552043418126175
Sep 13, 2021 2:54:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-09-13T14:54:28.892Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20batch0cogbk01-jenkins-091314-e8na. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Sep 13, 2021 2:54:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-13T14:54:32.609Z: Worker configuration: e2-standard-2 in us-central1-a.
Sep 13, 2021 2:54:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-13T14:54:33.473Z: Workflow failed. Causes: The quota check has failed., Requested quota metric CPUs is currently unavailable.
Sep 13, 2021 2:54:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-13T14:54:33.517Z: Cleaning up.
Sep 13, 2021 2:54:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-13T14:54:33.568Z: Worker pool stopped.
Sep 13, 2021 2:54:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-13T14:54:34.746Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Sep 13, 2021 2:54:39 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-09-13_07_54_24-17861552043418126175 failed with status FAILED.
Sep 13, 2021 2:54:39 PM org.apache.beam.sdk.testutils.metrics.MetricsReader getCounterMetric
SEVERE: Failed to get metric totalBytes.count, from namespace co_gbk
Load test results for test (ID): 8f0d7e66-3388-4e79-b754-a4828f3eaa34 and timestamp: 2021-09-13T14:54:19.092000000Z:
Metric: Value:
dataflow_v2_java11_runtime_sec 0.0
dataflow_v2_java11_total_bytes_count -1.0
Exception in thread "main" java.lang.RuntimeException: Invalid job state: FAILED.
at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55)
at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141)
at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)
> Task :sdks:java:testing:load-tests:run FAILED
> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210913145232
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:f871d9a17870a733716321ec7c56cf6602dfe355308a04fe572a8a7bb56b5ecb
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210913145232]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:f871d9a17870a733716321ec7c56cf6602dfe355308a04fe572a8a7bb56b5ecb]
Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210913145232] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:f871d9a17870a733716321ec7c56cf6602dfe355308a04fe572a8a7bb56b5ecb])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:f871d9a17870a733716321ec7c56cf6602dfe355308a04fe572a8a7bb56b5ecb
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:f871d9a17870a733716321ec7c56cf6602dfe355308a04fe572a8a7bb56b5ecb
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:f871d9a17870a733716321ec7c56cf6602dfe355308a04fe572a8a7bb56b5ecb].
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 2m 26s
101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date
Publishing build scan...
https://gradle.com/s/taot54ohwv4r4
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org