You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@beam.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2021/09/13 06:22:28 UTC
Build failed in Jenkins: beam_PerformanceTests_TFRecordIOIT #5317
See <https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/5317/display/redirect>
Changes:
------------------------------------------
[...truncated 274.65 KB...]
Task ':sdks:java:io:google-cloud-platform:jar' is not up-to-date because:
No history is available.
file or directory '<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/google-cloud-platform/build/resources/main',> not found
:sdks:java:io:google-cloud-platform:jar (Thread[Execution **** for ':' Thread 3,5,main]) completed. Took 0.23 secs.
:runners:google-cloud-dataflow-java:compileJava (Thread[Execution **** for ':' Thread 3,5,main]) started.
:sdks:java:io:google-cloud-platform:compileTestJava (Thread[Daemon ****,5,main]) started.
> Task :runners:google-cloud-dataflow-java:compileJava FROM-CACHE
Custom actions are attached to task ':runners:google-cloud-dataflow-java:compileJava'.
Build cache key for task ':runners:google-cloud-dataflow-java:compileJava' is f4d45dae85b41e621c919183804b6afa
Task ':runners:google-cloud-dataflow-java:compileJava' is not up-to-date because:
No history is available.
Loaded cache entry for task ':runners:google-cloud-dataflow-java:compileJava' with cache key f4d45dae85b41e621c919183804b6afa
:runners:google-cloud-dataflow-java:compileJava (Thread[Execution **** for ':' Thread 3,5,main]) completed. Took 0.214 secs.
:runners:google-cloud-dataflow-java:classes (Thread[Execution **** for ':' Thread 3,5,main]) started.
> Task :runners:google-cloud-dataflow-java:classes
Skipping task ':runners:google-cloud-dataflow-java:classes' as it has no actions.
:runners:google-cloud-dataflow-java:classes (Thread[Execution **** for ':' Thread 3,5,main]) completed. Took 0.0 secs.
:runners:google-cloud-dataflow-java:jar (Thread[Execution **** for ':' Thread 3,5,main]) started.
> Task :runners:google-cloud-dataflow-java:jar
Caching disabled for task ':runners:google-cloud-dataflow-java:jar' because:
Caching has not been enabled for the task
Task ':runners:google-cloud-dataflow-java:jar' is not up-to-date because:
No history is available.
:runners:google-cloud-dataflow-java:jar (Thread[Execution **** for ':' Thread 3,5,main]) completed. Took 0.065 secs.
:runners:google-cloud-dataflow-java:****:legacy-****:compileJava (Thread[Execution **** for ':' Thread 3,5,main]) started.
> Task :sdks:java:io:google-cloud-platform:compileTestJava FROM-CACHE
Custom actions are attached to task ':sdks:java:io:google-cloud-platform:compileTestJava'.
Build cache key for task ':sdks:java:io:google-cloud-platform:compileTestJava' is 99462c52f9bbbb5d949b676d07ee3bc7
Task ':sdks:java:io:google-cloud-platform:compileTestJava' is not up-to-date because:
No history is available.
Loaded cache entry for task ':sdks:java:io:google-cloud-platform:compileTestJava' with cache key 99462c52f9bbbb5d949b676d07ee3bc7
:sdks:java:io:google-cloud-platform:compileTestJava (Thread[Daemon ****,5,main]) completed. Took 0.425 secs.
:sdks:java:io:google-cloud-platform:testClasses (Thread[Daemon ****,5,main]) started.
> Task :sdks:java:io:google-cloud-platform:testClasses
Skipping task ':sdks:java:io:google-cloud-platform:testClasses' as it has no actions.
:sdks:java:io:google-cloud-platform:testClasses (Thread[Daemon ****,5,main]) completed. Took 0.0 secs.
:sdks:java:io:google-cloud-platform:testJar (Thread[Daemon ****,5,main]) started.
> Task :runners:google-cloud-dataflow-java:****:legacy-****:compileJava FROM-CACHE
file or directory '<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/runners/google-cloud-dataflow-java/****/legacy-****/src/main/java',> not found
Custom actions are attached to task ':runners:google-cloud-dataflow-java:****:legacy-****:compileJava'.
Build cache key for task ':runners:google-cloud-dataflow-java:****:legacy-****:compileJava' is cf672da70ce14149e9f2f602b34780e4
Task ':runners:google-cloud-dataflow-java:****:legacy-****:compileJava' is not up-to-date because:
No history is available.
Loaded cache entry for task ':runners:google-cloud-dataflow-java:****:legacy-****:compileJava' with cache key cf672da70ce14149e9f2f602b34780e4
:runners:google-cloud-dataflow-java:****:legacy-****:compileJava (Thread[Execution **** for ':' Thread 3,5,main]) completed. Took 0.275 secs.
:runners:google-cloud-dataflow-java:****:legacy-****:classes (Thread[Execution **** for ':' Thread 3,5,main]) started.
> Task :runners:google-cloud-dataflow-java:****:legacy-****:classes UP-TO-DATE
Skipping task ':runners:google-cloud-dataflow-java:****:legacy-****:classes' as it has no actions.
:runners:google-cloud-dataflow-java:****:legacy-****:classes (Thread[Execution **** for ':' Thread 3,5,main]) completed. Took 0.0 secs.
:runners:google-cloud-dataflow-java:****:legacy-****:shadowJar (Thread[Execution **** for ':' Thread 3,5,main]) started.
> Task :sdks:java:io:google-cloud-platform:testJar
Caching disabled for task ':sdks:java:io:google-cloud-platform:testJar' because:
Caching has not been enabled for the task
Task ':sdks:java:io:google-cloud-platform:testJar' is not up-to-date because:
No history is available.
:sdks:java:io:google-cloud-platform:testJar (Thread[Daemon ****,5,main]) completed. Took 0.248 secs.
:runners:google-cloud-dataflow-java:compileTestJava (Thread[Daemon ****,5,main]) started.
> Task :runners:google-cloud-dataflow-java:****:legacy-****:shadowJar FROM-CACHE
Custom actions are attached to task ':runners:google-cloud-dataflow-java:****:legacy-****:shadowJar'.
Build cache key for task ':runners:google-cloud-dataflow-java:****:legacy-****:shadowJar' is 165b57b3de0480886088cb77a0fc5516
Task ':runners:google-cloud-dataflow-java:****:legacy-****:shadowJar' is not up-to-date because:
No history is available.
Loaded cache entry for task ':runners:google-cloud-dataflow-java:****:legacy-****:shadowJar' with cache key 165b57b3de0480886088cb77a0fc5516
:runners:google-cloud-dataflow-java:****:legacy-****:shadowJar (Thread[Execution **** for ':' Thread 3,5,main]) completed. Took 0.159 secs.
> Task :runners:google-cloud-dataflow-java:compileTestJava FROM-CACHE
Custom actions are attached to task ':runners:google-cloud-dataflow-java:compileTestJava'.
Build cache key for task ':runners:google-cloud-dataflow-java:compileTestJava' is 861c7435ffd549666858392e9bbd7747
Task ':runners:google-cloud-dataflow-java:compileTestJava' is not up-to-date because:
No history is available.
Loaded cache entry for task ':runners:google-cloud-dataflow-java:compileTestJava' with cache key 861c7435ffd549666858392e9bbd7747
:runners:google-cloud-dataflow-java:compileTestJava (Thread[Daemon ****,5,main]) completed. Took 0.171 secs.
:runners:google-cloud-dataflow-java:testClasses (Thread[Daemon ****,5,main]) started.
> Task :runners:google-cloud-dataflow-java:testClasses UP-TO-DATE
Skipping task ':runners:google-cloud-dataflow-java:testClasses' as it has no actions.
:runners:google-cloud-dataflow-java:testClasses (Thread[Daemon ****,5,main]) completed. Took 0.0 secs.
:runners:google-cloud-dataflow-java:testJar (Thread[Daemon ****,5,main]) started.
> Task :runners:google-cloud-dataflow-java:testJar
Caching disabled for task ':runners:google-cloud-dataflow-java:testJar' because:
Caching has not been enabled for the task
Task ':runners:google-cloud-dataflow-java:testJar' is not up-to-date because:
No history is available.
file or directory '<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/runners/google-cloud-dataflow-java/build/resources/test',> not found
:runners:google-cloud-dataflow-java:testJar (Thread[Daemon ****,5,main]) completed. Took 0.029 secs.
:sdks:java:io:file-based-io-tests:integrationTest (Thread[Daemon ****,5,main]) started.
Gradle Test Executor 1 started executing tests.
> Task :sdks:java:io:file-based-io-tests:integrationTest
Custom actions are attached to task ':sdks:java:io:file-based-io-tests:integrationTest'.
Build cache key for task ':sdks:java:io:file-based-io-tests:integrationTest' is d113188b16b5dc9e6e52b7033c284315
Task ':sdks:java:io:file-based-io-tests:integrationTest' is not up-to-date because:
Task.upToDateWhen is false.
Starting process 'Gradle Test Executor 1'. Working directory: <https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/file-based-io-tests> Command: /usr/lib/jvm/java-8-openjdk-amd64/bin/java -DbeamTestPipelineOptions=["--runner=DataflowRunner","--project=apache-beam-testing","--tempRoot=gs://temp-storage-for-perf-tests","--filenamePrefix=gs://temp-storage-for-perf-tests/beam_PerformanceTests_TFRecordIOIT/5317/","--bigQueryDataset=beam_performance","--bigQueryTable=tfrecordioit_results","--influxMeasurement=tfrecordioit_results","--numberOfRecords=18000000","--expectedHash=543104423f8b6eb097acb9f111c19fe4","--datasetSize=1019380000","--numWorkers=5","--autoscalingAlgorithm=NONE","--influxDatabase=beam_test_metrics","--influxHost=http://10.128.0.96:8086","--****HarnessContainerImage=","--dataflowWorkerJar=<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/runners/google-cloud-dataflow-java/****/legacy-****/build/libs/beam-runners-google-cloud-dataflow-java-legacy-****-2.34.0-SNAPSHOT.jar","--region=us-central1"]> -Djava.security.manager=****.org.gradle.process.internal.****.child.BootstrapSecurityManager -Dorg.gradle.native=false -Xmx2g -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant -ea -cp /home/jenkins/.gradle/caches/6.8.3/****Main/gradle-****.jar ****.org.gradle.process.internal.****.GradleWorkerMain 'Gradle Test Executor 1'
Successfully started process 'Gradle Test Executor 1'
org.apache.beam.sdk.io.tfrecord.TFRecordIOIT STANDARD_ERROR
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/runners/google-cloud-dataflow-java/****/legacy-****/build/libs/beam-runners-google-cloud-dataflow-java-legacy-****-2.34.0-SNAPSHOT.jar!/org/slf4j/impl/StaticLoggerBinder.class]>
SLF4J: Found binding in [jar:file:/home/jenkins/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-log4j12/1.7.30/c21f55139d8141d2231214fb1feaf50a1edca95e/slf4j-log4j12-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/jenkins/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-jdk14/1.7.30/d35953dd2fe54ebe39fdf18cfd82fe6eb35b25ed/slf4j-jdk14-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.JDK14LoggerFactory]
org.apache.beam.sdk.io.tfrecord.TFRecordIOIT > writeThenReadAll STANDARD_ERROR
Sep 13, 2021 6:21:52 AM org.apache.beam.runners.dataflow.DataflowRunner validateSdkContainerImageOptions
WARNING: Prefer --sdkContainerImage over deprecated legacy option --****HarnessContainerImage.
Sep 13, 2021 6:21:53 AM org.apache.beam.sdk.extensions.gcp.options.GcpOptions$GcpTempLocationFactory tryCreateDefaultBucket
INFO: No tempLocation specified, attempting to use default bucket: dataflow-staging-us-central1-844138762903
Sep 13, 2021 6:21:54 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 409, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/storage/v1/b?predefinedAcl=projectPrivate&predefinedDefaultObjectAcl=projectPrivate&project=apache-beam-testing.
Sep 13, 2021 6:21:54 AM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Sep 13, 2021 6:21:54 AM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 317 files. Enable logging at DEBUG level to see which files will be staged.
Sep 13, 2021 6:21:54 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Sep 13, 2021 6:21:57 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 318 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Sep 13, 2021 6:21:58 AM org.apache.beam.runners.dataflow.util.PackageUtil$PackageAttributes forFileToStage
INFO: Staging custom dataflow-****.jar as beam-runners-google-cloud-dataflow-java-legacy-****-2.34.0-SNAPSHOT-YtvkOIsfQYfFzG89aVBO_8oNHoiij9SynwUJLdMkn5A.jar
Sep 13, 2021 6:21:58 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /tmp/test3335254973028486526.zip to gs://dataflow-staging-us-central1-844138762903/temp/staging/test-Rtqv9PiExTh-m1JJfn_mbfcCSRlKNPqigkGqJikYPeA.jar
Sep 13, 2021 6:21:58 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 317 files cached, 1 files newly uploaded in 0 seconds
Sep 13, 2021 6:21:58 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://dataflow-staging-us-central1-844138762903/temp/staging/
Sep 13, 2021 6:21:58 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <205290 bytes, hash a5e95a8e577993f02a9271d721ede350caf87bf65e8c4d6b04b2df35743fd5ff> to gs://dataflow-staging-us-central1-844138762903/temp/staging/pipeline-pelajld5k_AqknHXIe3jUMr4e_ZejE1rBLLfNXQ_1f8.pb
Sep 13, 2021 6:22:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Generate sequence/Read(BoundedCountingSource) as step s1
Sep 13, 2021 6:22:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Produce text lines as step s2
Sep 13, 2021 6:22:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Transform strings to bytes/Map as step s3
Sep 13, 2021 6:22:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Record time before writing as step s4
Sep 13, 2021 6:22:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to files/Write/RewindowIntoGlobal/Window.Assign as step s5
Sep 13, 2021 6:22:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to files/Write/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles as step s6
Sep 13, 2021 6:22:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten as step s7
Sep 13, 2021 6:22:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to files/Write/WriteUnshardedBundlesToTempFiles/WriteUnwritten as step s8
Sep 13, 2021 6:22:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to files/Write/WriteUnshardedBundlesToTempFiles/DropShardNum as step s9
Sep 13, 2021 6:22:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to files/Write/WriteUnshardedBundlesToTempFiles/Flatten.PCollections as step s10
Sep 13, 2021 6:22:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to files/Write/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow) as step s11
Sep 13, 2021 6:22:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to files/Write/GatherTempFileResults/View.AsList/CreateDataflowView as step s12
Sep 13, 2021 6:22:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Create.Values/Read(CreateSource) as step s13
Sep 13, 2021 6:22:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Reify.ReifyView/ParDo(Anonymous) as step s14
Sep 13, 2021 6:22:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Values/Values/Map as step s15
Sep 13, 2021 6:22:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Finalize as step s16
Sep 13, 2021 6:22:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random key as step s17
Sep 13, 2021 6:22:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign as step s18
Sep 13, 2021 6:22:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey as step s19
Sep 13, 2021 6:22:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable as step s20
Sep 13, 2021 6:22:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Values/Values/Map as step s21
Sep 13, 2021 6:22:00 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.34.0-SNAPSHOT
Sep 13, 2021 6:22:01 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-09-12_23_22_00-12179573304408372732?project=apache-beam-testing
Sep 13, 2021 6:22:01 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-09-12_23_22_00-12179573304408372732
Sep 13, 2021 6:22:01 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-09-12_23_22_00-12179573304408372732
Sep 13, 2021 6:22:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-13T06:22:19.719Z: Worker configuration: e2-standard-2 in us-central1-a.
Sep 13, 2021 6:22:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-13T06:22:20.724Z: Workflow failed. Causes: Project apache-beam-testing has insufficient quota(s) to execute this workflow with 1 instances in region us-central1. Quota summary (required/available): 1/24386 instances, 2/0 CPUs, 25/182241 disk GB, 0/2397 SSD disk GB, 1/283 instance groups, 1/286 managed instance groups, 1/512 instance templates, 1/615 in-use IP addresses.
Please see https://cloud.google.com/compute/docs/resource-quotas about requesting more quota.
Sep 13, 2021 6:22:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-13T06:22:20.771Z: Cleaning up.
Sep 13, 2021 6:22:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-13T06:22:20.820Z: Worker pool stopped.
Sep 13, 2021 6:22:24 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-13T06:22:22.035Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Sep 13, 2021 6:22:25 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-09-12_23_22_00-12179573304408372732 failed with status FAILED.
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:io:file-based-io-tests:integrationTest FAILED
org.apache.beam.sdk.io.tfrecord.TFRecordIOIT > writeThenReadAll FAILED
java.lang.IllegalStateException: Unable to find any files matching gs://temp-storage-for-perf-tests/beam_PerformanceTests_TFRecordIOIT/5317/_1631514111983*
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:588)
at org.apache.beam.sdk.io.TFRecordIO$Read.expand(TFRecordIO.java:191)
at org.apache.beam.sdk.io.TFRecordIO$Read.expand(TFRecordIO.java:112)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:548)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:482)
at org.apache.beam.sdk.values.PBegin.apply(PBegin.java:44)
at org.apache.beam.sdk.Pipeline.apply(Pipeline.java:177)
at org.apache.beam.sdk.io.tfrecord.TFRecordIOIT.writeThenReadAll(TFRecordIOIT.java:151)
1 test completed, 1 failed
Finished generating test XML results (0.02 secs) into: <https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/file-based-io-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.033 secs) into: <https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/file-based-io-tests/build/reports/tests/integrationTest>
:sdks:java:io:file-based-io-tests:integrationTest (Thread[Daemon ****,5,main]) completed. Took 37.316 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:io:file-based-io-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/file-based-io-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1m 5s
129 actionable tasks: 79 executed, 50 from cache
Publishing build scan...
https://gradle.com/s/d5ebmbrnxmlh6
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Jenkins build is back to normal :
beam_PerformanceTests_TFRecordIOIT #5320
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/5320/display/redirect?page=changes>
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PerformanceTests_TFRecordIOIT #5319
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/5319/display/redirect?page=changes>
Changes:
[rohde.samuel] [BEAM-12842] Add timestamp to test work item to deflake
[suztomo] [BEAM-12873] HL7v2IO: to leave schematizedData null, not empty
------------------------------------------
[...truncated 274.42 KB...]
No history is available.
file or directory '<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/google-cloud-platform/build/resources/main',> not found
:sdks:java:io:google-cloud-platform:jar (Thread[Execution **** for ':' Thread 5,5,main]) completed. Took 0.206 secs.
:runners:google-cloud-dataflow-java:compileJava (Thread[Execution **** for ':' Thread 5,5,main]) started.
:sdks:java:io:google-cloud-platform:compileTestJava (Thread[Execution **** for ':' Thread 2,5,main]) started.
> Task :runners:google-cloud-dataflow-java:compileJava FROM-CACHE
Custom actions are attached to task ':runners:google-cloud-dataflow-java:compileJava'.
Build cache key for task ':runners:google-cloud-dataflow-java:compileJava' is f4d45dae85b41e621c919183804b6afa
Task ':runners:google-cloud-dataflow-java:compileJava' is not up-to-date because:
No history is available.
Loaded cache entry for task ':runners:google-cloud-dataflow-java:compileJava' with cache key f4d45dae85b41e621c919183804b6afa
:runners:google-cloud-dataflow-java:compileJava (Thread[Execution **** for ':' Thread 5,5,main]) completed. Took 0.168 secs.
:runners:google-cloud-dataflow-java:classes (Thread[Execution **** for ':' Thread 5,5,main]) started.
> Task :runners:google-cloud-dataflow-java:classes
Skipping task ':runners:google-cloud-dataflow-java:classes' as it has no actions.
:runners:google-cloud-dataflow-java:classes (Thread[Execution **** for ':' Thread 5,5,main]) completed. Took 0.0 secs.
:runners:google-cloud-dataflow-java:jar (Thread[Execution **** for ':' Thread 8,5,main]) started.
> Task :runners:google-cloud-dataflow-java:jar
Caching disabled for task ':runners:google-cloud-dataflow-java:jar' because:
Caching has not been enabled for the task
Task ':runners:google-cloud-dataflow-java:jar' is not up-to-date because:
No history is available.
:runners:google-cloud-dataflow-java:jar (Thread[Execution **** for ':' Thread 8,5,main]) completed. Took 0.077 secs.
:runners:google-cloud-dataflow-java:****:legacy-****:compileJava (Thread[Execution **** for ':' Thread 8,5,main]) started.
> Task :sdks:java:io:google-cloud-platform:compileTestJava FROM-CACHE
Custom actions are attached to task ':sdks:java:io:google-cloud-platform:compileTestJava'.
Build cache key for task ':sdks:java:io:google-cloud-platform:compileTestJava' is 99462c52f9bbbb5d949b676d07ee3bc7
Task ':sdks:java:io:google-cloud-platform:compileTestJava' is not up-to-date because:
No history is available.
Loaded cache entry for task ':sdks:java:io:google-cloud-platform:compileTestJava' with cache key 99462c52f9bbbb5d949b676d07ee3bc7
:sdks:java:io:google-cloud-platform:compileTestJava (Thread[Execution **** for ':' Thread 2,5,main]) completed. Took 0.416 secs.
:sdks:java:io:google-cloud-platform:testClasses (Thread[Execution **** for ':' Thread 2,5,main]) started.
> Task :sdks:java:io:google-cloud-platform:testClasses
Skipping task ':sdks:java:io:google-cloud-platform:testClasses' as it has no actions.
:sdks:java:io:google-cloud-platform:testClasses (Thread[Execution **** for ':' Thread 2,5,main]) completed. Took 0.0 secs.
:sdks:java:io:google-cloud-platform:testJar (Thread[Execution **** for ':' Thread 2,5,main]) started.
> Task :runners:google-cloud-dataflow-java:****:legacy-****:compileJava FROM-CACHE
file or directory '<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/runners/google-cloud-dataflow-java/****/legacy-****/src/main/java',> not found
Custom actions are attached to task ':runners:google-cloud-dataflow-java:****:legacy-****:compileJava'.
Build cache key for task ':runners:google-cloud-dataflow-java:****:legacy-****:compileJava' is cf672da70ce14149e9f2f602b34780e4
Task ':runners:google-cloud-dataflow-java:****:legacy-****:compileJava' is not up-to-date because:
No history is available.
Loaded cache entry for task ':runners:google-cloud-dataflow-java:****:legacy-****:compileJava' with cache key cf672da70ce14149e9f2f602b34780e4
:runners:google-cloud-dataflow-java:****:legacy-****:compileJava (Thread[Execution **** for ':' Thread 8,5,main]) completed. Took 0.277 secs.
:runners:google-cloud-dataflow-java:****:legacy-****:classes (Thread[Execution **** for ':' Thread 8,5,main]) started.
> Task :runners:google-cloud-dataflow-java:****:legacy-****:classes UP-TO-DATE
Skipping task ':runners:google-cloud-dataflow-java:****:legacy-****:classes' as it has no actions.
:runners:google-cloud-dataflow-java:****:legacy-****:classes (Thread[Execution **** for ':' Thread 8,5,main]) completed. Took 0.0 secs.
:runners:google-cloud-dataflow-java:****:legacy-****:shadowJar (Thread[Execution **** for ':' Thread 8,5,main]) started.
> Task :sdks:java:io:google-cloud-platform:testJar
Caching disabled for task ':sdks:java:io:google-cloud-platform:testJar' because:
Caching has not been enabled for the task
Task ':sdks:java:io:google-cloud-platform:testJar' is not up-to-date because:
No history is available.
:sdks:java:io:google-cloud-platform:testJar (Thread[Execution **** for ':' Thread 2,5,main]) completed. Took 0.263 secs.
:runners:google-cloud-dataflow-java:compileTestJava (Thread[Execution **** for ':' Thread 2,5,main]) started.
> Task :runners:google-cloud-dataflow-java:****:legacy-****:shadowJar FROM-CACHE
Custom actions are attached to task ':runners:google-cloud-dataflow-java:****:legacy-****:shadowJar'.
Build cache key for task ':runners:google-cloud-dataflow-java:****:legacy-****:shadowJar' is 165b57b3de0480886088cb77a0fc5516
Task ':runners:google-cloud-dataflow-java:****:legacy-****:shadowJar' is not up-to-date because:
No history is available.
Loaded cache entry for task ':runners:google-cloud-dataflow-java:****:legacy-****:shadowJar' with cache key 165b57b3de0480886088cb77a0fc5516
:runners:google-cloud-dataflow-java:****:legacy-****:shadowJar (Thread[Execution **** for ':' Thread 8,5,main]) completed. Took 0.155 secs.
> Task :runners:google-cloud-dataflow-java:compileTestJava FROM-CACHE
Custom actions are attached to task ':runners:google-cloud-dataflow-java:compileTestJava'.
Build cache key for task ':runners:google-cloud-dataflow-java:compileTestJava' is 861c7435ffd549666858392e9bbd7747
Task ':runners:google-cloud-dataflow-java:compileTestJava' is not up-to-date because:
No history is available.
Loaded cache entry for task ':runners:google-cloud-dataflow-java:compileTestJava' with cache key 861c7435ffd549666858392e9bbd7747
:runners:google-cloud-dataflow-java:compileTestJava (Thread[Execution **** for ':' Thread 2,5,main]) completed. Took 0.215 secs.
:runners:google-cloud-dataflow-java:testClasses (Thread[Execution **** for ':' Thread 2,5,main]) started.
> Task :runners:google-cloud-dataflow-java:testClasses UP-TO-DATE
Skipping task ':runners:google-cloud-dataflow-java:testClasses' as it has no actions.
:runners:google-cloud-dataflow-java:testClasses (Thread[Execution **** for ':' Thread 2,5,main]) completed. Took 0.0 secs.
:runners:google-cloud-dataflow-java:testJar (Thread[Execution **** for ':' Thread 2,5,main]) started.
> Task :runners:google-cloud-dataflow-java:testJar
Caching disabled for task ':runners:google-cloud-dataflow-java:testJar' because:
Caching has not been enabled for the task
Task ':runners:google-cloud-dataflow-java:testJar' is not up-to-date because:
No history is available.
file or directory '<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/runners/google-cloud-dataflow-java/build/resources/test',> not found
:runners:google-cloud-dataflow-java:testJar (Thread[Execution **** for ':' Thread 2,5,main]) completed. Took 0.03 secs.
:sdks:java:io:file-based-io-tests:integrationTest (Thread[Execution **** for ':' Thread 2,5,main]) started.
> Task :sdks:java:io:file-based-io-tests:integrationTest
Custom actions are attached to task ':sdks:java:io:file-based-io-tests:integrationTest'.
Build cache key for task ':sdks:java:io:file-based-io-tests:integrationTest' is 7afd111fb1a96c3526e1f49036cea8a2
Task ':sdks:java:io:file-based-io-tests:integrationTest' is not up-to-date because:
Task.upToDateWhen is false.
Starting process 'Gradle Test Executor 1'. Working directory: <https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/file-based-io-tests> Command: /usr/lib/jvm/java-8-openjdk-amd64/bin/java -DbeamTestPipelineOptions=["--runner=DataflowRunner","--project=apache-beam-testing","--tempRoot=gs://temp-storage-for-perf-tests","--filenamePrefix=gs://temp-storage-for-perf-tests/beam_PerformanceTests_TFRecordIOIT/5319/","--bigQueryDataset=beam_performance","--bigQueryTable=tfrecordioit_results","--influxMeasurement=tfrecordioit_results","--numberOfRecords=18000000","--expectedHash=543104423f8b6eb097acb9f111c19fe4","--datasetSize=1019380000","--numWorkers=5","--autoscalingAlgorithm=NONE","--influxDatabase=beam_test_metrics","--influxHost=http://10.128.0.96:8086","--****HarnessContainerImage=","--dataflowWorkerJar=<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/runners/google-cloud-dataflow-java/****/legacy-****/build/libs/beam-runners-google-cloud-dataflow-java-legacy-****-2.34.0-SNAPSHOT.jar","--region=us-central1"]> -Djava.security.manager=****.org.gradle.process.internal.****.child.BootstrapSecurityManager -Dorg.gradle.native=false -Xmx2g -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant -ea -cp /home/jenkins/.gradle/caches/6.8.3/****Main/gradle-****.jar ****.org.gradle.process.internal.****.GradleWorkerMain 'Gradle Test Executor 1'
Successfully started process 'Gradle Test Executor 1'
Gradle Test Executor 1 started executing tests.
> Task :sdks:java:io:file-based-io-tests:integrationTest
org.apache.beam.sdk.io.tfrecord.TFRecordIOIT STANDARD_ERROR
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/runners/google-cloud-dataflow-java/****/legacy-****/build/libs/beam-runners-google-cloud-dataflow-java-legacy-****-2.34.0-SNAPSHOT.jar!/org/slf4j/impl/StaticLoggerBinder.class]>
SLF4J: Found binding in [jar:file:/home/jenkins/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-log4j12/1.7.30/c21f55139d8141d2231214fb1feaf50a1edca95e/slf4j-log4j12-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/jenkins/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-jdk14/1.7.30/d35953dd2fe54ebe39fdf18cfd82fe6eb35b25ed/slf4j-jdk14-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.JDK14LoggerFactory]
org.apache.beam.sdk.io.tfrecord.TFRecordIOIT > writeThenReadAll STANDARD_ERROR
Sep 13, 2021 6:21:51 PM org.apache.beam.runners.dataflow.DataflowRunner validateSdkContainerImageOptions
WARNING: Prefer --sdkContainerImage over deprecated legacy option --****HarnessContainerImage.
Sep 13, 2021 6:21:52 PM org.apache.beam.sdk.extensions.gcp.options.GcpOptions$GcpTempLocationFactory tryCreateDefaultBucket
INFO: No tempLocation specified, attempting to use default bucket: dataflow-staging-us-central1-844138762903
Sep 13, 2021 6:21:53 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 409, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/storage/v1/b?predefinedAcl=projectPrivate&predefinedDefaultObjectAcl=projectPrivate&project=apache-beam-testing.
Sep 13, 2021 6:21:53 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Sep 13, 2021 6:21:53 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 317 files. Enable logging at DEBUG level to see which files will be staged.
Sep 13, 2021 6:21:54 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Sep 13, 2021 6:21:57 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 318 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Sep 13, 2021 6:21:57 PM org.apache.beam.runners.dataflow.util.PackageUtil$PackageAttributes forFileToStage
INFO: Staging custom dataflow-****.jar as beam-runners-google-cloud-dataflow-java-legacy-****-2.34.0-SNAPSHOT-YtvkOIsfQYfFzG89aVBO_8oNHoiij9SynwUJLdMkn5A.jar
Sep 13, 2021 6:21:57 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 318 files cached, 0 files newly uploaded in 0 seconds
Sep 13, 2021 6:21:57 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://dataflow-staging-us-central1-844138762903/temp/staging/
Sep 13, 2021 6:21:57 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <205290 bytes, hash 9c00b8f6d51a76ad7b86305270ea4b60e9f2f9e5b00090b0cdceacaa184afe61> to gs://dataflow-staging-us-central1-844138762903/temp/staging/pipeline-nAC49tUadq17hjBScOpLYOny-eWwAJCwzc6sqhhK_mE.pb
Sep 13, 2021 6:21:59 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Generate sequence/Read(BoundedCountingSource) as step s1
Sep 13, 2021 6:21:59 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Produce text lines as step s2
Sep 13, 2021 6:21:59 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Transform strings to bytes/Map as step s3
Sep 13, 2021 6:21:59 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Record time before writing as step s4
Sep 13, 2021 6:21:59 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to files/Write/RewindowIntoGlobal/Window.Assign as step s5
Sep 13, 2021 6:21:59 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to files/Write/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles as step s6
Sep 13, 2021 6:21:59 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten as step s7
Sep 13, 2021 6:21:59 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to files/Write/WriteUnshardedBundlesToTempFiles/WriteUnwritten as step s8
Sep 13, 2021 6:21:59 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to files/Write/WriteUnshardedBundlesToTempFiles/DropShardNum as step s9
Sep 13, 2021 6:21:59 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to files/Write/WriteUnshardedBundlesToTempFiles/Flatten.PCollections as step s10
Sep 13, 2021 6:21:59 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to files/Write/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow) as step s11
Sep 13, 2021 6:21:59 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to files/Write/GatherTempFileResults/View.AsList/CreateDataflowView as step s12
Sep 13, 2021 6:21:59 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Create.Values/Read(CreateSource) as step s13
Sep 13, 2021 6:21:59 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Reify.ReifyView/ParDo(Anonymous) as step s14
Sep 13, 2021 6:22:00 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Values/Values/Map as step s15
Sep 13, 2021 6:22:00 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Finalize as step s16
Sep 13, 2021 6:22:00 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random key as step s17
Sep 13, 2021 6:22:00 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign as step s18
Sep 13, 2021 6:22:00 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey as step s19
Sep 13, 2021 6:22:00 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable as step s20
Sep 13, 2021 6:22:00 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Values/Values/Map as step s21
Sep 13, 2021 6:22:00 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.34.0-SNAPSHOT
Sep 13, 2021 6:22:01 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-09-13_11_22_00-8936937029889501075?project=apache-beam-testing
Sep 13, 2021 6:22:01 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-09-13_11_22_00-8936937029889501075
Sep 13, 2021 6:22:01 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-09-13_11_22_00-8936937029889501075
Sep 13, 2021 6:22:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-13T18:22:21.324Z: Worker configuration: e2-standard-2 in us-central1-c.
Sep 13, 2021 6:22:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-13T18:22:22.834Z: Workflow failed. Causes: Project apache-beam-testing has insufficient quota(s) to execute this workflow with 1 instances in region us-central1. Quota summary (required/available): 1/24385 instances, 2/0 CPUs, 25/183466 disk GB, 0/2397 SSD disk GB, 1/262 instance groups, 1/265 managed instance groups, 1/489 instance templates, 1/614 in-use IP addresses.
Please see https://cloud.google.com/compute/docs/resource-quotas about requesting more quota.
Sep 13, 2021 6:22:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-13T18:22:22.884Z: Cleaning up.
Sep 13, 2021 6:22:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-13T18:22:22.953Z: Worker pool stopped.
Sep 13, 2021 6:22:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-13T18:22:24.160Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Sep 13, 2021 6:22:28 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-09-13_11_22_00-8936937029889501075 failed with status FAILED.
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:io:file-based-io-tests:integrationTest FAILED
org.apache.beam.sdk.io.tfrecord.TFRecordIOIT > writeThenReadAll FAILED
java.lang.IllegalStateException: Unable to find any files matching gs://temp-storage-for-perf-tests/beam_PerformanceTests_TFRecordIOIT/5319/_1631557310966*
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:588)
at org.apache.beam.sdk.io.TFRecordIO$Read.expand(TFRecordIO.java:191)
at org.apache.beam.sdk.io.TFRecordIO$Read.expand(TFRecordIO.java:112)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:548)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:482)
at org.apache.beam.sdk.values.PBegin.apply(PBegin.java:44)
at org.apache.beam.sdk.Pipeline.apply(Pipeline.java:177)
at org.apache.beam.sdk.io.tfrecord.TFRecordIOIT.writeThenReadAll(TFRecordIOIT.java:151)
1 test completed, 1 failed
Finished generating test XML results (0.019 secs) into: <https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/file-based-io-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.028 secs) into: <https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/file-based-io-tests/build/reports/tests/integrationTest>
:sdks:java:io:file-based-io-tests:integrationTest (Thread[Execution **** for ':' Thread 2,5,main]) completed. Took 42.053 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:io:file-based-io-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/file-based-io-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1m 11s
129 actionable tasks: 79 executed, 50 from cache
Publishing build scan...
https://gradle.com/s/v5fmtiigke3ow
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PerformanceTests_TFRecordIOIT #5318
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/5318/display/redirect>
Changes:
------------------------------------------
[...truncated 274.38 KB...]
Task ':sdks:java:io:google-cloud-platform:jar' is not up-to-date because:
No history is available.
file or directory '<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/google-cloud-platform/build/resources/main',> not found
:sdks:java:io:google-cloud-platform:jar (Thread[Execution **** for ':' Thread 4,5,main]) completed. Took 0.237 secs.
:runners:google-cloud-dataflow-java:compileJava (Thread[Execution **** for ':' Thread 4,5,main]) started.
:sdks:java:io:google-cloud-platform:compileTestJava (Thread[Execution **** for ':' Thread 6,5,main]) started.
> Task :runners:google-cloud-dataflow-java:compileJava FROM-CACHE
Custom actions are attached to task ':runners:google-cloud-dataflow-java:compileJava'.
Build cache key for task ':runners:google-cloud-dataflow-java:compileJava' is f4d45dae85b41e621c919183804b6afa
Task ':runners:google-cloud-dataflow-java:compileJava' is not up-to-date because:
No history is available.
Loaded cache entry for task ':runners:google-cloud-dataflow-java:compileJava' with cache key f4d45dae85b41e621c919183804b6afa
:runners:google-cloud-dataflow-java:compileJava (Thread[Execution **** for ':' Thread 4,5,main]) completed. Took 0.205 secs.
:runners:google-cloud-dataflow-java:classes (Thread[Execution **** for ':' Thread 4,5,main]) started.
> Task :runners:google-cloud-dataflow-java:classes
Skipping task ':runners:google-cloud-dataflow-java:classes' as it has no actions.
:runners:google-cloud-dataflow-java:classes (Thread[Execution **** for ':' Thread 4,5,main]) completed. Took 0.0 secs.
:runners:google-cloud-dataflow-java:jar (Thread[Execution **** for ':' Thread 4,5,main]) started.
> Task :runners:google-cloud-dataflow-java:jar
Caching disabled for task ':runners:google-cloud-dataflow-java:jar' because:
Caching has not been enabled for the task
Task ':runners:google-cloud-dataflow-java:jar' is not up-to-date because:
No history is available.
:runners:google-cloud-dataflow-java:jar (Thread[Execution **** for ':' Thread 4,5,main]) completed. Took 0.07 secs.
:runners:google-cloud-dataflow-java:****:legacy-****:compileJava (Thread[Execution **** for ':' Thread 4,5,main]) started.
> Task :sdks:java:io:google-cloud-platform:compileTestJava FROM-CACHE
Custom actions are attached to task ':sdks:java:io:google-cloud-platform:compileTestJava'.
Build cache key for task ':sdks:java:io:google-cloud-platform:compileTestJava' is 99462c52f9bbbb5d949b676d07ee3bc7
Task ':sdks:java:io:google-cloud-platform:compileTestJava' is not up-to-date because:
No history is available.
Loaded cache entry for task ':sdks:java:io:google-cloud-platform:compileTestJava' with cache key 99462c52f9bbbb5d949b676d07ee3bc7
:sdks:java:io:google-cloud-platform:compileTestJava (Thread[Execution **** for ':' Thread 6,5,main]) completed. Took 0.475 secs.
:sdks:java:io:google-cloud-platform:testClasses (Thread[Execution **** for ':' Thread 6,5,main]) started.
> Task :sdks:java:io:google-cloud-platform:testClasses
Skipping task ':sdks:java:io:google-cloud-platform:testClasses' as it has no actions.
:sdks:java:io:google-cloud-platform:testClasses (Thread[Execution **** for ':' Thread 6,5,main]) completed. Took 0.0 secs.
:sdks:java:io:google-cloud-platform:testJar (Thread[Execution **** for ':' Thread 6,5,main]) started.
> Task :runners:google-cloud-dataflow-java:****:legacy-****:compileJava FROM-CACHE
file or directory '<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/runners/google-cloud-dataflow-java/****/legacy-****/src/main/java',> not found
Custom actions are attached to task ':runners:google-cloud-dataflow-java:****:legacy-****:compileJava'.
Build cache key for task ':runners:google-cloud-dataflow-java:****:legacy-****:compileJava' is cf672da70ce14149e9f2f602b34780e4
Task ':runners:google-cloud-dataflow-java:****:legacy-****:compileJava' is not up-to-date because:
No history is available.
Loaded cache entry for task ':runners:google-cloud-dataflow-java:****:legacy-****:compileJava' with cache key cf672da70ce14149e9f2f602b34780e4
:runners:google-cloud-dataflow-java:****:legacy-****:compileJava (Thread[Execution **** for ':' Thread 4,5,main]) completed. Took 0.327 secs.
:runners:google-cloud-dataflow-java:****:legacy-****:classes (Thread[Execution **** for ':' Thread 4,5,main]) started.
> Task :runners:google-cloud-dataflow-java:****:legacy-****:classes UP-TO-DATE
Skipping task ':runners:google-cloud-dataflow-java:****:legacy-****:classes' as it has no actions.
:runners:google-cloud-dataflow-java:****:legacy-****:classes (Thread[Execution **** for ':' Thread 4,5,main]) completed. Took 0.0 secs.
:runners:google-cloud-dataflow-java:****:legacy-****:shadowJar (Thread[Execution **** for ':' Thread 4,5,main]) started.
> Task :sdks:java:io:google-cloud-platform:testJar
Caching disabled for task ':sdks:java:io:google-cloud-platform:testJar' because:
Caching has not been enabled for the task
Task ':sdks:java:io:google-cloud-platform:testJar' is not up-to-date because:
No history is available.
:sdks:java:io:google-cloud-platform:testJar (Thread[Execution **** for ':' Thread 6,5,main]) completed. Took 0.265 secs.
:runners:google-cloud-dataflow-java:compileTestJava (Thread[Execution **** for ':' Thread 6,5,main]) started.
> Task :runners:google-cloud-dataflow-java:****:legacy-****:shadowJar FROM-CACHE
Custom actions are attached to task ':runners:google-cloud-dataflow-java:****:legacy-****:shadowJar'.
Build cache key for task ':runners:google-cloud-dataflow-java:****:legacy-****:shadowJar' is b03e31546baded1569b9e22785f539a5
Task ':runners:google-cloud-dataflow-java:****:legacy-****:shadowJar' is not up-to-date because:
No history is available.
Loaded cache entry for task ':runners:google-cloud-dataflow-java:****:legacy-****:shadowJar' with cache key b03e31546baded1569b9e22785f539a5
:runners:google-cloud-dataflow-java:****:legacy-****:shadowJar (Thread[Execution **** for ':' Thread 4,5,main]) completed. Took 0.174 secs.
> Task :runners:google-cloud-dataflow-java:compileTestJava FROM-CACHE
Custom actions are attached to task ':runners:google-cloud-dataflow-java:compileTestJava'.
Build cache key for task ':runners:google-cloud-dataflow-java:compileTestJava' is 861c7435ffd549666858392e9bbd7747
Task ':runners:google-cloud-dataflow-java:compileTestJava' is not up-to-date because:
No history is available.
Loaded cache entry for task ':runners:google-cloud-dataflow-java:compileTestJava' with cache key 861c7435ffd549666858392e9bbd7747
:runners:google-cloud-dataflow-java:compileTestJava (Thread[Execution **** for ':' Thread 6,5,main]) completed. Took 0.195 secs.
:runners:google-cloud-dataflow-java:testClasses (Thread[Execution **** for ':' Thread 6,5,main]) started.
> Task :runners:google-cloud-dataflow-java:testClasses UP-TO-DATE
Skipping task ':runners:google-cloud-dataflow-java:testClasses' as it has no actions.
:runners:google-cloud-dataflow-java:testClasses (Thread[Execution **** for ':' Thread 6,5,main]) completed. Took 0.0 secs.
:runners:google-cloud-dataflow-java:testJar (Thread[Execution **** for ':' Thread 6,5,main]) started.
> Task :runners:google-cloud-dataflow-java:testJar
Caching disabled for task ':runners:google-cloud-dataflow-java:testJar' because:
Caching has not been enabled for the task
Task ':runners:google-cloud-dataflow-java:testJar' is not up-to-date because:
No history is available.
file or directory '<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/runners/google-cloud-dataflow-java/build/resources/test',> not found
:runners:google-cloud-dataflow-java:testJar (Thread[Execution **** for ':' Thread 6,5,main]) completed. Took 0.03 secs.
:sdks:java:io:file-based-io-tests:integrationTest (Thread[Execution **** for ':' Thread 6,5,main]) started.
Gradle Test Executor 1 started executing tests.
> Task :sdks:java:io:file-based-io-tests:integrationTest
Custom actions are attached to task ':sdks:java:io:file-based-io-tests:integrationTest'.
Build cache key for task ':sdks:java:io:file-based-io-tests:integrationTest' is 2954f27e833e06d9b6038317a4b1cf00
Task ':sdks:java:io:file-based-io-tests:integrationTest' is not up-to-date because:
Task.upToDateWhen is false.
Starting process 'Gradle Test Executor 1'. Working directory: <https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/file-based-io-tests> Command: /usr/lib/jvm/java-8-openjdk-amd64/bin/java -DbeamTestPipelineOptions=["--runner=DataflowRunner","--project=apache-beam-testing","--tempRoot=gs://temp-storage-for-perf-tests","--filenamePrefix=gs://temp-storage-for-perf-tests/beam_PerformanceTests_TFRecordIOIT/5318/","--bigQueryDataset=beam_performance","--bigQueryTable=tfrecordioit_results","--influxMeasurement=tfrecordioit_results","--numberOfRecords=18000000","--expectedHash=543104423f8b6eb097acb9f111c19fe4","--datasetSize=1019380000","--numWorkers=5","--autoscalingAlgorithm=NONE","--influxDatabase=beam_test_metrics","--influxHost=http://10.128.0.96:8086","--****HarnessContainerImage=","--dataflowWorkerJar=<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/runners/google-cloud-dataflow-java/****/legacy-****/build/libs/beam-runners-google-cloud-dataflow-java-legacy-****-2.34.0-SNAPSHOT.jar","--region=us-central1"]> -Djava.security.manager=****.org.gradle.process.internal.****.child.BootstrapSecurityManager -Dorg.gradle.native=false -Xmx2g -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant -ea -cp /home/jenkins/.gradle/caches/6.8.3/****Main/gradle-****.jar ****.org.gradle.process.internal.****.GradleWorkerMain 'Gradle Test Executor 1'
Successfully started process 'Gradle Test Executor 1'
org.apache.beam.sdk.io.tfrecord.TFRecordIOIT STANDARD_ERROR
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/runners/google-cloud-dataflow-java/****/legacy-****/build/libs/beam-runners-google-cloud-dataflow-java-legacy-****-2.34.0-SNAPSHOT.jar!/org/slf4j/impl/StaticLoggerBinder.class]>
SLF4J: Found binding in [jar:file:/home/jenkins/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-log4j12/1.7.30/c21f55139d8141d2231214fb1feaf50a1edca95e/slf4j-log4j12-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/jenkins/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-jdk14/1.7.30/d35953dd2fe54ebe39fdf18cfd82fe6eb35b25ed/slf4j-jdk14-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.JDK14LoggerFactory]
org.apache.beam.sdk.io.tfrecord.TFRecordIOIT > writeThenReadAll STANDARD_ERROR
Sep 13, 2021 12:21:54 PM org.apache.beam.runners.dataflow.DataflowRunner validateSdkContainerImageOptions
WARNING: Prefer --sdkContainerImage over deprecated legacy option --****HarnessContainerImage.
Sep 13, 2021 12:21:55 PM org.apache.beam.sdk.extensions.gcp.options.GcpOptions$GcpTempLocationFactory tryCreateDefaultBucket
INFO: No tempLocation specified, attempting to use default bucket: dataflow-staging-us-central1-844138762903
Sep 13, 2021 12:21:56 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 409, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/storage/v1/b?predefinedAcl=projectPrivate&predefinedDefaultObjectAcl=projectPrivate&project=apache-beam-testing.
Sep 13, 2021 12:21:56 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Sep 13, 2021 12:21:56 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 317 files. Enable logging at DEBUG level to see which files will be staged.
Sep 13, 2021 12:21:56 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Sep 13, 2021 12:21:59 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 318 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Sep 13, 2021 12:21:59 PM org.apache.beam.runners.dataflow.util.PackageUtil$PackageAttributes forFileToStage
INFO: Staging custom dataflow-****.jar as beam-runners-google-cloud-dataflow-java-legacy-****-2.34.0-SNAPSHOT-YtvkOIsfQYfFzG89aVBO_8oNHoiij9SynwUJLdMkn5A.jar
Sep 13, 2021 12:21:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /tmp/test7812669155832243219.zip to gs://dataflow-staging-us-central1-844138762903/temp/staging/test-nL1rtRtLm90VKwPJkj_v7x1yOXeqaHoz2ine0xdRZ-4.jar
Sep 13, 2021 12:22:00 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 317 files cached, 1 files newly uploaded in 1 seconds
Sep 13, 2021 12:22:00 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://dataflow-staging-us-central1-844138762903/temp/staging/
Sep 13, 2021 12:22:00 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <205290 bytes, hash 657a029cba0782f3eaba8cb9b2fec74a43cea4fbd705f64abcbd9660d3414855> to gs://dataflow-staging-us-central1-844138762903/temp/staging/pipeline-ZXoCnLoHgvPquoy5sv7HSkPOpPvXBfZKvL2WYNNBSFU.pb
Sep 13, 2021 12:22:02 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Generate sequence/Read(BoundedCountingSource) as step s1
Sep 13, 2021 12:22:02 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Produce text lines as step s2
Sep 13, 2021 12:22:02 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Transform strings to bytes/Map as step s3
Sep 13, 2021 12:22:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Record time before writing as step s4
Sep 13, 2021 12:22:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to files/Write/RewindowIntoGlobal/Window.Assign as step s5
Sep 13, 2021 12:22:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to files/Write/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles as step s6
Sep 13, 2021 12:22:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten as step s7
Sep 13, 2021 12:22:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to files/Write/WriteUnshardedBundlesToTempFiles/WriteUnwritten as step s8
Sep 13, 2021 12:22:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to files/Write/WriteUnshardedBundlesToTempFiles/DropShardNum as step s9
Sep 13, 2021 12:22:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to files/Write/WriteUnshardedBundlesToTempFiles/Flatten.PCollections as step s10
Sep 13, 2021 12:22:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to files/Write/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow) as step s11
Sep 13, 2021 12:22:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to files/Write/GatherTempFileResults/View.AsList/CreateDataflowView as step s12
Sep 13, 2021 12:22:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Create.Values/Read(CreateSource) as step s13
Sep 13, 2021 12:22:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Reify.ReifyView/ParDo(Anonymous) as step s14
Sep 13, 2021 12:22:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Values/Values/Map as step s15
Sep 13, 2021 12:22:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Finalize as step s16
Sep 13, 2021 12:22:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random key as step s17
Sep 13, 2021 12:22:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign as step s18
Sep 13, 2021 12:22:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey as step s19
Sep 13, 2021 12:22:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable as step s20
Sep 13, 2021 12:22:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Values/Values/Map as step s21
Sep 13, 2021 12:22:03 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.34.0-SNAPSHOT
Sep 13, 2021 12:22:04 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-09-13_05_22_03-14479944029719028474?project=apache-beam-testing
Sep 13, 2021 12:22:04 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-09-13_05_22_03-14479944029719028474
Sep 13, 2021 12:22:04 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-09-13_05_22_03-14479944029719028474
Sep 13, 2021 12:22:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-13T12:22:26.838Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Sep 13, 2021 12:22:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-13T12:22:28.228Z: Worker configuration: e2-standard-2 in us-central1-c.
Sep 13, 2021 12:22:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-13T12:22:28.936Z: Workflow failed. Causes: Project apache-beam-testing has insufficient quota(s) to execute this workflow with 1 instances in region us-central1. Quota summary (required/available): 1/24386 instances, 2/0 CPUs, 25/183716 disk GB, 0/2397 SSD disk GB, 1/286 instance groups, 1/289 managed instance groups, 1/515 instance templates, 1/615 in-use IP addresses.
Please see https://cloud.google.com/compute/docs/resource-quotas about requesting more quota.
Sep 13, 2021 12:22:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-13T12:22:28.984Z: Cleaning up.
Sep 13, 2021 12:22:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-13T12:22:29.041Z: Worker pool stopped.
Sep 13, 2021 12:22:36 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-09-13_05_22_03-14479944029719028474 failed with status FAILED.
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:io:file-based-io-tests:integrationTest FAILED
org.apache.beam.sdk.io.tfrecord.TFRecordIOIT > writeThenReadAll FAILED
java.lang.IllegalStateException: Unable to find any files matching gs://temp-storage-for-perf-tests/beam_PerformanceTests_TFRecordIOIT/5318/_1631535713235*
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:588)
at org.apache.beam.sdk.io.TFRecordIO$Read.expand(TFRecordIO.java:191)
at org.apache.beam.sdk.io.TFRecordIO$Read.expand(TFRecordIO.java:112)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:548)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:482)
at org.apache.beam.sdk.values.PBegin.apply(PBegin.java:44)
at org.apache.beam.sdk.Pipeline.apply(Pipeline.java:177)
at org.apache.beam.sdk.io.tfrecord.TFRecordIOIT.writeThenReadAll(TFRecordIOIT.java:151)
1 test completed, 1 failed
Finished generating test XML results (0.021 secs) into: <https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/file-based-io-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.032 secs) into: <https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/file-based-io-tests/build/reports/tests/integrationTest>
:sdks:java:io:file-based-io-tests:integrationTest (Thread[Execution **** for ':' Thread 6,5,main]) completed. Took 46.847 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:io:file-based-io-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/file-based-io-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1m 17s
129 actionable tasks: 79 executed, 50 from cache
Publishing build scan...
https://gradle.com/s/cg474npib7pok
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org