You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@beam.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2021/09/09 18:22:22 UTC
Build failed in Jenkins: beam_PerformanceTests_TFRecordIOIT #5303
See <https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/5303/display/redirect?page=changes>
Changes:
[kawaigin] [BEAM-10708] Support streaming cache in beam_sql magic
------------------------------------------
[...truncated 272.93 KB...]
Task ':sdks:java:io:google-cloud-platform:jar' is not up-to-date because:
No history is available.
file or directory '<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/google-cloud-platform/build/resources/main',> not found
:sdks:java:io:google-cloud-platform:jar (Thread[Execution **** for ':' Thread 6,5,main]) completed. Took 0.215 secs.
:runners:google-cloud-dataflow-java:compileJava (Thread[Execution **** for ':' Thread 6,5,main]) started.
:sdks:java:io:google-cloud-platform:compileTestJava (Thread[Execution **** for ':' Thread 3,5,main]) started.
> Task :runners:google-cloud-dataflow-java:compileJava FROM-CACHE
Custom actions are attached to task ':runners:google-cloud-dataflow-java:compileJava'.
Build cache key for task ':runners:google-cloud-dataflow-java:compileJava' is 1dc0ba779f805737f17b2d73fbfcd08c
Task ':runners:google-cloud-dataflow-java:compileJava' is not up-to-date because:
No history is available.
Loaded cache entry for task ':runners:google-cloud-dataflow-java:compileJava' with cache key 1dc0ba779f805737f17b2d73fbfcd08c
:runners:google-cloud-dataflow-java:compileJava (Thread[Execution **** for ':' Thread 6,5,main]) completed. Took 0.22 secs.
:runners:google-cloud-dataflow-java:classes (Thread[Execution **** for ':' Thread 6,5,main]) started.
> Task :runners:google-cloud-dataflow-java:classes
Skipping task ':runners:google-cloud-dataflow-java:classes' as it has no actions.
:runners:google-cloud-dataflow-java:classes (Thread[Execution **** for ':' Thread 6,5,main]) completed. Took 0.0 secs.
:runners:google-cloud-dataflow-java:jar (Thread[Execution **** for ':' Thread 6,5,main]) started.
> Task :runners:google-cloud-dataflow-java:jar
Caching disabled for task ':runners:google-cloud-dataflow-java:jar' because:
Caching has not been enabled for the task
Task ':runners:google-cloud-dataflow-java:jar' is not up-to-date because:
No history is available.
:runners:google-cloud-dataflow-java:jar (Thread[Execution **** for ':' Thread 6,5,main]) completed. Took 0.068 secs.
:runners:google-cloud-dataflow-java:****:legacy-****:compileJava (Thread[Execution **** for ':' Thread 6,5,main]) started.
> Task :sdks:java:io:google-cloud-platform:compileTestJava FROM-CACHE
Custom actions are attached to task ':sdks:java:io:google-cloud-platform:compileTestJava'.
Build cache key for task ':sdks:java:io:google-cloud-platform:compileTestJava' is 9c662d258223b57c41342cf5606bdf52
Task ':sdks:java:io:google-cloud-platform:compileTestJava' is not up-to-date because:
No history is available.
Loaded cache entry for task ':sdks:java:io:google-cloud-platform:compileTestJava' with cache key 9c662d258223b57c41342cf5606bdf52
:sdks:java:io:google-cloud-platform:compileTestJava (Thread[Execution **** for ':' Thread 3,5,main]) completed. Took 0.468 secs.
:sdks:java:io:google-cloud-platform:testClasses (Thread[Execution **** for ':' Thread 3,5,main]) started.
> Task :sdks:java:io:google-cloud-platform:testClasses
Skipping task ':sdks:java:io:google-cloud-platform:testClasses' as it has no actions.
:sdks:java:io:google-cloud-platform:testClasses (Thread[Execution **** for ':' Thread 3,5,main]) completed. Took 0.0 secs.
:sdks:java:io:google-cloud-platform:testJar (Thread[Execution **** for ':' Thread 3,5,main]) started.
> Task :runners:google-cloud-dataflow-java:****:legacy-****:compileJava FROM-CACHE
file or directory '<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/runners/google-cloud-dataflow-java/****/legacy-****/src/main/java',> not found
Custom actions are attached to task ':runners:google-cloud-dataflow-java:****:legacy-****:compileJava'.
Build cache key for task ':runners:google-cloud-dataflow-java:****:legacy-****:compileJava' is 0b8acac276dc48250a47545b5fb0d3ce
Task ':runners:google-cloud-dataflow-java:****:legacy-****:compileJava' is not up-to-date because:
No history is available.
Loaded cache entry for task ':runners:google-cloud-dataflow-java:****:legacy-****:compileJava' with cache key 0b8acac276dc48250a47545b5fb0d3ce
:runners:google-cloud-dataflow-java:****:legacy-****:compileJava (Thread[Execution **** for ':' Thread 6,5,main]) completed. Took 0.291 secs.
:runners:google-cloud-dataflow-java:****:legacy-****:classes (Thread[Execution **** for ':' Thread 6,5,main]) started.
> Task :runners:google-cloud-dataflow-java:****:legacy-****:classes UP-TO-DATE
Skipping task ':runners:google-cloud-dataflow-java:****:legacy-****:classes' as it has no actions.
:runners:google-cloud-dataflow-java:****:legacy-****:classes (Thread[Execution **** for ':' Thread 6,5,main]) completed. Took 0.0 secs.
:runners:google-cloud-dataflow-java:****:legacy-****:shadowJar (Thread[Execution **** for ':' Thread 6,5,main]) started.
> Task :sdks:java:io:google-cloud-platform:testJar
Caching disabled for task ':sdks:java:io:google-cloud-platform:testJar' because:
Caching has not been enabled for the task
Task ':sdks:java:io:google-cloud-platform:testJar' is not up-to-date because:
No history is available.
:sdks:java:io:google-cloud-platform:testJar (Thread[Execution **** for ':' Thread 3,5,main]) completed. Took 0.264 secs.
:runners:google-cloud-dataflow-java:compileTestJava (Thread[Execution **** for ':' Thread 3,5,main]) started.
> Task :runners:google-cloud-dataflow-java:****:legacy-****:shadowJar FROM-CACHE
Custom actions are attached to task ':runners:google-cloud-dataflow-java:****:legacy-****:shadowJar'.
Build cache key for task ':runners:google-cloud-dataflow-java:****:legacy-****:shadowJar' is 85bdb746551041d5fa13afb38e58a5d6
Task ':runners:google-cloud-dataflow-java:****:legacy-****:shadowJar' is not up-to-date because:
No history is available.
Loaded cache entry for task ':runners:google-cloud-dataflow-java:****:legacy-****:shadowJar' with cache key 85bdb746551041d5fa13afb38e58a5d6
:runners:google-cloud-dataflow-java:****:legacy-****:shadowJar (Thread[Execution **** for ':' Thread 6,5,main]) completed. Took 0.166 secs.
> Task :runners:google-cloud-dataflow-java:compileTestJava FROM-CACHE
Custom actions are attached to task ':runners:google-cloud-dataflow-java:compileTestJava'.
Build cache key for task ':runners:google-cloud-dataflow-java:compileTestJava' is ff78c35185f8eafcb177b218837ed92d
Task ':runners:google-cloud-dataflow-java:compileTestJava' is not up-to-date because:
No history is available.
Loaded cache entry for task ':runners:google-cloud-dataflow-java:compileTestJava' with cache key ff78c35185f8eafcb177b218837ed92d
:runners:google-cloud-dataflow-java:compileTestJava (Thread[Execution **** for ':' Thread 3,5,main]) completed. Took 0.188 secs.
:runners:google-cloud-dataflow-java:testClasses (Thread[Execution **** for ':' Thread 3,5,main]) started.
> Task :runners:google-cloud-dataflow-java:testClasses UP-TO-DATE
Skipping task ':runners:google-cloud-dataflow-java:testClasses' as it has no actions.
:runners:google-cloud-dataflow-java:testClasses (Thread[Execution **** for ':' Thread 3,5,main]) completed. Took 0.0 secs.
:runners:google-cloud-dataflow-java:testJar (Thread[Execution **** for ':' Thread 3,5,main]) started.
> Task :runners:google-cloud-dataflow-java:testJar
Caching disabled for task ':runners:google-cloud-dataflow-java:testJar' because:
Caching has not been enabled for the task
Task ':runners:google-cloud-dataflow-java:testJar' is not up-to-date because:
No history is available.
file or directory '<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/runners/google-cloud-dataflow-java/build/resources/test',> not found
:runners:google-cloud-dataflow-java:testJar (Thread[Execution **** for ':' Thread 3,5,main]) completed. Took 0.029 secs.
:sdks:java:io:file-based-io-tests:integrationTest (Thread[Execution **** for ':' Thread 3,5,main]) started.
Gradle Test Executor 6 started executing tests.
> Task :sdks:java:io:file-based-io-tests:integrationTest
Custom actions are attached to task ':sdks:java:io:file-based-io-tests:integrationTest'.
Build cache key for task ':sdks:java:io:file-based-io-tests:integrationTest' is 930d2c7090329d0ec3e4ab63b1a0071b
Task ':sdks:java:io:file-based-io-tests:integrationTest' is not up-to-date because:
Task.upToDateWhen is false.
Starting process 'Gradle Test Executor 6'. Working directory: <https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/file-based-io-tests> Command: /usr/lib/jvm/java-8-openjdk-amd64/bin/java -DbeamTestPipelineOptions=["--runner=DataflowRunner","--project=apache-beam-testing","--tempRoot=gs://temp-storage-for-perf-tests","--filenamePrefix=gs://temp-storage-for-perf-tests/beam_PerformanceTests_TFRecordIOIT/5303/","--bigQueryDataset=beam_performance","--bigQueryTable=tfrecordioit_results","--influxMeasurement=tfrecordioit_results","--numberOfRecords=18000000","--expectedHash=543104423f8b6eb097acb9f111c19fe4","--datasetSize=1019380000","--numWorkers=5","--autoscalingAlgorithm=NONE","--influxDatabase=beam_test_metrics","--influxHost=http://10.128.0.96:8086","--****HarnessContainerImage=","--dataflowWorkerJar=<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/runners/google-cloud-dataflow-java/****/legacy-****/build/libs/beam-runners-google-cloud-dataflow-java-legacy-****-2.34.0-SNAPSHOT.jar","--region=us-central1"]> -Djava.security.manager=****.org.gradle.process.internal.****.child.BootstrapSecurityManager -Dorg.gradle.native=false -Xmx2g -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant -ea -cp /home/jenkins/.gradle/caches/6.8.3/****Main/gradle-****.jar ****.org.gradle.process.internal.****.GradleWorkerMain 'Gradle Test Executor 6'
Successfully started process 'Gradle Test Executor 6'
org.apache.beam.sdk.io.tfrecord.TFRecordIOIT STANDARD_ERROR
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/runners/google-cloud-dataflow-java/****/legacy-****/build/libs/beam-runners-google-cloud-dataflow-java-legacy-****-2.34.0-SNAPSHOT.jar!/org/slf4j/impl/StaticLoggerBinder.class]>
SLF4J: Found binding in [jar:file:/home/jenkins/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-log4j12/1.7.30/c21f55139d8141d2231214fb1feaf50a1edca95e/slf4j-log4j12-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/jenkins/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-jdk14/1.7.30/d35953dd2fe54ebe39fdf18cfd82fe6eb35b25ed/slf4j-jdk14-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.JDK14LoggerFactory]
org.apache.beam.sdk.io.tfrecord.TFRecordIOIT > writeThenReadAll STANDARD_ERROR
Sep 09, 2021 6:21:40 PM org.apache.beam.runners.dataflow.DataflowRunner validateSdkContainerImageOptions
WARNING: Prefer --sdkContainerImage over deprecated legacy option --****HarnessContainerImage.
Sep 09, 2021 6:21:41 PM org.apache.beam.sdk.extensions.gcp.options.GcpOptions$GcpTempLocationFactory tryCreateDefaultBucket
INFO: No tempLocation specified, attempting to use default bucket: dataflow-staging-us-central1-844138762903
Sep 09, 2021 6:21:41 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 409, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/storage/v1/b?predefinedAcl=projectPrivate&predefinedDefaultObjectAcl=projectPrivate&project=apache-beam-testing.
Sep 09, 2021 6:21:41 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Sep 09, 2021 6:21:42 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 317 files. Enable logging at DEBUG level to see which files will be staged.
Sep 09, 2021 6:21:42 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Sep 09, 2021 6:21:45 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 318 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Sep 09, 2021 6:21:45 PM org.apache.beam.runners.dataflow.util.PackageUtil$PackageAttributes forFileToStage
INFO: Staging custom dataflow-****.jar as beam-runners-google-cloud-dataflow-java-legacy-****-2.34.0-SNAPSHOT-E-KPx8odQzZ4kFtSdYrJ4EpS3UhAdfKiSlspRWHWbTM.jar
Sep 09, 2021 6:21:45 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /tmp/test1742249712797023910.zip to gs://dataflow-staging-us-central1-844138762903/temp/staging/test-JOVfLnIhovkSRUNBP-Go-aBRDyWmfvW6fa-RZwWRI7Y.jar
Sep 09, 2021 6:21:46 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 317 files cached, 1 files newly uploaded in 0 seconds
Sep 09, 2021 6:21:46 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://dataflow-staging-us-central1-844138762903/temp/staging/
Sep 09, 2021 6:21:46 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <205296 bytes, hash 3fb42937ff5eef649337185091dcf73d5469c2eff220443dacb328c8e5c84025> to gs://dataflow-staging-us-central1-844138762903/temp/staging/pipeline-P7QpN_9e72STNxhQkdz3PVRpwu_yIEQ9rLMoyOXIQCU.pb
Sep 09, 2021 6:21:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Generate sequence/Read(BoundedCountingSource) as step s1
Sep 09, 2021 6:21:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Produce text lines as step s2
Sep 09, 2021 6:21:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Transform strings to bytes/Map as step s3
Sep 09, 2021 6:21:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Record time before writing as step s4
Sep 09, 2021 6:21:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to files/Write/RewindowIntoGlobal/Window.Assign as step s5
Sep 09, 2021 6:21:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to files/Write/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles as step s6
Sep 09, 2021 6:21:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten as step s7
Sep 09, 2021 6:21:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to files/Write/WriteUnshardedBundlesToTempFiles/WriteUnwritten as step s8
Sep 09, 2021 6:21:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to files/Write/WriteUnshardedBundlesToTempFiles/DropShardNum as step s9
Sep 09, 2021 6:21:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to files/Write/WriteUnshardedBundlesToTempFiles/Flatten.PCollections as step s10
Sep 09, 2021 6:21:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to files/Write/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow) as step s11
Sep 09, 2021 6:21:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to files/Write/GatherTempFileResults/View.AsList/CreateDataflowView as step s12
Sep 09, 2021 6:21:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Create.Values/Read(CreateSource) as step s13
Sep 09, 2021 6:21:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Reify.ReifyView/ParDo(Anonymous) as step s14
Sep 09, 2021 6:21:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Values/Values/Map as step s15
Sep 09, 2021 6:21:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Finalize as step s16
Sep 09, 2021 6:21:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random key as step s17
Sep 09, 2021 6:21:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign as step s18
Sep 09, 2021 6:21:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey as step s19
Sep 09, 2021 6:21:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable as step s20
Sep 09, 2021 6:21:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Values/Values/Map as step s21
Sep 09, 2021 6:21:48 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.34.0-SNAPSHOT
Sep 09, 2021 6:21:49 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-09-09_11_21_48-13083074022768655218?project=apache-beam-testing
Sep 09, 2021 6:21:49 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-09-09_11_21_48-13083074022768655218
Sep 09, 2021 6:21:49 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-09-09_11_21_48-13083074022768655218
Sep 09, 2021 6:22:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-09T18:22:14.320Z: Worker configuration: e2-standard-2 in us-central1-c.
Sep 09, 2021 6:22:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-09T18:22:14.458Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Sep 09, 2021 6:22:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-09T18:22:15.088Z: Workflow failed. Causes: Project apache-beam-testing has insufficient quota(s) to execute this workflow with 1 instances in region us-central1. Quota summary (required/available): 1/19492 instances, 2/0 CPUs, 25/245351 disk GB, 0/2397 SSD disk GB, 1/254 instance groups, 1/258 managed instance groups, 1/482 instance templates, 1/721 in-use IP addresses.
Please see https://cloud.google.com/compute/docs/resource-quotas about requesting more quota.
Sep 09, 2021 6:22:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-09T18:22:15.142Z: Cleaning up.
Sep 09, 2021 6:22:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-09T18:22:15.234Z: Worker pool stopped.
Sep 09, 2021 6:22:21 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-09-09_11_21_48-13083074022768655218 failed with status FAILED.
Gradle Test Executor 6 finished executing tests.
> Task :sdks:java:io:file-based-io-tests:integrationTest FAILED
org.apache.beam.sdk.io.tfrecord.TFRecordIOIT > writeThenReadAll FAILED
java.lang.IllegalStateException: Unable to find any files matching gs://temp-storage-for-perf-tests/beam_PerformanceTests_TFRecordIOIT/5303/_1631211699215*
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:588)
at org.apache.beam.sdk.io.TFRecordIO$Read.expand(TFRecordIO.java:191)
at org.apache.beam.sdk.io.TFRecordIO$Read.expand(TFRecordIO.java:112)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:548)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:482)
at org.apache.beam.sdk.values.PBegin.apply(PBegin.java:44)
at org.apache.beam.sdk.Pipeline.apply(Pipeline.java:177)
at org.apache.beam.sdk.io.tfrecord.TFRecordIOIT.writeThenReadAll(TFRecordIOIT.java:151)
1 test completed, 1 failed
Finished generating test XML results (0.001 secs) into: <https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/file-based-io-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.004 secs) into: <https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/file-based-io-tests/build/reports/tests/integrationTest>
:sdks:java:io:file-based-io-tests:integrationTest (Thread[Execution **** for ':' Thread 3,5,main]) completed. Took 46.509 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:io:file-based-io-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/file-based-io-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1m 2s
129 actionable tasks: 79 executed, 50 from cache
Publishing build scan...
https://gradle.com/s/yo3p57j6fffy6
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Jenkins build is back to normal :
beam_PerformanceTests_TFRecordIOIT #5305
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/5305/display/redirect?page=changes>
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PerformanceTests_TFRecordIOIT #5304
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/5304/display/redirect?page=changes>
Changes:
[ruwan.lambrichts] Clarify additional_bq_parameters argument
[noreply] Fix broken 'differences from pandas' link
[noreply] Added GroupBy row in Aggregation table.
[Luke Cwik] [BEAM-12769] Fix typo in test class name, CLass -> Class
[Etienne Chauchot] [BEAM-5172] Temporary ignore testSplit and testSizes tests waiting for a
[samuelw] [BEAM-12740] Remove matching to filter files when renaming gcs files in
[noreply] [BEAM-3304] Helper functions for triggers (#15430)
[esert] Bump a throttling counter on BigQueryRead retries due to
------------------------------------------
[...truncated 287.37 KB...]
Note: Recompile with -Xlint:deprecation for details.
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
Created classpath snapshot for incremental compilation in 0.079 secs. 1 duplicate classes found in classpath (see all with --debug).
Stored cache entry for task ':runners:google-cloud-dataflow-java:compileJava' with cache key babebdf25338a9a0b9a7b1271fa39c1e
:runners:google-cloud-dataflow-java:compileJava (Thread[Execution **** for ':' Thread 10,5,main]) completed. Took 45.043 secs.
:runners:google-cloud-dataflow-java:classes (Thread[Execution **** for ':' Thread 10,5,main]) started.
> Task :runners:google-cloud-dataflow-java:classes
Skipping task ':runners:google-cloud-dataflow-java:classes' as it has no actions.
:runners:google-cloud-dataflow-java:classes (Thread[Execution **** for ':' Thread 10,5,main]) completed. Took 0.0 secs.
:runners:google-cloud-dataflow-java:jar (Thread[Execution **** for ':' Thread 10,5,main]) started.
> Task :runners:google-cloud-dataflow-java:jar
Caching disabled for task ':runners:google-cloud-dataflow-java:jar' because:
Caching has not been enabled for the task
Task ':runners:google-cloud-dataflow-java:jar' is not up-to-date because:
No history is available.
:runners:google-cloud-dataflow-java:jar (Thread[Execution **** for ':' Thread 10,5,main]) completed. Took 0.061 secs.
:runners:google-cloud-dataflow-java:compileTestJava (Thread[Execution **** for ':' Thread 10,5,main]) started.
:runners:google-cloud-dataflow-java:****:legacy-****:compileJava (Thread[Execution **** for ':' Thread 5,5,main]) started.
> Task :runners:google-cloud-dataflow-java:compileTestJava
Custom actions are attached to task ':runners:google-cloud-dataflow-java:compileTestJava'.
Build cache key for task ':runners:google-cloud-dataflow-java:compileTestJava' is b2c134fa0c93b77f00e481a5f40047c0
Task ':runners:google-cloud-dataflow-java:compileTestJava' is not up-to-date because:
No history is available.
The input changes require a full rebuild for incremental task ':runners:google-cloud-dataflow-java:compileTestJava'.
Full recompilation is required because no incremental change information is available. This is usually caused by clean builds or changing compiler arguments.
Compiling with JDK Java compiler API.
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
Created classpath snapshot for incremental compilation in 0.719 secs. 3298 duplicate classes found in classpath (see all with --debug).
Stored cache entry for task ':runners:google-cloud-dataflow-java:compileTestJava' with cache key b2c134fa0c93b77f00e481a5f40047c0
:runners:google-cloud-dataflow-java:compileTestJava (Thread[Execution **** for ':' Thread 10,5,main]) completed. Took 7.091 secs.
:runners:google-cloud-dataflow-java:testClasses (Thread[Execution **** for ':' Thread 6,5,main]) started.
> Task :runners:google-cloud-dataflow-java:testClasses
Skipping task ':runners:google-cloud-dataflow-java:testClasses' as it has no actions.
:runners:google-cloud-dataflow-java:testClasses (Thread[Execution **** for ':' Thread 6,5,main]) completed. Took 0.0 secs.
:runners:google-cloud-dataflow-java:testJar (Thread[Execution **** for ':' Thread 6,5,main]) started.
> Task :runners:google-cloud-dataflow-java:****:legacy-****:compileJava
file or directory '<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/runners/google-cloud-dataflow-java/****/legacy-****/src/main/java',> not found
Custom actions are attached to task ':runners:google-cloud-dataflow-java:****:legacy-****:compileJava'.
Build cache key for task ':runners:google-cloud-dataflow-java:****:legacy-****:compileJava' is 9052e79cb7369abc5957ba3d2ca618e7
Task ':runners:google-cloud-dataflow-java:****:legacy-****:compileJava' is not up-to-date because:
No history is available.
The input changes require a full rebuild for incremental task ':runners:google-cloud-dataflow-java:****:legacy-****:compileJava'.
Full recompilation is required because no incremental change information is available. This is usually caused by clean builds or changing compiler arguments.
file or directory '<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/runners/google-cloud-dataflow-java/****/legacy-****/src/main/java',> not found
Compiling with JDK Java compiler API.
> Task :runners:google-cloud-dataflow-java:testJar
Caching disabled for task ':runners:google-cloud-dataflow-java:testJar' because:
Caching has not been enabled for the task
Task ':runners:google-cloud-dataflow-java:testJar' is not up-to-date because:
No history is available.
file or directory '<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/runners/google-cloud-dataflow-java/build/resources/test',> not found
:runners:google-cloud-dataflow-java:testJar (Thread[Execution **** for ':' Thread 6,5,main]) completed. Took 0.039 secs.
> Task :runners:google-cloud-dataflow-java:****:legacy-****:compileJava
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
Created classpath snapshot for incremental compilation in 0.139 secs. 22 duplicate classes found in classpath (see all with --debug).
Stored cache entry for task ':runners:google-cloud-dataflow-java:****:legacy-****:compileJava' with cache key 9052e79cb7369abc5957ba3d2ca618e7
:runners:google-cloud-dataflow-java:****:legacy-****:compileJava (Thread[Execution **** for ':' Thread 5,5,main]) completed. Took 1 mins 51.929 secs.
:runners:google-cloud-dataflow-java:****:legacy-****:classes (Thread[Execution **** for ':' Thread 5,5,main]) started.
> Task :runners:google-cloud-dataflow-java:****:legacy-****:classes
Skipping task ':runners:google-cloud-dataflow-java:****:legacy-****:classes' as it has no actions.
:runners:google-cloud-dataflow-java:****:legacy-****:classes (Thread[Execution **** for ':' Thread 5,5,main]) completed. Took 0.0 secs.
:runners:google-cloud-dataflow-java:****:legacy-****:shadowJar (Thread[Execution **** for ':' Thread 5,5,main]) started.
> Task :runners:google-cloud-dataflow-java:****:legacy-****:shadowJar
Custom actions are attached to task ':runners:google-cloud-dataflow-java:****:legacy-****:shadowJar'.
Build cache key for task ':runners:google-cloud-dataflow-java:****:legacy-****:shadowJar' is d05b6d09844c2585e23428184cbcc793
Task ':runners:google-cloud-dataflow-java:****:legacy-****:shadowJar' is not up-to-date because:
No history is available.
file or directory '<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/runners/google-cloud-dataflow-java/****/legacy-****/build/resources/main',> not found
file or directory '<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/runners/google-cloud-dataflow-java/****/legacy-****/build/original_sources_to_package',> not found
*******************
GRADLE SHADOW STATS
Total Jars: 15 (includes project)
Total Time: 2.899s [2899ms]
Average Time/Jar: 0.19326666666669998s [193.2666666667ms]
*******************
Stored cache entry for task ':runners:google-cloud-dataflow-java:****:legacy-****:shadowJar' with cache key d05b6d09844c2585e23428184cbcc793
:runners:google-cloud-dataflow-java:****:legacy-****:shadowJar (Thread[Execution **** for ':' Thread 5,5,main]) completed. Took 4.34 secs.
:sdks:java:io:file-based-io-tests:integrationTest (Thread[Execution **** for ':' Thread 5,5,main]) started.
Gradle Test Executor 44 started executing tests.
> Task :sdks:java:io:file-based-io-tests:integrationTest
Custom actions are attached to task ':sdks:java:io:file-based-io-tests:integrationTest'.
Build cache key for task ':sdks:java:io:file-based-io-tests:integrationTest' is 0084fd255ce81320627de80dd8f07f80
Task ':sdks:java:io:file-based-io-tests:integrationTest' is not up-to-date because:
Task.upToDateWhen is false.
Starting process 'Gradle Test Executor 44'. Working directory: <https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/file-based-io-tests> Command: /usr/lib/jvm/java-8-openjdk-amd64/bin/java -DbeamTestPipelineOptions=["--runner=DataflowRunner","--project=apache-beam-testing","--tempRoot=gs://temp-storage-for-perf-tests","--filenamePrefix=gs://temp-storage-for-perf-tests/beam_PerformanceTests_TFRecordIOIT/5304/","--bigQueryDataset=beam_performance","--bigQueryTable=tfrecordioit_results","--influxMeasurement=tfrecordioit_results","--numberOfRecords=18000000","--expectedHash=543104423f8b6eb097acb9f111c19fe4","--datasetSize=1019380000","--numWorkers=5","--autoscalingAlgorithm=NONE","--influxDatabase=beam_test_metrics","--influxHost=http://10.128.0.96:8086","--****HarnessContainerImage=","--dataflowWorkerJar=<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/runners/google-cloud-dataflow-java/****/legacy-****/build/libs/beam-runners-google-cloud-dataflow-java-legacy-****-2.34.0-SNAPSHOT.jar","--region=us-central1"]> -Djava.security.manager=****.org.gradle.process.internal.****.child.BootstrapSecurityManager -Dorg.gradle.native=false -Xmx2g -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant -ea -cp /home/jenkins/.gradle/caches/6.8.3/****Main/gradle-****.jar ****.org.gradle.process.internal.****.GradleWorkerMain 'Gradle Test Executor 44'
Successfully started process 'Gradle Test Executor 44'
org.apache.beam.sdk.io.tfrecord.TFRecordIOIT STANDARD_ERROR
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/runners/google-cloud-dataflow-java/****/legacy-****/build/libs/beam-runners-google-cloud-dataflow-java-legacy-****-2.34.0-SNAPSHOT.jar!/org/slf4j/impl/StaticLoggerBinder.class]>
SLF4J: Found binding in [jar:file:/home/jenkins/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-log4j12/1.7.30/c21f55139d8141d2231214fb1feaf50a1edca95e/slf4j-log4j12-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/jenkins/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-jdk14/1.7.30/d35953dd2fe54ebe39fdf18cfd82fe6eb35b25ed/slf4j-jdk14-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.JDK14LoggerFactory]
org.apache.beam.sdk.io.tfrecord.TFRecordIOIT > writeThenReadAll STANDARD_ERROR
Sep 10, 2021 12:35:15 AM org.apache.beam.runners.dataflow.DataflowRunner validateSdkContainerImageOptions
WARNING: Prefer --sdkContainerImage over deprecated legacy option --****HarnessContainerImage.
Sep 10, 2021 12:35:16 AM org.apache.beam.sdk.extensions.gcp.options.GcpOptions$GcpTempLocationFactory tryCreateDefaultBucket
INFO: No tempLocation specified, attempting to use default bucket: dataflow-staging-us-central1-844138762903
Sep 10, 2021 12:35:16 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 409, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/storage/v1/b?predefinedAcl=projectPrivate&predefinedDefaultObjectAcl=projectPrivate&project=apache-beam-testing.
Sep 10, 2021 12:35:16 AM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Sep 10, 2021 12:35:17 AM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 317 files. Enable logging at DEBUG level to see which files will be staged.
Sep 10, 2021 12:35:17 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
Sep 10, 2021 12:35:22 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Uploading 318 files from PipelineOptions.filesToStage to staging location to prepare for execution.
Sep 10, 2021 12:35:22 AM org.apache.beam.runners.dataflow.util.PackageUtil$PackageAttributes forFileToStage
INFO: Staging custom dataflow-****.jar as beam-runners-google-cloud-dataflow-java-legacy-****-2.34.0-SNAPSHOT-ucAidJrqxK96xuFPkyoKLCyoB4CvF-dHe3bPkncTQpU.jar
Sep 10, 2021 12:35:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading /tmp/test997112994967036392.zip to gs://dataflow-staging-us-central1-844138762903/temp/staging/test-5odTVE6jnYbQiXHBk9P8SfUR8YoSl_jClQMSUWRxiDA.jar
Sep 10, 2021 12:35:23 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 317 files cached, 1 files newly uploaded in 0 seconds
Sep 10, 2021 12:35:23 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to gs://dataflow-staging-us-central1-844138762903/temp/staging/
Sep 10, 2021 12:35:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <205296 bytes, hash 39ae7b360cc35ce4017fc41be72f937132eb0b421a7dc550e9d46270611410a5> to gs://dataflow-staging-us-central1-844138762903/temp/staging/pipeline-Oa57NgzDXOQBf8Qb5y-TcTLrC0IafcVQ6dRicGEUEKU.pb
Sep 10, 2021 12:35:25 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Generate sequence/Read(BoundedCountingSource) as step s1
Sep 10, 2021 12:35:25 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Produce text lines as step s2
Sep 10, 2021 12:35:25 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Transform strings to bytes/Map as step s3
Sep 10, 2021 12:35:25 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Record time before writing as step s4
Sep 10, 2021 12:35:25 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to files/Write/RewindowIntoGlobal/Window.Assign as step s5
Sep 10, 2021 12:35:25 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to files/Write/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles as step s6
Sep 10, 2021 12:35:25 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten as step s7
Sep 10, 2021 12:35:25 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to files/Write/WriteUnshardedBundlesToTempFiles/WriteUnwritten as step s8
Sep 10, 2021 12:35:25 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to files/Write/WriteUnshardedBundlesToTempFiles/DropShardNum as step s9
Sep 10, 2021 12:35:25 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to files/Write/WriteUnshardedBundlesToTempFiles/Flatten.PCollections as step s10
Sep 10, 2021 12:35:25 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to files/Write/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow) as step s11
Sep 10, 2021 12:35:25 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to files/Write/GatherTempFileResults/View.AsList/CreateDataflowView as step s12
Sep 10, 2021 12:35:25 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Create.Values/Read(CreateSource) as step s13
Sep 10, 2021 12:35:25 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Reify.ReifyView/ParDo(Anonymous) as step s14
Sep 10, 2021 12:35:25 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Values/Values/Map as step s15
Sep 10, 2021 12:35:25 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Finalize as step s16
Sep 10, 2021 12:35:25 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random key as step s17
Sep 10, 2021 12:35:25 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign as step s18
Sep 10, 2021 12:35:25 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey as step s19
Sep 10, 2021 12:35:25 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable as step s20
Sep 10, 2021 12:35:25 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Values/Values/Map as step s21
Sep 10, 2021 12:35:25 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.34.0-SNAPSHOT
Sep 10, 2021 12:35:26 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-09-09_17_35_25-8898352174485764482?project=apache-beam-testing
Sep 10, 2021 12:35:26 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-09-09_17_35_25-8898352174485764482
Sep 10, 2021 12:35:26 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-09-09_17_35_25-8898352174485764482
Sep 10, 2021 12:35:45 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-10T00:35:44.933Z: Worker configuration: e2-standard-2 in us-central1-c.
Sep 10, 2021 12:35:45 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-09-10T00:35:45.785Z: Workflow failed. Causes: Project apache-beam-testing has insufficient quota(s) to execute this workflow with 1 instances in region us-central1. Quota summary (required/available): 1/19491 instances, 2/0 CPUs, 25/244116 disk GB, 0/2397 SSD disk GB, 1/238 instance groups, 1/245 managed instance groups, 1/463 instance templates, 1/720 in-use IP addresses.
Please see https://cloud.google.com/compute/docs/resource-quotas about requesting more quota.
Sep 10, 2021 12:35:45 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-10T00:35:45.869Z: Cleaning up.
Sep 10, 2021 12:35:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-10T00:35:45.991Z: Worker pool stopped.
Sep 10, 2021 12:35:48 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-10T00:35:47.262Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Sep 10, 2021 12:35:50 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-09-09_17_35_25-8898352174485764482 failed with status FAILED.
Gradle Test Executor 44 finished executing tests.
> Task :sdks:java:io:file-based-io-tests:integrationTest FAILED
org.apache.beam.sdk.io.tfrecord.TFRecordIOIT > writeThenReadAll FAILED
java.lang.IllegalStateException: Unable to find any files matching gs://temp-storage-for-perf-tests/beam_PerformanceTests_TFRecordIOIT/5304/_1631234114680*
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:588)
at org.apache.beam.sdk.io.TFRecordIO$Read.expand(TFRecordIO.java:191)
at org.apache.beam.sdk.io.TFRecordIO$Read.expand(TFRecordIO.java:112)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:548)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:482)
at org.apache.beam.sdk.values.PBegin.apply(PBegin.java:44)
at org.apache.beam.sdk.Pipeline.apply(Pipeline.java:177)
at org.apache.beam.sdk.io.tfrecord.TFRecordIOIT.writeThenReadAll(TFRecordIOIT.java:151)
1 test completed, 1 failed
Finished generating test XML results (0.005 secs) into: <https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/file-based-io-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.005 secs) into: <https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/file-based-io-tests/build/reports/tests/integrationTest>
:sdks:java:io:file-based-io-tests:integrationTest (Thread[Execution **** for ':' Thread 5,5,main]) completed. Took 39.675 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:io:file-based-io-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/file-based-io-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 58s
129 actionable tasks: 89 executed, 40 from cache
Publishing build scan...
https://gradle.com/s/dojuth6jxyp6w
Stopped 3 **** daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org