You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@beam.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2021/09/11 00:36:06 UTC

Build failed in Jenkins: beam_PerformanceTests_TFRecordIOIT #5308

See <https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/5308/display/redirect?page=changes>

Changes:

[noreply] Port changes from Pub/Sub Lite to beam (#15418)

[heejong] [BEAM-12805] Fix XLang CombinePerKey test by explicitly assigning the

[BenWhitehead] [BEAM-8376] Google Cloud Firestore Connector - Add handling for

[noreply] Decreasing peak memory usage for beam.TupleCombineFn (#15494)

[noreply] [BEAM-12802] Add support for prefetch through data layers down through

[noreply] [BEAM-11097] Add implementation of side input cache (#15483)


------------------------------------------
[...truncated 281.96 KB...]
Caching disabled for task ':runners:google-cloud-dataflow-java:testJar' because:
  Caching has not been enabled for the task
Task ':runners:google-cloud-dataflow-java:testJar' is not up-to-date because:
  No history is available.
file or directory '<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/runners/google-cloud-dataflow-java/build/resources/test',> not found
:runners:google-cloud-dataflow-java:testJar (Thread[Execution **** for ':' Thread 8,5,main]) completed. Took 0.043 secs.

> Task :sdks:java:io:parquet:compileTestJava
Created classpath snapshot for incremental compilation in 5.118 secs. 1922 duplicate classes found in classpath (see all with --debug).
Stored cache entry for task ':sdks:java:io:parquet:compileTestJava' with cache key c6521b93bfa2fe0d7fcccb0ee0c0cfd3
:sdks:java:io:parquet:compileTestJava (Thread[Daemon ****,5,main]) completed. Took 10.7 secs.
:sdks:java:io:parquet:testClasses (Thread[Daemon ****,5,main]) started.

> Task :sdks:java:io:parquet:testClasses
Skipping task ':sdks:java:io:parquet:testClasses' as it has no actions.
:sdks:java:io:parquet:testClasses (Thread[Daemon ****,5,main]) completed. Took 0.0 secs.
:sdks:java:io:parquet:testJar (Thread[Daemon ****,5,main]) started.

> Task :sdks:java:io:parquet:testJar
Caching disabled for task ':sdks:java:io:parquet:testJar' because:
  Caching has not been enabled for the task
Task ':sdks:java:io:parquet:testJar' is not up-to-date because:
  No history is available.
:sdks:java:io:parquet:testJar (Thread[Daemon ****,5,main]) completed. Took 0.008 secs.

> Task :sdks:java:io:xml:compileJava
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Created classpath snapshot for incremental compilation in 0.011 secs. 1 duplicate classes found in classpath (see all with --debug).
Stored cache entry for task ':sdks:java:io:xml:compileJava' with cache key 8dab0cbeb3a5b05abb991284acd12372
:sdks:java:io:xml:compileJava (Thread[Execution **** for ':' Thread 9,5,main]) completed. Took 14.189 secs.
:sdks:java:io:xml:classes (Thread[Execution **** for ':' Thread 9,5,main]) started.

> Task :sdks:java:io:xml:classes
Skipping task ':sdks:java:io:xml:classes' as it has no actions.
:sdks:java:io:xml:classes (Thread[Execution **** for ':' Thread 9,5,main]) completed. Took 0.0 secs.
:sdks:java:io:xml:jar (Thread[Execution **** for ':' Thread 9,5,main]) started.

> Task :sdks:java:io:xml:jar
Caching disabled for task ':sdks:java:io:xml:jar' because:
  Caching has not been enabled for the task
Task ':sdks:java:io:xml:jar' is not up-to-date because:
  No history is available.
file or directory '<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/xml/build/resources/main',> not found
:sdks:java:io:xml:jar (Thread[Execution **** for ':' Thread 9,5,main]) completed. Took 0.018 secs.
:sdks:java:io:xml:compileTestJava (Thread[Execution **** for ':' Thread 9,5,main]) started.

> Task :sdks:java:io:xml:compileTestJava
Custom actions are attached to task ':sdks:java:io:xml:compileTestJava'.
Build cache key for task ':sdks:java:io:xml:compileTestJava' is a9c21598404b4d5d5967a46a3a47333b
Task ':sdks:java:io:xml:compileTestJava' is not up-to-date because:
  No history is available.
The input changes require a full rebuild for incremental task ':sdks:java:io:xml:compileTestJava'.
Full recompilation is required because no incremental change information is available. This is usually caused by clean builds or changing compiler arguments.
Compiling with JDK Java compiler API.
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Created classpath snapshot for incremental compilation in 0.03 secs. 1075 duplicate classes found in classpath (see all with --debug).
Stored cache entry for task ':sdks:java:io:xml:compileTestJava' with cache key a9c21598404b4d5d5967a46a3a47333b
:sdks:java:io:xml:compileTestJava (Thread[Execution **** for ':' Thread 9,5,main]) completed. Took 2.139 secs.
:sdks:java:io:xml:testClasses (Thread[Execution **** for ':' Thread 9,5,main]) started.

> Task :sdks:java:io:xml:testClasses
Skipping task ':sdks:java:io:xml:testClasses' as it has no actions.
:sdks:java:io:xml:testClasses (Thread[Execution **** for ':' Thread 9,5,main]) completed. Took 0.0 secs.
:sdks:java:io:xml:testJar (Thread[Execution **** for ':' Thread 9,5,main]) started.

> Task :sdks:java:io:xml:testJar
Caching disabled for task ':sdks:java:io:xml:testJar' because:
  Caching has not been enabled for the task
Task ':sdks:java:io:xml:testJar' is not up-to-date because:
  No history is available.
file or directory '<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/xml/build/resources/test',> not found
:sdks:java:io:xml:testJar (Thread[Execution **** for ':' Thread 9,5,main]) completed. Took 0.009 secs.
:sdks:java:io:file-based-io-tests:compileTestJava (Thread[Execution **** for ':' Thread 9,5,main]) started.

> Task :sdks:java:io:file-based-io-tests:compileTestJava
Custom actions are attached to task ':sdks:java:io:file-based-io-tests:compileTestJava'.
Build cache key for task ':sdks:java:io:file-based-io-tests:compileTestJava' is 338c1e5cd51c0d1cb4c8e1b2aabeba08
Task ':sdks:java:io:file-based-io-tests:compileTestJava' is not up-to-date because:
  No history is available.
The input changes require a full rebuild for incremental task ':sdks:java:io:file-based-io-tests:compileTestJava'.
Full recompilation is required because no incremental change information is available. This is usually caused by clean builds or changing compiler arguments.
Compiling with JDK Java compiler API.
Note: <https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/file-based-io-tests/src/test/java/org/apache/beam/sdk/io/tfrecord/TFRecordIOIT.java> uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Created classpath snapshot for incremental compilation in 0.027 secs. 1748 duplicate classes found in classpath (see all with --debug).
Stored cache entry for task ':sdks:java:io:file-based-io-tests:compileTestJava' with cache key 338c1e5cd51c0d1cb4c8e1b2aabeba08
:sdks:java:io:file-based-io-tests:compileTestJava (Thread[Execution **** for ':' Thread 9,5,main]) completed. Took 2.362 secs.
:sdks:java:io:file-based-io-tests:testClasses (Thread[Execution **** for ':' Thread 9,5,main]) started.

> Task :sdks:java:io:file-based-io-tests:testClasses
Skipping task ':sdks:java:io:file-based-io-tests:testClasses' as it has no actions.
:sdks:java:io:file-based-io-tests:testClasses (Thread[Execution **** for ':' Thread 9,5,main]) completed. Took 0.0 secs.
:sdks:java:io:file-based-io-tests:integrationTest (Thread[Execution **** for ':' Thread 9,5,main]) started.
Gradle Test Executor 3 started executing tests.

> Task :sdks:java:io:file-based-io-tests:integrationTest
Custom actions are attached to task ':sdks:java:io:file-based-io-tests:integrationTest'.
Build cache key for task ':sdks:java:io:file-based-io-tests:integrationTest' is d113188b16b5dc9e6e52b7033c284315
Task ':sdks:java:io:file-based-io-tests:integrationTest' is not up-to-date because:
  Task.upToDateWhen is false.
Starting process 'Gradle Test Executor 3'. Working directory: <https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/file-based-io-tests> Command: /usr/lib/jvm/java-8-openjdk-amd64/bin/java -DbeamTestPipelineOptions=["--runner=DataflowRunner","--project=apache-beam-testing","--tempRoot=gs://temp-storage-for-perf-tests","--filenamePrefix=gs://temp-storage-for-perf-tests/beam_PerformanceTests_TFRecordIOIT/5308/","--bigQueryDataset=beam_performance","--bigQueryTable=tfrecordioit_results","--influxMeasurement=tfrecordioit_results","--numberOfRecords=18000000","--expectedHash=543104423f8b6eb097acb9f111c19fe4","--datasetSize=1019380000","--numWorkers=5","--autoscalingAlgorithm=NONE","--influxDatabase=beam_test_metrics","--influxHost=http://10.128.0.96:8086","--****HarnessContainerImage=","--dataflowWorkerJar=<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/runners/google-cloud-dataflow-java/****/legacy-****/build/libs/beam-runners-google-cloud-dataflow-java-legacy-****-2.34.0-SNAPSHOT.jar","--region=us-central1"]> -Djava.security.manager=****.org.gradle.process.internal.****.child.BootstrapSecurityManager -Dorg.gradle.native=false -Xmx2g -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant -ea -cp /home/jenkins/.gradle/caches/6.8.3/****Main/gradle-****.jar ****.org.gradle.process.internal.****.GradleWorkerMain 'Gradle Test Executor 3'
Successfully started process 'Gradle Test Executor 3'

org.apache.beam.sdk.io.tfrecord.TFRecordIOIT STANDARD_ERROR
    SLF4J: Class path contains multiple SLF4J bindings.
    SLF4J: Found binding in [jar:<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/runners/google-cloud-dataflow-java/****/legacy-****/build/libs/beam-runners-google-cloud-dataflow-java-legacy-****-2.34.0-SNAPSHOT.jar!/org/slf4j/impl/StaticLoggerBinder.class]>
    SLF4J: Found binding in [jar:file:/home/jenkins/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-log4j12/1.7.30/c21f55139d8141d2231214fb1feaf50a1edca95e/slf4j-log4j12-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class]
    SLF4J: Found binding in [jar:file:/home/jenkins/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-jdk14/1.7.30/d35953dd2fe54ebe39fdf18cfd82fe6eb35b25ed/slf4j-jdk14-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class]
    SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
    SLF4J: Actual binding is of type [org.slf4j.impl.JDK14LoggerFactory]

org.apache.beam.sdk.io.tfrecord.TFRecordIOIT > writeThenReadAll STANDARD_ERROR
    Sep 11, 2021 12:35:27 AM org.apache.beam.runners.dataflow.DataflowRunner validateSdkContainerImageOptions
    WARNING: Prefer --sdkContainerImage over deprecated legacy option --****HarnessContainerImage.
    Sep 11, 2021 12:35:28 AM org.apache.beam.sdk.extensions.gcp.options.GcpOptions$GcpTempLocationFactory tryCreateDefaultBucket
    INFO: No tempLocation specified, attempting to use default bucket: dataflow-staging-us-central1-844138762903
    Sep 11, 2021 12:35:28 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
    WARNING: Request failed with code 409, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/storage/v1/b?predefinedAcl=projectPrivate&predefinedDefaultObjectAcl=projectPrivate&project=apache-beam-testing. 
    Sep 11, 2021 12:35:28 AM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
    INFO: No stagingLocation provided, falling back to gcpTempLocation
    Sep 11, 2021 12:35:29 AM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
    INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 317 files. Enable logging at DEBUG level to see which files will be staged.
    Sep 11, 2021 12:35:29 AM org.apache.beam.runners.dataflow.DataflowRunner run
    INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
    Sep 11, 2021 12:35:32 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
    INFO: Uploading 318 files from PipelineOptions.filesToStage to staging location to prepare for execution.
    Sep 11, 2021 12:35:32 AM org.apache.beam.runners.dataflow.util.PackageUtil$PackageAttributes forFileToStage
    INFO: Staging custom dataflow-****.jar as beam-runners-google-cloud-dataflow-java-legacy-****-2.34.0-SNAPSHOT-YtvkOIsfQYfFzG89aVBO_8oNHoiij9SynwUJLdMkn5A.jar
    Sep 11, 2021 12:35:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
    INFO: Uploading /tmp/test2240700900018133866.zip to gs://dataflow-staging-us-central1-844138762903/temp/staging/test-RFJlt6smrXAR4WwEksL2I3zreQAuP1YUAr92KMQQcUM.jar
    Sep 11, 2021 12:35:33 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
    INFO: Staging files complete: 317 files cached, 1 files newly uploaded in 0 seconds
    Sep 11, 2021 12:35:33 AM org.apache.beam.runners.dataflow.DataflowRunner run
    INFO: Staging portable pipeline proto to gs://dataflow-staging-us-central1-844138762903/temp/staging/
    Sep 11, 2021 12:35:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
    INFO: Uploading <205290 bytes, hash 4b47d566e04e08f1da1b84c97d638f1b91feed431d72d76db1a3b20aa4cfdaf7> to gs://dataflow-staging-us-central1-844138762903/temp/staging/pipeline-S0fVZuBOCPHaG4TJfWOPG5H-7UMdctdtsaOyCqTP2vc.pb
    Sep 11, 2021 12:35:35 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Generate sequence/Read(BoundedCountingSource) as step s1
    Sep 11, 2021 12:35:35 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Produce text lines as step s2
    Sep 11, 2021 12:35:35 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Transform strings to bytes/Map as step s3
    Sep 11, 2021 12:35:35 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Record time before writing as step s4
    Sep 11, 2021 12:35:35 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/RewindowIntoGlobal/Window.Assign as step s5
    Sep 11, 2021 12:35:35 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles as step s6
    Sep 11, 2021 12:35:35 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten as step s7
    Sep 11, 2021 12:35:35 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/WriteUnshardedBundlesToTempFiles/WriteUnwritten as step s8
    Sep 11, 2021 12:35:35 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/WriteUnshardedBundlesToTempFiles/DropShardNum as step s9
    Sep 11, 2021 12:35:35 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/WriteUnshardedBundlesToTempFiles/Flatten.PCollections as step s10
    Sep 11, 2021 12:35:35 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow) as step s11
    Sep 11, 2021 12:35:35 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/GatherTempFileResults/View.AsList/CreateDataflowView as step s12
    Sep 11, 2021 12:35:35 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Create.Values/Read(CreateSource) as step s13
    Sep 11, 2021 12:35:35 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Reify.ReifyView/ParDo(Anonymous) as step s14
    Sep 11, 2021 12:35:35 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Values/Values/Map as step s15
    Sep 11, 2021 12:35:35 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Finalize as step s16
    Sep 11, 2021 12:35:35 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random key as step s17
    Sep 11, 2021 12:35:35 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign as step s18
    Sep 11, 2021 12:35:35 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey as step s19
    Sep 11, 2021 12:35:35 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable as step s20
    Sep 11, 2021 12:35:35 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Values/Values/Map as step s21
    Sep 11, 2021 12:35:35 AM org.apache.beam.runners.dataflow.DataflowRunner run
    INFO: Dataflow SDK version: 2.34.0-SNAPSHOT
    Sep 11, 2021 12:35:36 AM org.apache.beam.runners.dataflow.DataflowRunner run
    INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-09-10_17_35_35-8643916549075032418?project=apache-beam-testing
    Sep 11, 2021 12:35:36 AM org.apache.beam.runners.dataflow.DataflowRunner run
    INFO: Submitted job: 2021-09-10_17_35_35-8643916549075032418
    Sep 11, 2021 12:35:36 AM org.apache.beam.runners.dataflow.DataflowRunner run
    INFO: To cancel the job using the 'gcloud' tool, run:
    > gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-09-10_17_35_35-8643916549075032418
    Sep 11, 2021 12:35:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-11T00:35:53.670Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
    Sep 11, 2021 12:35:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-11T00:35:57.635Z: Worker configuration: e2-standard-2 in us-central1-c.
    Sep 11, 2021 12:35:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    SEVERE: 2021-09-11T00:35:58.389Z: Workflow failed. Causes: Project apache-beam-testing has insufficient quota(s) to execute this workflow with 1 instances in region us-central1. Quota summary (required/available): 1/24318 instances, 2/0 CPUs, 25/174291 disk GB, 0/2397 SSD disk GB, 1/208 instance groups, 1/216 managed instance groups, 1/428 instance templates, 1/547 in-use IP addresses.

    Please see https://cloud.google.com/compute/docs/resource-quotas about requesting more quota.
    Sep 11, 2021 12:35:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-11T00:35:58.427Z: Cleaning up.
    Sep 11, 2021 12:35:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-11T00:35:58.486Z: Worker pool stopped.
    Sep 11, 2021 12:36:04 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
    INFO: Job 2021-09-10_17_35_35-8643916549075032418 failed with status FAILED.

Gradle Test Executor 3 finished executing tests.

> Task :sdks:java:io:file-based-io-tests:integrationTest FAILED

org.apache.beam.sdk.io.tfrecord.TFRecordIOIT > writeThenReadAll FAILED
    java.lang.IllegalStateException: Unable to find any files matching gs://temp-storage-for-perf-tests/beam_PerformanceTests_TFRecordIOIT/5308/_1631320526349*
        at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:588)
        at org.apache.beam.sdk.io.TFRecordIO$Read.expand(TFRecordIO.java:191)
        at org.apache.beam.sdk.io.TFRecordIO$Read.expand(TFRecordIO.java:112)
        at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:548)
        at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:482)
        at org.apache.beam.sdk.values.PBegin.apply(PBegin.java:44)
        at org.apache.beam.sdk.Pipeline.apply(Pipeline.java:177)
        at org.apache.beam.sdk.io.tfrecord.TFRecordIOIT.writeThenReadAll(TFRecordIOIT.java:151)

1 test completed, 1 failed
Finished generating test XML results (0.021 secs) into: <https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/file-based-io-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.031 secs) into: <https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/file-based-io-tests/build/reports/tests/integrationTest>
:sdks:java:io:file-based-io-tests:integrationTest (Thread[Execution **** for ':' Thread 9,5,main]) completed. Took 41.564 secs.

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:io:file-based-io-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/file-based-io-tests/build/reports/tests/integrationTest/index.html>

* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 1m 28s
129 actionable tasks: 83 executed, 46 from cache

Publishing build scan...
https://gradle.com/s/rvyzttipsegxe

Stopped 2 **** daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Jenkins build is back to normal : beam_PerformanceTests_TFRecordIOIT #5320

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/5320/display/redirect?page=changes>


---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_PerformanceTests_TFRecordIOIT #5319

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/5319/display/redirect?page=changes>

Changes:

[rohde.samuel] [BEAM-12842] Add timestamp to test work item to deflake

[suztomo] [BEAM-12873] HL7v2IO: to leave schematizedData null, not empty


------------------------------------------
[...truncated 274.42 KB...]
  No history is available.
file or directory '<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/google-cloud-platform/build/resources/main',> not found
:sdks:java:io:google-cloud-platform:jar (Thread[Execution **** for ':' Thread 5,5,main]) completed. Took 0.206 secs.
:runners:google-cloud-dataflow-java:compileJava (Thread[Execution **** for ':' Thread 5,5,main]) started.
:sdks:java:io:google-cloud-platform:compileTestJava (Thread[Execution **** for ':' Thread 2,5,main]) started.

> Task :runners:google-cloud-dataflow-java:compileJava FROM-CACHE
Custom actions are attached to task ':runners:google-cloud-dataflow-java:compileJava'.
Build cache key for task ':runners:google-cloud-dataflow-java:compileJava' is f4d45dae85b41e621c919183804b6afa
Task ':runners:google-cloud-dataflow-java:compileJava' is not up-to-date because:
  No history is available.
Loaded cache entry for task ':runners:google-cloud-dataflow-java:compileJava' with cache key f4d45dae85b41e621c919183804b6afa
:runners:google-cloud-dataflow-java:compileJava (Thread[Execution **** for ':' Thread 5,5,main]) completed. Took 0.168 secs.
:runners:google-cloud-dataflow-java:classes (Thread[Execution **** for ':' Thread 5,5,main]) started.

> Task :runners:google-cloud-dataflow-java:classes
Skipping task ':runners:google-cloud-dataflow-java:classes' as it has no actions.
:runners:google-cloud-dataflow-java:classes (Thread[Execution **** for ':' Thread 5,5,main]) completed. Took 0.0 secs.
:runners:google-cloud-dataflow-java:jar (Thread[Execution **** for ':' Thread 8,5,main]) started.

> Task :runners:google-cloud-dataflow-java:jar
Caching disabled for task ':runners:google-cloud-dataflow-java:jar' because:
  Caching has not been enabled for the task
Task ':runners:google-cloud-dataflow-java:jar' is not up-to-date because:
  No history is available.
:runners:google-cloud-dataflow-java:jar (Thread[Execution **** for ':' Thread 8,5,main]) completed. Took 0.077 secs.
:runners:google-cloud-dataflow-java:****:legacy-****:compileJava (Thread[Execution **** for ':' Thread 8,5,main]) started.

> Task :sdks:java:io:google-cloud-platform:compileTestJava FROM-CACHE
Custom actions are attached to task ':sdks:java:io:google-cloud-platform:compileTestJava'.
Build cache key for task ':sdks:java:io:google-cloud-platform:compileTestJava' is 99462c52f9bbbb5d949b676d07ee3bc7
Task ':sdks:java:io:google-cloud-platform:compileTestJava' is not up-to-date because:
  No history is available.
Loaded cache entry for task ':sdks:java:io:google-cloud-platform:compileTestJava' with cache key 99462c52f9bbbb5d949b676d07ee3bc7
:sdks:java:io:google-cloud-platform:compileTestJava (Thread[Execution **** for ':' Thread 2,5,main]) completed. Took 0.416 secs.
:sdks:java:io:google-cloud-platform:testClasses (Thread[Execution **** for ':' Thread 2,5,main]) started.

> Task :sdks:java:io:google-cloud-platform:testClasses
Skipping task ':sdks:java:io:google-cloud-platform:testClasses' as it has no actions.
:sdks:java:io:google-cloud-platform:testClasses (Thread[Execution **** for ':' Thread 2,5,main]) completed. Took 0.0 secs.
:sdks:java:io:google-cloud-platform:testJar (Thread[Execution **** for ':' Thread 2,5,main]) started.

> Task :runners:google-cloud-dataflow-java:****:legacy-****:compileJava FROM-CACHE
file or directory '<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/runners/google-cloud-dataflow-java/****/legacy-****/src/main/java',> not found
Custom actions are attached to task ':runners:google-cloud-dataflow-java:****:legacy-****:compileJava'.
Build cache key for task ':runners:google-cloud-dataflow-java:****:legacy-****:compileJava' is cf672da70ce14149e9f2f602b34780e4
Task ':runners:google-cloud-dataflow-java:****:legacy-****:compileJava' is not up-to-date because:
  No history is available.
Loaded cache entry for task ':runners:google-cloud-dataflow-java:****:legacy-****:compileJava' with cache key cf672da70ce14149e9f2f602b34780e4
:runners:google-cloud-dataflow-java:****:legacy-****:compileJava (Thread[Execution **** for ':' Thread 8,5,main]) completed. Took 0.277 secs.
:runners:google-cloud-dataflow-java:****:legacy-****:classes (Thread[Execution **** for ':' Thread 8,5,main]) started.

> Task :runners:google-cloud-dataflow-java:****:legacy-****:classes UP-TO-DATE
Skipping task ':runners:google-cloud-dataflow-java:****:legacy-****:classes' as it has no actions.
:runners:google-cloud-dataflow-java:****:legacy-****:classes (Thread[Execution **** for ':' Thread 8,5,main]) completed. Took 0.0 secs.
:runners:google-cloud-dataflow-java:****:legacy-****:shadowJar (Thread[Execution **** for ':' Thread 8,5,main]) started.

> Task :sdks:java:io:google-cloud-platform:testJar
Caching disabled for task ':sdks:java:io:google-cloud-platform:testJar' because:
  Caching has not been enabled for the task
Task ':sdks:java:io:google-cloud-platform:testJar' is not up-to-date because:
  No history is available.
:sdks:java:io:google-cloud-platform:testJar (Thread[Execution **** for ':' Thread 2,5,main]) completed. Took 0.263 secs.
:runners:google-cloud-dataflow-java:compileTestJava (Thread[Execution **** for ':' Thread 2,5,main]) started.

> Task :runners:google-cloud-dataflow-java:****:legacy-****:shadowJar FROM-CACHE
Custom actions are attached to task ':runners:google-cloud-dataflow-java:****:legacy-****:shadowJar'.
Build cache key for task ':runners:google-cloud-dataflow-java:****:legacy-****:shadowJar' is 165b57b3de0480886088cb77a0fc5516
Task ':runners:google-cloud-dataflow-java:****:legacy-****:shadowJar' is not up-to-date because:
  No history is available.
Loaded cache entry for task ':runners:google-cloud-dataflow-java:****:legacy-****:shadowJar' with cache key 165b57b3de0480886088cb77a0fc5516
:runners:google-cloud-dataflow-java:****:legacy-****:shadowJar (Thread[Execution **** for ':' Thread 8,5,main]) completed. Took 0.155 secs.

> Task :runners:google-cloud-dataflow-java:compileTestJava FROM-CACHE
Custom actions are attached to task ':runners:google-cloud-dataflow-java:compileTestJava'.
Build cache key for task ':runners:google-cloud-dataflow-java:compileTestJava' is 861c7435ffd549666858392e9bbd7747
Task ':runners:google-cloud-dataflow-java:compileTestJava' is not up-to-date because:
  No history is available.
Loaded cache entry for task ':runners:google-cloud-dataflow-java:compileTestJava' with cache key 861c7435ffd549666858392e9bbd7747
:runners:google-cloud-dataflow-java:compileTestJava (Thread[Execution **** for ':' Thread 2,5,main]) completed. Took 0.215 secs.
:runners:google-cloud-dataflow-java:testClasses (Thread[Execution **** for ':' Thread 2,5,main]) started.

> Task :runners:google-cloud-dataflow-java:testClasses UP-TO-DATE
Skipping task ':runners:google-cloud-dataflow-java:testClasses' as it has no actions.
:runners:google-cloud-dataflow-java:testClasses (Thread[Execution **** for ':' Thread 2,5,main]) completed. Took 0.0 secs.
:runners:google-cloud-dataflow-java:testJar (Thread[Execution **** for ':' Thread 2,5,main]) started.

> Task :runners:google-cloud-dataflow-java:testJar
Caching disabled for task ':runners:google-cloud-dataflow-java:testJar' because:
  Caching has not been enabled for the task
Task ':runners:google-cloud-dataflow-java:testJar' is not up-to-date because:
  No history is available.
file or directory '<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/runners/google-cloud-dataflow-java/build/resources/test',> not found
:runners:google-cloud-dataflow-java:testJar (Thread[Execution **** for ':' Thread 2,5,main]) completed. Took 0.03 secs.
:sdks:java:io:file-based-io-tests:integrationTest (Thread[Execution **** for ':' Thread 2,5,main]) started.

> Task :sdks:java:io:file-based-io-tests:integrationTest
Custom actions are attached to task ':sdks:java:io:file-based-io-tests:integrationTest'.
Build cache key for task ':sdks:java:io:file-based-io-tests:integrationTest' is 7afd111fb1a96c3526e1f49036cea8a2
Task ':sdks:java:io:file-based-io-tests:integrationTest' is not up-to-date because:
  Task.upToDateWhen is false.
Starting process 'Gradle Test Executor 1'. Working directory: <https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/file-based-io-tests> Command: /usr/lib/jvm/java-8-openjdk-amd64/bin/java -DbeamTestPipelineOptions=["--runner=DataflowRunner","--project=apache-beam-testing","--tempRoot=gs://temp-storage-for-perf-tests","--filenamePrefix=gs://temp-storage-for-perf-tests/beam_PerformanceTests_TFRecordIOIT/5319/","--bigQueryDataset=beam_performance","--bigQueryTable=tfrecordioit_results","--influxMeasurement=tfrecordioit_results","--numberOfRecords=18000000","--expectedHash=543104423f8b6eb097acb9f111c19fe4","--datasetSize=1019380000","--numWorkers=5","--autoscalingAlgorithm=NONE","--influxDatabase=beam_test_metrics","--influxHost=http://10.128.0.96:8086","--****HarnessContainerImage=","--dataflowWorkerJar=<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/runners/google-cloud-dataflow-java/****/legacy-****/build/libs/beam-runners-google-cloud-dataflow-java-legacy-****-2.34.0-SNAPSHOT.jar","--region=us-central1"]> -Djava.security.manager=****.org.gradle.process.internal.****.child.BootstrapSecurityManager -Dorg.gradle.native=false -Xmx2g -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant -ea -cp /home/jenkins/.gradle/caches/6.8.3/****Main/gradle-****.jar ****.org.gradle.process.internal.****.GradleWorkerMain 'Gradle Test Executor 1'
Successfully started process 'Gradle Test Executor 1'

Gradle Test Executor 1 started executing tests.

> Task :sdks:java:io:file-based-io-tests:integrationTest

org.apache.beam.sdk.io.tfrecord.TFRecordIOIT STANDARD_ERROR
    SLF4J: Class path contains multiple SLF4J bindings.
    SLF4J: Found binding in [jar:<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/runners/google-cloud-dataflow-java/****/legacy-****/build/libs/beam-runners-google-cloud-dataflow-java-legacy-****-2.34.0-SNAPSHOT.jar!/org/slf4j/impl/StaticLoggerBinder.class]>
    SLF4J: Found binding in [jar:file:/home/jenkins/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-log4j12/1.7.30/c21f55139d8141d2231214fb1feaf50a1edca95e/slf4j-log4j12-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class]
    SLF4J: Found binding in [jar:file:/home/jenkins/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-jdk14/1.7.30/d35953dd2fe54ebe39fdf18cfd82fe6eb35b25ed/slf4j-jdk14-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class]
    SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
    SLF4J: Actual binding is of type [org.slf4j.impl.JDK14LoggerFactory]

org.apache.beam.sdk.io.tfrecord.TFRecordIOIT > writeThenReadAll STANDARD_ERROR
    Sep 13, 2021 6:21:51 PM org.apache.beam.runners.dataflow.DataflowRunner validateSdkContainerImageOptions
    WARNING: Prefer --sdkContainerImage over deprecated legacy option --****HarnessContainerImage.
    Sep 13, 2021 6:21:52 PM org.apache.beam.sdk.extensions.gcp.options.GcpOptions$GcpTempLocationFactory tryCreateDefaultBucket
    INFO: No tempLocation specified, attempting to use default bucket: dataflow-staging-us-central1-844138762903
    Sep 13, 2021 6:21:53 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
    WARNING: Request failed with code 409, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/storage/v1/b?predefinedAcl=projectPrivate&predefinedDefaultObjectAcl=projectPrivate&project=apache-beam-testing. 
    Sep 13, 2021 6:21:53 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
    INFO: No stagingLocation provided, falling back to gcpTempLocation
    Sep 13, 2021 6:21:53 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
    INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 317 files. Enable logging at DEBUG level to see which files will be staged.
    Sep 13, 2021 6:21:54 PM org.apache.beam.runners.dataflow.DataflowRunner run
    INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
    Sep 13, 2021 6:21:57 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
    INFO: Uploading 318 files from PipelineOptions.filesToStage to staging location to prepare for execution.
    Sep 13, 2021 6:21:57 PM org.apache.beam.runners.dataflow.util.PackageUtil$PackageAttributes forFileToStage
    INFO: Staging custom dataflow-****.jar as beam-runners-google-cloud-dataflow-java-legacy-****-2.34.0-SNAPSHOT-YtvkOIsfQYfFzG89aVBO_8oNHoiij9SynwUJLdMkn5A.jar
    Sep 13, 2021 6:21:57 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
    INFO: Staging files complete: 318 files cached, 0 files newly uploaded in 0 seconds
    Sep 13, 2021 6:21:57 PM org.apache.beam.runners.dataflow.DataflowRunner run
    INFO: Staging portable pipeline proto to gs://dataflow-staging-us-central1-844138762903/temp/staging/
    Sep 13, 2021 6:21:57 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
    INFO: Uploading <205290 bytes, hash 9c00b8f6d51a76ad7b86305270ea4b60e9f2f9e5b00090b0cdceacaa184afe61> to gs://dataflow-staging-us-central1-844138762903/temp/staging/pipeline-nAC49tUadq17hjBScOpLYOny-eWwAJCwzc6sqhhK_mE.pb
    Sep 13, 2021 6:21:59 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Generate sequence/Read(BoundedCountingSource) as step s1
    Sep 13, 2021 6:21:59 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Produce text lines as step s2
    Sep 13, 2021 6:21:59 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Transform strings to bytes/Map as step s3
    Sep 13, 2021 6:21:59 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Record time before writing as step s4
    Sep 13, 2021 6:21:59 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/RewindowIntoGlobal/Window.Assign as step s5
    Sep 13, 2021 6:21:59 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles as step s6
    Sep 13, 2021 6:21:59 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten as step s7
    Sep 13, 2021 6:21:59 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/WriteUnshardedBundlesToTempFiles/WriteUnwritten as step s8
    Sep 13, 2021 6:21:59 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/WriteUnshardedBundlesToTempFiles/DropShardNum as step s9
    Sep 13, 2021 6:21:59 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/WriteUnshardedBundlesToTempFiles/Flatten.PCollections as step s10
    Sep 13, 2021 6:21:59 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow) as step s11
    Sep 13, 2021 6:21:59 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/GatherTempFileResults/View.AsList/CreateDataflowView as step s12
    Sep 13, 2021 6:21:59 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Create.Values/Read(CreateSource) as step s13
    Sep 13, 2021 6:21:59 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Reify.ReifyView/ParDo(Anonymous) as step s14
    Sep 13, 2021 6:22:00 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Values/Values/Map as step s15
    Sep 13, 2021 6:22:00 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Finalize as step s16
    Sep 13, 2021 6:22:00 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random key as step s17
    Sep 13, 2021 6:22:00 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign as step s18
    Sep 13, 2021 6:22:00 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey as step s19
    Sep 13, 2021 6:22:00 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable as step s20
    Sep 13, 2021 6:22:00 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Values/Values/Map as step s21
    Sep 13, 2021 6:22:00 PM org.apache.beam.runners.dataflow.DataflowRunner run
    INFO: Dataflow SDK version: 2.34.0-SNAPSHOT
    Sep 13, 2021 6:22:01 PM org.apache.beam.runners.dataflow.DataflowRunner run
    INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-09-13_11_22_00-8936937029889501075?project=apache-beam-testing
    Sep 13, 2021 6:22:01 PM org.apache.beam.runners.dataflow.DataflowRunner run
    INFO: Submitted job: 2021-09-13_11_22_00-8936937029889501075
    Sep 13, 2021 6:22:01 PM org.apache.beam.runners.dataflow.DataflowRunner run
    INFO: To cancel the job using the 'gcloud' tool, run:
    > gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-09-13_11_22_00-8936937029889501075
    Sep 13, 2021 6:22:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-13T18:22:21.324Z: Worker configuration: e2-standard-2 in us-central1-c.
    Sep 13, 2021 6:22:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    SEVERE: 2021-09-13T18:22:22.834Z: Workflow failed. Causes: Project apache-beam-testing has insufficient quota(s) to execute this workflow with 1 instances in region us-central1. Quota summary (required/available): 1/24385 instances, 2/0 CPUs, 25/183466 disk GB, 0/2397 SSD disk GB, 1/262 instance groups, 1/265 managed instance groups, 1/489 instance templates, 1/614 in-use IP addresses.

    Please see https://cloud.google.com/compute/docs/resource-quotas about requesting more quota.
    Sep 13, 2021 6:22:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-13T18:22:22.884Z: Cleaning up.
    Sep 13, 2021 6:22:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-13T18:22:22.953Z: Worker pool stopped.
    Sep 13, 2021 6:22:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-13T18:22:24.160Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
    Sep 13, 2021 6:22:28 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
    INFO: Job 2021-09-13_11_22_00-8936937029889501075 failed with status FAILED.

Gradle Test Executor 1 finished executing tests.

> Task :sdks:java:io:file-based-io-tests:integrationTest FAILED

org.apache.beam.sdk.io.tfrecord.TFRecordIOIT > writeThenReadAll FAILED
    java.lang.IllegalStateException: Unable to find any files matching gs://temp-storage-for-perf-tests/beam_PerformanceTests_TFRecordIOIT/5319/_1631557310966*
        at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:588)
        at org.apache.beam.sdk.io.TFRecordIO$Read.expand(TFRecordIO.java:191)
        at org.apache.beam.sdk.io.TFRecordIO$Read.expand(TFRecordIO.java:112)
        at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:548)
        at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:482)
        at org.apache.beam.sdk.values.PBegin.apply(PBegin.java:44)
        at org.apache.beam.sdk.Pipeline.apply(Pipeline.java:177)
        at org.apache.beam.sdk.io.tfrecord.TFRecordIOIT.writeThenReadAll(TFRecordIOIT.java:151)

1 test completed, 1 failed
Finished generating test XML results (0.019 secs) into: <https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/file-based-io-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.028 secs) into: <https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/file-based-io-tests/build/reports/tests/integrationTest>
:sdks:java:io:file-based-io-tests:integrationTest (Thread[Execution **** for ':' Thread 2,5,main]) completed. Took 42.053 secs.

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:io:file-based-io-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/file-based-io-tests/build/reports/tests/integrationTest/index.html>

* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 1m 11s
129 actionable tasks: 79 executed, 50 from cache

Publishing build scan...
https://gradle.com/s/v5fmtiigke3ow

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_PerformanceTests_TFRecordIOIT #5318

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/5318/display/redirect>

Changes:


------------------------------------------
[...truncated 274.38 KB...]
Task ':sdks:java:io:google-cloud-platform:jar' is not up-to-date because:
  No history is available.
file or directory '<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/google-cloud-platform/build/resources/main',> not found
:sdks:java:io:google-cloud-platform:jar (Thread[Execution **** for ':' Thread 4,5,main]) completed. Took 0.237 secs.
:runners:google-cloud-dataflow-java:compileJava (Thread[Execution **** for ':' Thread 4,5,main]) started.
:sdks:java:io:google-cloud-platform:compileTestJava (Thread[Execution **** for ':' Thread 6,5,main]) started.

> Task :runners:google-cloud-dataflow-java:compileJava FROM-CACHE
Custom actions are attached to task ':runners:google-cloud-dataflow-java:compileJava'.
Build cache key for task ':runners:google-cloud-dataflow-java:compileJava' is f4d45dae85b41e621c919183804b6afa
Task ':runners:google-cloud-dataflow-java:compileJava' is not up-to-date because:
  No history is available.
Loaded cache entry for task ':runners:google-cloud-dataflow-java:compileJava' with cache key f4d45dae85b41e621c919183804b6afa
:runners:google-cloud-dataflow-java:compileJava (Thread[Execution **** for ':' Thread 4,5,main]) completed. Took 0.205 secs.
:runners:google-cloud-dataflow-java:classes (Thread[Execution **** for ':' Thread 4,5,main]) started.

> Task :runners:google-cloud-dataflow-java:classes
Skipping task ':runners:google-cloud-dataflow-java:classes' as it has no actions.
:runners:google-cloud-dataflow-java:classes (Thread[Execution **** for ':' Thread 4,5,main]) completed. Took 0.0 secs.
:runners:google-cloud-dataflow-java:jar (Thread[Execution **** for ':' Thread 4,5,main]) started.

> Task :runners:google-cloud-dataflow-java:jar
Caching disabled for task ':runners:google-cloud-dataflow-java:jar' because:
  Caching has not been enabled for the task
Task ':runners:google-cloud-dataflow-java:jar' is not up-to-date because:
  No history is available.
:runners:google-cloud-dataflow-java:jar (Thread[Execution **** for ':' Thread 4,5,main]) completed. Took 0.07 secs.
:runners:google-cloud-dataflow-java:****:legacy-****:compileJava (Thread[Execution **** for ':' Thread 4,5,main]) started.

> Task :sdks:java:io:google-cloud-platform:compileTestJava FROM-CACHE
Custom actions are attached to task ':sdks:java:io:google-cloud-platform:compileTestJava'.
Build cache key for task ':sdks:java:io:google-cloud-platform:compileTestJava' is 99462c52f9bbbb5d949b676d07ee3bc7
Task ':sdks:java:io:google-cloud-platform:compileTestJava' is not up-to-date because:
  No history is available.
Loaded cache entry for task ':sdks:java:io:google-cloud-platform:compileTestJava' with cache key 99462c52f9bbbb5d949b676d07ee3bc7
:sdks:java:io:google-cloud-platform:compileTestJava (Thread[Execution **** for ':' Thread 6,5,main]) completed. Took 0.475 secs.
:sdks:java:io:google-cloud-platform:testClasses (Thread[Execution **** for ':' Thread 6,5,main]) started.

> Task :sdks:java:io:google-cloud-platform:testClasses
Skipping task ':sdks:java:io:google-cloud-platform:testClasses' as it has no actions.
:sdks:java:io:google-cloud-platform:testClasses (Thread[Execution **** for ':' Thread 6,5,main]) completed. Took 0.0 secs.
:sdks:java:io:google-cloud-platform:testJar (Thread[Execution **** for ':' Thread 6,5,main]) started.

> Task :runners:google-cloud-dataflow-java:****:legacy-****:compileJava FROM-CACHE
file or directory '<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/runners/google-cloud-dataflow-java/****/legacy-****/src/main/java',> not found
Custom actions are attached to task ':runners:google-cloud-dataflow-java:****:legacy-****:compileJava'.
Build cache key for task ':runners:google-cloud-dataflow-java:****:legacy-****:compileJava' is cf672da70ce14149e9f2f602b34780e4
Task ':runners:google-cloud-dataflow-java:****:legacy-****:compileJava' is not up-to-date because:
  No history is available.
Loaded cache entry for task ':runners:google-cloud-dataflow-java:****:legacy-****:compileJava' with cache key cf672da70ce14149e9f2f602b34780e4
:runners:google-cloud-dataflow-java:****:legacy-****:compileJava (Thread[Execution **** for ':' Thread 4,5,main]) completed. Took 0.327 secs.
:runners:google-cloud-dataflow-java:****:legacy-****:classes (Thread[Execution **** for ':' Thread 4,5,main]) started.

> Task :runners:google-cloud-dataflow-java:****:legacy-****:classes UP-TO-DATE
Skipping task ':runners:google-cloud-dataflow-java:****:legacy-****:classes' as it has no actions.
:runners:google-cloud-dataflow-java:****:legacy-****:classes (Thread[Execution **** for ':' Thread 4,5,main]) completed. Took 0.0 secs.
:runners:google-cloud-dataflow-java:****:legacy-****:shadowJar (Thread[Execution **** for ':' Thread 4,5,main]) started.

> Task :sdks:java:io:google-cloud-platform:testJar
Caching disabled for task ':sdks:java:io:google-cloud-platform:testJar' because:
  Caching has not been enabled for the task
Task ':sdks:java:io:google-cloud-platform:testJar' is not up-to-date because:
  No history is available.
:sdks:java:io:google-cloud-platform:testJar (Thread[Execution **** for ':' Thread 6,5,main]) completed. Took 0.265 secs.
:runners:google-cloud-dataflow-java:compileTestJava (Thread[Execution **** for ':' Thread 6,5,main]) started.

> Task :runners:google-cloud-dataflow-java:****:legacy-****:shadowJar FROM-CACHE
Custom actions are attached to task ':runners:google-cloud-dataflow-java:****:legacy-****:shadowJar'.
Build cache key for task ':runners:google-cloud-dataflow-java:****:legacy-****:shadowJar' is b03e31546baded1569b9e22785f539a5
Task ':runners:google-cloud-dataflow-java:****:legacy-****:shadowJar' is not up-to-date because:
  No history is available.
Loaded cache entry for task ':runners:google-cloud-dataflow-java:****:legacy-****:shadowJar' with cache key b03e31546baded1569b9e22785f539a5
:runners:google-cloud-dataflow-java:****:legacy-****:shadowJar (Thread[Execution **** for ':' Thread 4,5,main]) completed. Took 0.174 secs.

> Task :runners:google-cloud-dataflow-java:compileTestJava FROM-CACHE
Custom actions are attached to task ':runners:google-cloud-dataflow-java:compileTestJava'.
Build cache key for task ':runners:google-cloud-dataflow-java:compileTestJava' is 861c7435ffd549666858392e9bbd7747
Task ':runners:google-cloud-dataflow-java:compileTestJava' is not up-to-date because:
  No history is available.
Loaded cache entry for task ':runners:google-cloud-dataflow-java:compileTestJava' with cache key 861c7435ffd549666858392e9bbd7747
:runners:google-cloud-dataflow-java:compileTestJava (Thread[Execution **** for ':' Thread 6,5,main]) completed. Took 0.195 secs.
:runners:google-cloud-dataflow-java:testClasses (Thread[Execution **** for ':' Thread 6,5,main]) started.

> Task :runners:google-cloud-dataflow-java:testClasses UP-TO-DATE
Skipping task ':runners:google-cloud-dataflow-java:testClasses' as it has no actions.
:runners:google-cloud-dataflow-java:testClasses (Thread[Execution **** for ':' Thread 6,5,main]) completed. Took 0.0 secs.
:runners:google-cloud-dataflow-java:testJar (Thread[Execution **** for ':' Thread 6,5,main]) started.

> Task :runners:google-cloud-dataflow-java:testJar
Caching disabled for task ':runners:google-cloud-dataflow-java:testJar' because:
  Caching has not been enabled for the task
Task ':runners:google-cloud-dataflow-java:testJar' is not up-to-date because:
  No history is available.
file or directory '<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/runners/google-cloud-dataflow-java/build/resources/test',> not found
:runners:google-cloud-dataflow-java:testJar (Thread[Execution **** for ':' Thread 6,5,main]) completed. Took 0.03 secs.
:sdks:java:io:file-based-io-tests:integrationTest (Thread[Execution **** for ':' Thread 6,5,main]) started.
Gradle Test Executor 1 started executing tests.

> Task :sdks:java:io:file-based-io-tests:integrationTest
Custom actions are attached to task ':sdks:java:io:file-based-io-tests:integrationTest'.
Build cache key for task ':sdks:java:io:file-based-io-tests:integrationTest' is 2954f27e833e06d9b6038317a4b1cf00
Task ':sdks:java:io:file-based-io-tests:integrationTest' is not up-to-date because:
  Task.upToDateWhen is false.
Starting process 'Gradle Test Executor 1'. Working directory: <https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/file-based-io-tests> Command: /usr/lib/jvm/java-8-openjdk-amd64/bin/java -DbeamTestPipelineOptions=["--runner=DataflowRunner","--project=apache-beam-testing","--tempRoot=gs://temp-storage-for-perf-tests","--filenamePrefix=gs://temp-storage-for-perf-tests/beam_PerformanceTests_TFRecordIOIT/5318/","--bigQueryDataset=beam_performance","--bigQueryTable=tfrecordioit_results","--influxMeasurement=tfrecordioit_results","--numberOfRecords=18000000","--expectedHash=543104423f8b6eb097acb9f111c19fe4","--datasetSize=1019380000","--numWorkers=5","--autoscalingAlgorithm=NONE","--influxDatabase=beam_test_metrics","--influxHost=http://10.128.0.96:8086","--****HarnessContainerImage=","--dataflowWorkerJar=<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/runners/google-cloud-dataflow-java/****/legacy-****/build/libs/beam-runners-google-cloud-dataflow-java-legacy-****-2.34.0-SNAPSHOT.jar","--region=us-central1"]> -Djava.security.manager=****.org.gradle.process.internal.****.child.BootstrapSecurityManager -Dorg.gradle.native=false -Xmx2g -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant -ea -cp /home/jenkins/.gradle/caches/6.8.3/****Main/gradle-****.jar ****.org.gradle.process.internal.****.GradleWorkerMain 'Gradle Test Executor 1'
Successfully started process 'Gradle Test Executor 1'

org.apache.beam.sdk.io.tfrecord.TFRecordIOIT STANDARD_ERROR
    SLF4J: Class path contains multiple SLF4J bindings.
    SLF4J: Found binding in [jar:<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/runners/google-cloud-dataflow-java/****/legacy-****/build/libs/beam-runners-google-cloud-dataflow-java-legacy-****-2.34.0-SNAPSHOT.jar!/org/slf4j/impl/StaticLoggerBinder.class]>
    SLF4J: Found binding in [jar:file:/home/jenkins/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-log4j12/1.7.30/c21f55139d8141d2231214fb1feaf50a1edca95e/slf4j-log4j12-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class]
    SLF4J: Found binding in [jar:file:/home/jenkins/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-jdk14/1.7.30/d35953dd2fe54ebe39fdf18cfd82fe6eb35b25ed/slf4j-jdk14-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class]
    SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
    SLF4J: Actual binding is of type [org.slf4j.impl.JDK14LoggerFactory]

org.apache.beam.sdk.io.tfrecord.TFRecordIOIT > writeThenReadAll STANDARD_ERROR
    Sep 13, 2021 12:21:54 PM org.apache.beam.runners.dataflow.DataflowRunner validateSdkContainerImageOptions
    WARNING: Prefer --sdkContainerImage over deprecated legacy option --****HarnessContainerImage.
    Sep 13, 2021 12:21:55 PM org.apache.beam.sdk.extensions.gcp.options.GcpOptions$GcpTempLocationFactory tryCreateDefaultBucket
    INFO: No tempLocation specified, attempting to use default bucket: dataflow-staging-us-central1-844138762903
    Sep 13, 2021 12:21:56 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
    WARNING: Request failed with code 409, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/storage/v1/b?predefinedAcl=projectPrivate&predefinedDefaultObjectAcl=projectPrivate&project=apache-beam-testing. 
    Sep 13, 2021 12:21:56 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
    INFO: No stagingLocation provided, falling back to gcpTempLocation
    Sep 13, 2021 12:21:56 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
    INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 317 files. Enable logging at DEBUG level to see which files will be staged.
    Sep 13, 2021 12:21:56 PM org.apache.beam.runners.dataflow.DataflowRunner run
    INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
    Sep 13, 2021 12:21:59 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
    INFO: Uploading 318 files from PipelineOptions.filesToStage to staging location to prepare for execution.
    Sep 13, 2021 12:21:59 PM org.apache.beam.runners.dataflow.util.PackageUtil$PackageAttributes forFileToStage
    INFO: Staging custom dataflow-****.jar as beam-runners-google-cloud-dataflow-java-legacy-****-2.34.0-SNAPSHOT-YtvkOIsfQYfFzG89aVBO_8oNHoiij9SynwUJLdMkn5A.jar
    Sep 13, 2021 12:21:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
    INFO: Uploading /tmp/test7812669155832243219.zip to gs://dataflow-staging-us-central1-844138762903/temp/staging/test-nL1rtRtLm90VKwPJkj_v7x1yOXeqaHoz2ine0xdRZ-4.jar
    Sep 13, 2021 12:22:00 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
    INFO: Staging files complete: 317 files cached, 1 files newly uploaded in 1 seconds
    Sep 13, 2021 12:22:00 PM org.apache.beam.runners.dataflow.DataflowRunner run
    INFO: Staging portable pipeline proto to gs://dataflow-staging-us-central1-844138762903/temp/staging/
    Sep 13, 2021 12:22:00 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
    INFO: Uploading <205290 bytes, hash 657a029cba0782f3eaba8cb9b2fec74a43cea4fbd705f64abcbd9660d3414855> to gs://dataflow-staging-us-central1-844138762903/temp/staging/pipeline-ZXoCnLoHgvPquoy5sv7HSkPOpPvXBfZKvL2WYNNBSFU.pb
    Sep 13, 2021 12:22:02 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Generate sequence/Read(BoundedCountingSource) as step s1
    Sep 13, 2021 12:22:02 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Produce text lines as step s2
    Sep 13, 2021 12:22:02 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Transform strings to bytes/Map as step s3
    Sep 13, 2021 12:22:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Record time before writing as step s4
    Sep 13, 2021 12:22:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/RewindowIntoGlobal/Window.Assign as step s5
    Sep 13, 2021 12:22:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles as step s6
    Sep 13, 2021 12:22:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten as step s7
    Sep 13, 2021 12:22:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/WriteUnshardedBundlesToTempFiles/WriteUnwritten as step s8
    Sep 13, 2021 12:22:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/WriteUnshardedBundlesToTempFiles/DropShardNum as step s9
    Sep 13, 2021 12:22:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/WriteUnshardedBundlesToTempFiles/Flatten.PCollections as step s10
    Sep 13, 2021 12:22:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow) as step s11
    Sep 13, 2021 12:22:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/GatherTempFileResults/View.AsList/CreateDataflowView as step s12
    Sep 13, 2021 12:22:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Create.Values/Read(CreateSource) as step s13
    Sep 13, 2021 12:22:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Reify.ReifyView/ParDo(Anonymous) as step s14
    Sep 13, 2021 12:22:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Values/Values/Map as step s15
    Sep 13, 2021 12:22:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Finalize as step s16
    Sep 13, 2021 12:22:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random key as step s17
    Sep 13, 2021 12:22:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign as step s18
    Sep 13, 2021 12:22:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey as step s19
    Sep 13, 2021 12:22:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable as step s20
    Sep 13, 2021 12:22:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Values/Values/Map as step s21
    Sep 13, 2021 12:22:03 PM org.apache.beam.runners.dataflow.DataflowRunner run
    INFO: Dataflow SDK version: 2.34.0-SNAPSHOT
    Sep 13, 2021 12:22:04 PM org.apache.beam.runners.dataflow.DataflowRunner run
    INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-09-13_05_22_03-14479944029719028474?project=apache-beam-testing
    Sep 13, 2021 12:22:04 PM org.apache.beam.runners.dataflow.DataflowRunner run
    INFO: Submitted job: 2021-09-13_05_22_03-14479944029719028474
    Sep 13, 2021 12:22:04 PM org.apache.beam.runners.dataflow.DataflowRunner run
    INFO: To cancel the job using the 'gcloud' tool, run:
    > gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-09-13_05_22_03-14479944029719028474
    Sep 13, 2021 12:22:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-13T12:22:26.838Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
    Sep 13, 2021 12:22:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-13T12:22:28.228Z: Worker configuration: e2-standard-2 in us-central1-c.
    Sep 13, 2021 12:22:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    SEVERE: 2021-09-13T12:22:28.936Z: Workflow failed. Causes: Project apache-beam-testing has insufficient quota(s) to execute this workflow with 1 instances in region us-central1. Quota summary (required/available): 1/24386 instances, 2/0 CPUs, 25/183716 disk GB, 0/2397 SSD disk GB, 1/286 instance groups, 1/289 managed instance groups, 1/515 instance templates, 1/615 in-use IP addresses.

    Please see https://cloud.google.com/compute/docs/resource-quotas about requesting more quota.
    Sep 13, 2021 12:22:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-13T12:22:28.984Z: Cleaning up.
    Sep 13, 2021 12:22:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-13T12:22:29.041Z: Worker pool stopped.
    Sep 13, 2021 12:22:36 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
    INFO: Job 2021-09-13_05_22_03-14479944029719028474 failed with status FAILED.

Gradle Test Executor 1 finished executing tests.

> Task :sdks:java:io:file-based-io-tests:integrationTest FAILED

org.apache.beam.sdk.io.tfrecord.TFRecordIOIT > writeThenReadAll FAILED
    java.lang.IllegalStateException: Unable to find any files matching gs://temp-storage-for-perf-tests/beam_PerformanceTests_TFRecordIOIT/5318/_1631535713235*
        at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:588)
        at org.apache.beam.sdk.io.TFRecordIO$Read.expand(TFRecordIO.java:191)
        at org.apache.beam.sdk.io.TFRecordIO$Read.expand(TFRecordIO.java:112)
        at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:548)
        at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:482)
        at org.apache.beam.sdk.values.PBegin.apply(PBegin.java:44)
        at org.apache.beam.sdk.Pipeline.apply(Pipeline.java:177)
        at org.apache.beam.sdk.io.tfrecord.TFRecordIOIT.writeThenReadAll(TFRecordIOIT.java:151)

1 test completed, 1 failed
Finished generating test XML results (0.021 secs) into: <https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/file-based-io-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.032 secs) into: <https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/file-based-io-tests/build/reports/tests/integrationTest>
:sdks:java:io:file-based-io-tests:integrationTest (Thread[Execution **** for ':' Thread 6,5,main]) completed. Took 46.847 secs.

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:io:file-based-io-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/file-based-io-tests/build/reports/tests/integrationTest/index.html>

* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 1m 17s
129 actionable tasks: 79 executed, 50 from cache

Publishing build scan...
https://gradle.com/s/cg474npib7pok

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_PerformanceTests_TFRecordIOIT #5317

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/5317/display/redirect>

Changes:


------------------------------------------
[...truncated 274.65 KB...]
Task ':sdks:java:io:google-cloud-platform:jar' is not up-to-date because:
  No history is available.
file or directory '<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/google-cloud-platform/build/resources/main',> not found
:sdks:java:io:google-cloud-platform:jar (Thread[Execution **** for ':' Thread 3,5,main]) completed. Took 0.23 secs.
:runners:google-cloud-dataflow-java:compileJava (Thread[Execution **** for ':' Thread 3,5,main]) started.
:sdks:java:io:google-cloud-platform:compileTestJava (Thread[Daemon ****,5,main]) started.

> Task :runners:google-cloud-dataflow-java:compileJava FROM-CACHE
Custom actions are attached to task ':runners:google-cloud-dataflow-java:compileJava'.
Build cache key for task ':runners:google-cloud-dataflow-java:compileJava' is f4d45dae85b41e621c919183804b6afa
Task ':runners:google-cloud-dataflow-java:compileJava' is not up-to-date because:
  No history is available.
Loaded cache entry for task ':runners:google-cloud-dataflow-java:compileJava' with cache key f4d45dae85b41e621c919183804b6afa
:runners:google-cloud-dataflow-java:compileJava (Thread[Execution **** for ':' Thread 3,5,main]) completed. Took 0.214 secs.
:runners:google-cloud-dataflow-java:classes (Thread[Execution **** for ':' Thread 3,5,main]) started.

> Task :runners:google-cloud-dataflow-java:classes
Skipping task ':runners:google-cloud-dataflow-java:classes' as it has no actions.
:runners:google-cloud-dataflow-java:classes (Thread[Execution **** for ':' Thread 3,5,main]) completed. Took 0.0 secs.
:runners:google-cloud-dataflow-java:jar (Thread[Execution **** for ':' Thread 3,5,main]) started.

> Task :runners:google-cloud-dataflow-java:jar
Caching disabled for task ':runners:google-cloud-dataflow-java:jar' because:
  Caching has not been enabled for the task
Task ':runners:google-cloud-dataflow-java:jar' is not up-to-date because:
  No history is available.
:runners:google-cloud-dataflow-java:jar (Thread[Execution **** for ':' Thread 3,5,main]) completed. Took 0.065 secs.
:runners:google-cloud-dataflow-java:****:legacy-****:compileJava (Thread[Execution **** for ':' Thread 3,5,main]) started.

> Task :sdks:java:io:google-cloud-platform:compileTestJava FROM-CACHE
Custom actions are attached to task ':sdks:java:io:google-cloud-platform:compileTestJava'.
Build cache key for task ':sdks:java:io:google-cloud-platform:compileTestJava' is 99462c52f9bbbb5d949b676d07ee3bc7
Task ':sdks:java:io:google-cloud-platform:compileTestJava' is not up-to-date because:
  No history is available.
Loaded cache entry for task ':sdks:java:io:google-cloud-platform:compileTestJava' with cache key 99462c52f9bbbb5d949b676d07ee3bc7
:sdks:java:io:google-cloud-platform:compileTestJava (Thread[Daemon ****,5,main]) completed. Took 0.425 secs.
:sdks:java:io:google-cloud-platform:testClasses (Thread[Daemon ****,5,main]) started.

> Task :sdks:java:io:google-cloud-platform:testClasses
Skipping task ':sdks:java:io:google-cloud-platform:testClasses' as it has no actions.
:sdks:java:io:google-cloud-platform:testClasses (Thread[Daemon ****,5,main]) completed. Took 0.0 secs.
:sdks:java:io:google-cloud-platform:testJar (Thread[Daemon ****,5,main]) started.

> Task :runners:google-cloud-dataflow-java:****:legacy-****:compileJava FROM-CACHE
file or directory '<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/runners/google-cloud-dataflow-java/****/legacy-****/src/main/java',> not found
Custom actions are attached to task ':runners:google-cloud-dataflow-java:****:legacy-****:compileJava'.
Build cache key for task ':runners:google-cloud-dataflow-java:****:legacy-****:compileJava' is cf672da70ce14149e9f2f602b34780e4
Task ':runners:google-cloud-dataflow-java:****:legacy-****:compileJava' is not up-to-date because:
  No history is available.
Loaded cache entry for task ':runners:google-cloud-dataflow-java:****:legacy-****:compileJava' with cache key cf672da70ce14149e9f2f602b34780e4
:runners:google-cloud-dataflow-java:****:legacy-****:compileJava (Thread[Execution **** for ':' Thread 3,5,main]) completed. Took 0.275 secs.
:runners:google-cloud-dataflow-java:****:legacy-****:classes (Thread[Execution **** for ':' Thread 3,5,main]) started.

> Task :runners:google-cloud-dataflow-java:****:legacy-****:classes UP-TO-DATE
Skipping task ':runners:google-cloud-dataflow-java:****:legacy-****:classes' as it has no actions.
:runners:google-cloud-dataflow-java:****:legacy-****:classes (Thread[Execution **** for ':' Thread 3,5,main]) completed. Took 0.0 secs.
:runners:google-cloud-dataflow-java:****:legacy-****:shadowJar (Thread[Execution **** for ':' Thread 3,5,main]) started.

> Task :sdks:java:io:google-cloud-platform:testJar
Caching disabled for task ':sdks:java:io:google-cloud-platform:testJar' because:
  Caching has not been enabled for the task
Task ':sdks:java:io:google-cloud-platform:testJar' is not up-to-date because:
  No history is available.
:sdks:java:io:google-cloud-platform:testJar (Thread[Daemon ****,5,main]) completed. Took 0.248 secs.
:runners:google-cloud-dataflow-java:compileTestJava (Thread[Daemon ****,5,main]) started.

> Task :runners:google-cloud-dataflow-java:****:legacy-****:shadowJar FROM-CACHE
Custom actions are attached to task ':runners:google-cloud-dataflow-java:****:legacy-****:shadowJar'.
Build cache key for task ':runners:google-cloud-dataflow-java:****:legacy-****:shadowJar' is 165b57b3de0480886088cb77a0fc5516
Task ':runners:google-cloud-dataflow-java:****:legacy-****:shadowJar' is not up-to-date because:
  No history is available.
Loaded cache entry for task ':runners:google-cloud-dataflow-java:****:legacy-****:shadowJar' with cache key 165b57b3de0480886088cb77a0fc5516
:runners:google-cloud-dataflow-java:****:legacy-****:shadowJar (Thread[Execution **** for ':' Thread 3,5,main]) completed. Took 0.159 secs.

> Task :runners:google-cloud-dataflow-java:compileTestJava FROM-CACHE
Custom actions are attached to task ':runners:google-cloud-dataflow-java:compileTestJava'.
Build cache key for task ':runners:google-cloud-dataflow-java:compileTestJava' is 861c7435ffd549666858392e9bbd7747
Task ':runners:google-cloud-dataflow-java:compileTestJava' is not up-to-date because:
  No history is available.
Loaded cache entry for task ':runners:google-cloud-dataflow-java:compileTestJava' with cache key 861c7435ffd549666858392e9bbd7747
:runners:google-cloud-dataflow-java:compileTestJava (Thread[Daemon ****,5,main]) completed. Took 0.171 secs.
:runners:google-cloud-dataflow-java:testClasses (Thread[Daemon ****,5,main]) started.

> Task :runners:google-cloud-dataflow-java:testClasses UP-TO-DATE
Skipping task ':runners:google-cloud-dataflow-java:testClasses' as it has no actions.
:runners:google-cloud-dataflow-java:testClasses (Thread[Daemon ****,5,main]) completed. Took 0.0 secs.
:runners:google-cloud-dataflow-java:testJar (Thread[Daemon ****,5,main]) started.

> Task :runners:google-cloud-dataflow-java:testJar
Caching disabled for task ':runners:google-cloud-dataflow-java:testJar' because:
  Caching has not been enabled for the task
Task ':runners:google-cloud-dataflow-java:testJar' is not up-to-date because:
  No history is available.
file or directory '<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/runners/google-cloud-dataflow-java/build/resources/test',> not found
:runners:google-cloud-dataflow-java:testJar (Thread[Daemon ****,5,main]) completed. Took 0.029 secs.
:sdks:java:io:file-based-io-tests:integrationTest (Thread[Daemon ****,5,main]) started.
Gradle Test Executor 1 started executing tests.

> Task :sdks:java:io:file-based-io-tests:integrationTest
Custom actions are attached to task ':sdks:java:io:file-based-io-tests:integrationTest'.
Build cache key for task ':sdks:java:io:file-based-io-tests:integrationTest' is d113188b16b5dc9e6e52b7033c284315
Task ':sdks:java:io:file-based-io-tests:integrationTest' is not up-to-date because:
  Task.upToDateWhen is false.
Starting process 'Gradle Test Executor 1'. Working directory: <https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/file-based-io-tests> Command: /usr/lib/jvm/java-8-openjdk-amd64/bin/java -DbeamTestPipelineOptions=["--runner=DataflowRunner","--project=apache-beam-testing","--tempRoot=gs://temp-storage-for-perf-tests","--filenamePrefix=gs://temp-storage-for-perf-tests/beam_PerformanceTests_TFRecordIOIT/5317/","--bigQueryDataset=beam_performance","--bigQueryTable=tfrecordioit_results","--influxMeasurement=tfrecordioit_results","--numberOfRecords=18000000","--expectedHash=543104423f8b6eb097acb9f111c19fe4","--datasetSize=1019380000","--numWorkers=5","--autoscalingAlgorithm=NONE","--influxDatabase=beam_test_metrics","--influxHost=http://10.128.0.96:8086","--****HarnessContainerImage=","--dataflowWorkerJar=<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/runners/google-cloud-dataflow-java/****/legacy-****/build/libs/beam-runners-google-cloud-dataflow-java-legacy-****-2.34.0-SNAPSHOT.jar","--region=us-central1"]> -Djava.security.manager=****.org.gradle.process.internal.****.child.BootstrapSecurityManager -Dorg.gradle.native=false -Xmx2g -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant -ea -cp /home/jenkins/.gradle/caches/6.8.3/****Main/gradle-****.jar ****.org.gradle.process.internal.****.GradleWorkerMain 'Gradle Test Executor 1'
Successfully started process 'Gradle Test Executor 1'

org.apache.beam.sdk.io.tfrecord.TFRecordIOIT STANDARD_ERROR
    SLF4J: Class path contains multiple SLF4J bindings.
    SLF4J: Found binding in [jar:<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/runners/google-cloud-dataflow-java/****/legacy-****/build/libs/beam-runners-google-cloud-dataflow-java-legacy-****-2.34.0-SNAPSHOT.jar!/org/slf4j/impl/StaticLoggerBinder.class]>
    SLF4J: Found binding in [jar:file:/home/jenkins/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-log4j12/1.7.30/c21f55139d8141d2231214fb1feaf50a1edca95e/slf4j-log4j12-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class]
    SLF4J: Found binding in [jar:file:/home/jenkins/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-jdk14/1.7.30/d35953dd2fe54ebe39fdf18cfd82fe6eb35b25ed/slf4j-jdk14-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class]
    SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
    SLF4J: Actual binding is of type [org.slf4j.impl.JDK14LoggerFactory]

org.apache.beam.sdk.io.tfrecord.TFRecordIOIT > writeThenReadAll STANDARD_ERROR
    Sep 13, 2021 6:21:52 AM org.apache.beam.runners.dataflow.DataflowRunner validateSdkContainerImageOptions
    WARNING: Prefer --sdkContainerImage over deprecated legacy option --****HarnessContainerImage.
    Sep 13, 2021 6:21:53 AM org.apache.beam.sdk.extensions.gcp.options.GcpOptions$GcpTempLocationFactory tryCreateDefaultBucket
    INFO: No tempLocation specified, attempting to use default bucket: dataflow-staging-us-central1-844138762903
    Sep 13, 2021 6:21:54 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
    WARNING: Request failed with code 409, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/storage/v1/b?predefinedAcl=projectPrivate&predefinedDefaultObjectAcl=projectPrivate&project=apache-beam-testing. 
    Sep 13, 2021 6:21:54 AM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
    INFO: No stagingLocation provided, falling back to gcpTempLocation
    Sep 13, 2021 6:21:54 AM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
    INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 317 files. Enable logging at DEBUG level to see which files will be staged.
    Sep 13, 2021 6:21:54 AM org.apache.beam.runners.dataflow.DataflowRunner run
    INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
    Sep 13, 2021 6:21:57 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
    INFO: Uploading 318 files from PipelineOptions.filesToStage to staging location to prepare for execution.
    Sep 13, 2021 6:21:58 AM org.apache.beam.runners.dataflow.util.PackageUtil$PackageAttributes forFileToStage
    INFO: Staging custom dataflow-****.jar as beam-runners-google-cloud-dataflow-java-legacy-****-2.34.0-SNAPSHOT-YtvkOIsfQYfFzG89aVBO_8oNHoiij9SynwUJLdMkn5A.jar
    Sep 13, 2021 6:21:58 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
    INFO: Uploading /tmp/test3335254973028486526.zip to gs://dataflow-staging-us-central1-844138762903/temp/staging/test-Rtqv9PiExTh-m1JJfn_mbfcCSRlKNPqigkGqJikYPeA.jar
    Sep 13, 2021 6:21:58 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
    INFO: Staging files complete: 317 files cached, 1 files newly uploaded in 0 seconds
    Sep 13, 2021 6:21:58 AM org.apache.beam.runners.dataflow.DataflowRunner run
    INFO: Staging portable pipeline proto to gs://dataflow-staging-us-central1-844138762903/temp/staging/
    Sep 13, 2021 6:21:58 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
    INFO: Uploading <205290 bytes, hash a5e95a8e577993f02a9271d721ede350caf87bf65e8c4d6b04b2df35743fd5ff> to gs://dataflow-staging-us-central1-844138762903/temp/staging/pipeline-pelajld5k_AqknHXIe3jUMr4e_ZejE1rBLLfNXQ_1f8.pb
    Sep 13, 2021 6:22:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Generate sequence/Read(BoundedCountingSource) as step s1
    Sep 13, 2021 6:22:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Produce text lines as step s2
    Sep 13, 2021 6:22:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Transform strings to bytes/Map as step s3
    Sep 13, 2021 6:22:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Record time before writing as step s4
    Sep 13, 2021 6:22:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/RewindowIntoGlobal/Window.Assign as step s5
    Sep 13, 2021 6:22:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles as step s6
    Sep 13, 2021 6:22:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten as step s7
    Sep 13, 2021 6:22:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/WriteUnshardedBundlesToTempFiles/WriteUnwritten as step s8
    Sep 13, 2021 6:22:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/WriteUnshardedBundlesToTempFiles/DropShardNum as step s9
    Sep 13, 2021 6:22:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/WriteUnshardedBundlesToTempFiles/Flatten.PCollections as step s10
    Sep 13, 2021 6:22:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow) as step s11
    Sep 13, 2021 6:22:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/GatherTempFileResults/View.AsList/CreateDataflowView as step s12
    Sep 13, 2021 6:22:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Create.Values/Read(CreateSource) as step s13
    Sep 13, 2021 6:22:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Reify.ReifyView/ParDo(Anonymous) as step s14
    Sep 13, 2021 6:22:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Values/Values/Map as step s15
    Sep 13, 2021 6:22:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Finalize as step s16
    Sep 13, 2021 6:22:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random key as step s17
    Sep 13, 2021 6:22:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign as step s18
    Sep 13, 2021 6:22:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey as step s19
    Sep 13, 2021 6:22:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable as step s20
    Sep 13, 2021 6:22:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Values/Values/Map as step s21
    Sep 13, 2021 6:22:00 AM org.apache.beam.runners.dataflow.DataflowRunner run
    INFO: Dataflow SDK version: 2.34.0-SNAPSHOT
    Sep 13, 2021 6:22:01 AM org.apache.beam.runners.dataflow.DataflowRunner run
    INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-09-12_23_22_00-12179573304408372732?project=apache-beam-testing
    Sep 13, 2021 6:22:01 AM org.apache.beam.runners.dataflow.DataflowRunner run
    INFO: Submitted job: 2021-09-12_23_22_00-12179573304408372732
    Sep 13, 2021 6:22:01 AM org.apache.beam.runners.dataflow.DataflowRunner run
    INFO: To cancel the job using the 'gcloud' tool, run:
    > gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-09-12_23_22_00-12179573304408372732
    Sep 13, 2021 6:22:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-13T06:22:19.719Z: Worker configuration: e2-standard-2 in us-central1-a.
    Sep 13, 2021 6:22:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    SEVERE: 2021-09-13T06:22:20.724Z: Workflow failed. Causes: Project apache-beam-testing has insufficient quota(s) to execute this workflow with 1 instances in region us-central1. Quota summary (required/available): 1/24386 instances, 2/0 CPUs, 25/182241 disk GB, 0/2397 SSD disk GB, 1/283 instance groups, 1/286 managed instance groups, 1/512 instance templates, 1/615 in-use IP addresses.

    Please see https://cloud.google.com/compute/docs/resource-quotas about requesting more quota.
    Sep 13, 2021 6:22:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-13T06:22:20.771Z: Cleaning up.
    Sep 13, 2021 6:22:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-13T06:22:20.820Z: Worker pool stopped.
    Sep 13, 2021 6:22:24 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-13T06:22:22.035Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
    Sep 13, 2021 6:22:25 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
    INFO: Job 2021-09-12_23_22_00-12179573304408372732 failed with status FAILED.

Gradle Test Executor 1 finished executing tests.

> Task :sdks:java:io:file-based-io-tests:integrationTest FAILED

org.apache.beam.sdk.io.tfrecord.TFRecordIOIT > writeThenReadAll FAILED
    java.lang.IllegalStateException: Unable to find any files matching gs://temp-storage-for-perf-tests/beam_PerformanceTests_TFRecordIOIT/5317/_1631514111983*
        at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:588)
        at org.apache.beam.sdk.io.TFRecordIO$Read.expand(TFRecordIO.java:191)
        at org.apache.beam.sdk.io.TFRecordIO$Read.expand(TFRecordIO.java:112)
        at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:548)
        at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:482)
        at org.apache.beam.sdk.values.PBegin.apply(PBegin.java:44)
        at org.apache.beam.sdk.Pipeline.apply(Pipeline.java:177)
        at org.apache.beam.sdk.io.tfrecord.TFRecordIOIT.writeThenReadAll(TFRecordIOIT.java:151)

1 test completed, 1 failed
Finished generating test XML results (0.02 secs) into: <https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/file-based-io-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.033 secs) into: <https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/file-based-io-tests/build/reports/tests/integrationTest>
:sdks:java:io:file-based-io-tests:integrationTest (Thread[Daemon ****,5,main]) completed. Took 37.316 secs.

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:io:file-based-io-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/file-based-io-tests/build/reports/tests/integrationTest/index.html>

* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 1m 5s
129 actionable tasks: 79 executed, 50 from cache

Publishing build scan...
https://gradle.com/s/d5ebmbrnxmlh6

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_PerformanceTests_TFRecordIOIT #5316

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/5316/display/redirect>

Changes:


------------------------------------------
[...truncated 274.02 KB...]
Task ':sdks:java:io:google-cloud-platform:jar' is not up-to-date because:
  No history is available.
file or directory '<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/google-cloud-platform/build/resources/main',> not found
:sdks:java:io:google-cloud-platform:jar (Thread[Execution **** for ':' Thread 5,5,main]) completed. Took 0.31 secs.
:runners:google-cloud-dataflow-java:compileJava (Thread[Execution **** for ':' Thread 5,5,main]) started.
:sdks:java:io:google-cloud-platform:compileTestJava (Thread[Execution **** for ':' Thread 11,5,main]) started.

> Task :runners:google-cloud-dataflow-java:compileJava FROM-CACHE
Custom actions are attached to task ':runners:google-cloud-dataflow-java:compileJava'.
Build cache key for task ':runners:google-cloud-dataflow-java:compileJava' is f4d45dae85b41e621c919183804b6afa
Task ':runners:google-cloud-dataflow-java:compileJava' is not up-to-date because:
  No history is available.
Loaded cache entry for task ':runners:google-cloud-dataflow-java:compileJava' with cache key f4d45dae85b41e621c919183804b6afa
:runners:google-cloud-dataflow-java:compileJava (Thread[Execution **** for ':' Thread 5,5,main]) completed. Took 0.212 secs.
:runners:google-cloud-dataflow-java:classes (Thread[Execution **** for ':' Thread 5,5,main]) started.

> Task :runners:google-cloud-dataflow-java:classes
Skipping task ':runners:google-cloud-dataflow-java:classes' as it has no actions.
:runners:google-cloud-dataflow-java:classes (Thread[Execution **** for ':' Thread 5,5,main]) completed. Took 0.0 secs.
:runners:google-cloud-dataflow-java:jar (Thread[Execution **** for ':' Thread 4,5,main]) started.

> Task :runners:google-cloud-dataflow-java:jar
Caching disabled for task ':runners:google-cloud-dataflow-java:jar' because:
  Caching has not been enabled for the task
Task ':runners:google-cloud-dataflow-java:jar' is not up-to-date because:
  No history is available.
:runners:google-cloud-dataflow-java:jar (Thread[Execution **** for ':' Thread 4,5,main]) completed. Took 0.07 secs.
:runners:google-cloud-dataflow-java:****:legacy-****:compileJava (Thread[Execution **** for ':' Thread 4,5,main]) started.

> Task :sdks:java:io:google-cloud-platform:compileTestJava FROM-CACHE
Custom actions are attached to task ':sdks:java:io:google-cloud-platform:compileTestJava'.
Build cache key for task ':sdks:java:io:google-cloud-platform:compileTestJava' is 99462c52f9bbbb5d949b676d07ee3bc7
Task ':sdks:java:io:google-cloud-platform:compileTestJava' is not up-to-date because:
  No history is available.
Loaded cache entry for task ':sdks:java:io:google-cloud-platform:compileTestJava' with cache key 99462c52f9bbbb5d949b676d07ee3bc7
:sdks:java:io:google-cloud-platform:compileTestJava (Thread[Execution **** for ':' Thread 11,5,main]) completed. Took 0.563 secs.
:sdks:java:io:google-cloud-platform:testClasses (Thread[Execution **** for ':' Thread 11,5,main]) started.

> Task :sdks:java:io:google-cloud-platform:testClasses
Skipping task ':sdks:java:io:google-cloud-platform:testClasses' as it has no actions.
:sdks:java:io:google-cloud-platform:testClasses (Thread[Execution **** for ':' Thread 11,5,main]) completed. Took 0.0 secs.
:sdks:java:io:google-cloud-platform:testJar (Thread[Execution **** for ':' Thread 2,5,main]) started.

> Task :runners:google-cloud-dataflow-java:****:legacy-****:compileJava FROM-CACHE
file or directory '<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/runners/google-cloud-dataflow-java/****/legacy-****/src/main/java',> not found
Custom actions are attached to task ':runners:google-cloud-dataflow-java:****:legacy-****:compileJava'.
Build cache key for task ':runners:google-cloud-dataflow-java:****:legacy-****:compileJava' is cf672da70ce14149e9f2f602b34780e4
Task ':runners:google-cloud-dataflow-java:****:legacy-****:compileJava' is not up-to-date because:
  No history is available.
Loaded cache entry for task ':runners:google-cloud-dataflow-java:****:legacy-****:compileJava' with cache key cf672da70ce14149e9f2f602b34780e4
:runners:google-cloud-dataflow-java:****:legacy-****:compileJava (Thread[Execution **** for ':' Thread 4,5,main]) completed. Took 0.356 secs.
:runners:google-cloud-dataflow-java:****:legacy-****:classes (Thread[Execution **** for ':' Thread 4,5,main]) started.

> Task :runners:google-cloud-dataflow-java:****:legacy-****:classes UP-TO-DATE
Skipping task ':runners:google-cloud-dataflow-java:****:legacy-****:classes' as it has no actions.
:runners:google-cloud-dataflow-java:****:legacy-****:classes (Thread[Execution **** for ':' Thread 4,5,main]) completed. Took 0.0 secs.
:runners:google-cloud-dataflow-java:****:legacy-****:shadowJar (Thread[Execution **** for ':' Thread 4,5,main]) started.

> Task :runners:google-cloud-dataflow-java:****:legacy-****:shadowJar FROM-CACHE
Custom actions are attached to task ':runners:google-cloud-dataflow-java:****:legacy-****:shadowJar'.
Build cache key for task ':runners:google-cloud-dataflow-java:****:legacy-****:shadowJar' is b03e31546baded1569b9e22785f539a5
Task ':runners:google-cloud-dataflow-java:****:legacy-****:shadowJar' is not up-to-date because:
  No history is available.
Loaded cache entry for task ':runners:google-cloud-dataflow-java:****:legacy-****:shadowJar' with cache key b03e31546baded1569b9e22785f539a5
:runners:google-cloud-dataflow-java:****:legacy-****:shadowJar (Thread[Execution **** for ':' Thread 4,5,main]) completed. Took 0.162 secs.

> Task :sdks:java:io:google-cloud-platform:testJar
Caching disabled for task ':sdks:java:io:google-cloud-platform:testJar' because:
  Caching has not been enabled for the task
Task ':sdks:java:io:google-cloud-platform:testJar' is not up-to-date because:
  No history is available.
:sdks:java:io:google-cloud-platform:testJar (Thread[Execution **** for ':' Thread 2,5,main]) completed. Took 0.257 secs.
:runners:google-cloud-dataflow-java:compileTestJava (Thread[Execution **** for ':' Thread 2,5,main]) started.

> Task :runners:google-cloud-dataflow-java:compileTestJava FROM-CACHE
Custom actions are attached to task ':runners:google-cloud-dataflow-java:compileTestJava'.
Build cache key for task ':runners:google-cloud-dataflow-java:compileTestJava' is 861c7435ffd549666858392e9bbd7747
Task ':runners:google-cloud-dataflow-java:compileTestJava' is not up-to-date because:
  No history is available.
Loaded cache entry for task ':runners:google-cloud-dataflow-java:compileTestJava' with cache key 861c7435ffd549666858392e9bbd7747
:runners:google-cloud-dataflow-java:compileTestJava (Thread[Execution **** for ':' Thread 2,5,main]) completed. Took 0.187 secs.
:runners:google-cloud-dataflow-java:testClasses (Thread[Execution **** for ':' Thread 2,5,main]) started.

> Task :runners:google-cloud-dataflow-java:testClasses UP-TO-DATE
Skipping task ':runners:google-cloud-dataflow-java:testClasses' as it has no actions.
:runners:google-cloud-dataflow-java:testClasses (Thread[Execution **** for ':' Thread 2,5,main]) completed. Took 0.0 secs.
:runners:google-cloud-dataflow-java:testJar (Thread[Execution **** for ':' Thread 2,5,main]) started.

> Task :runners:google-cloud-dataflow-java:testJar
Caching disabled for task ':runners:google-cloud-dataflow-java:testJar' because:
  Caching has not been enabled for the task
Task ':runners:google-cloud-dataflow-java:testJar' is not up-to-date because:
  No history is available.
file or directory '<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/runners/google-cloud-dataflow-java/build/resources/test',> not found
:runners:google-cloud-dataflow-java:testJar (Thread[Execution **** for ':' Thread 2,5,main]) completed. Took 0.03 secs.
:sdks:java:io:file-based-io-tests:integrationTest (Thread[Execution **** for ':' Thread 2,5,main]) started.
Gradle Test Executor 1 started executing tests.

> Task :sdks:java:io:file-based-io-tests:integrationTest
Custom actions are attached to task ':sdks:java:io:file-based-io-tests:integrationTest'.
Build cache key for task ':sdks:java:io:file-based-io-tests:integrationTest' is 2954f27e833e06d9b6038317a4b1cf00
Task ':sdks:java:io:file-based-io-tests:integrationTest' is not up-to-date because:
  Task.upToDateWhen is false.
Starting process 'Gradle Test Executor 1'. Working directory: <https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/file-based-io-tests> Command: /usr/lib/jvm/java-8-openjdk-amd64/bin/java -DbeamTestPipelineOptions=["--runner=DataflowRunner","--project=apache-beam-testing","--tempRoot=gs://temp-storage-for-perf-tests","--filenamePrefix=gs://temp-storage-for-perf-tests/beam_PerformanceTests_TFRecordIOIT/5316/","--bigQueryDataset=beam_performance","--bigQueryTable=tfrecordioit_results","--influxMeasurement=tfrecordioit_results","--numberOfRecords=18000000","--expectedHash=543104423f8b6eb097acb9f111c19fe4","--datasetSize=1019380000","--numWorkers=5","--autoscalingAlgorithm=NONE","--influxDatabase=beam_test_metrics","--influxHost=http://10.128.0.96:8086","--****HarnessContainerImage=","--dataflowWorkerJar=<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/runners/google-cloud-dataflow-java/****/legacy-****/build/libs/beam-runners-google-cloud-dataflow-java-legacy-****-2.34.0-SNAPSHOT.jar","--region=us-central1"]> -Djava.security.manager=****.org.gradle.process.internal.****.child.BootstrapSecurityManager -Dorg.gradle.native=false -Xmx2g -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant -ea -cp /home/jenkins/.gradle/caches/6.8.3/****Main/gradle-****.jar ****.org.gradle.process.internal.****.GradleWorkerMain 'Gradle Test Executor 1'
Successfully started process 'Gradle Test Executor 1'

org.apache.beam.sdk.io.tfrecord.TFRecordIOIT STANDARD_ERROR
    SLF4J: Class path contains multiple SLF4J bindings.
    SLF4J: Found binding in [jar:<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/runners/google-cloud-dataflow-java/****/legacy-****/build/libs/beam-runners-google-cloud-dataflow-java-legacy-****-2.34.0-SNAPSHOT.jar!/org/slf4j/impl/StaticLoggerBinder.class]>
    SLF4J: Found binding in [jar:file:/home/jenkins/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-log4j12/1.7.30/c21f55139d8141d2231214fb1feaf50a1edca95e/slf4j-log4j12-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class]
    SLF4J: Found binding in [jar:file:/home/jenkins/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-jdk14/1.7.30/d35953dd2fe54ebe39fdf18cfd82fe6eb35b25ed/slf4j-jdk14-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class]
    SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
    SLF4J: Actual binding is of type [org.slf4j.impl.JDK14LoggerFactory]

org.apache.beam.sdk.io.tfrecord.TFRecordIOIT > writeThenReadAll STANDARD_ERROR
    Sep 13, 2021 12:21:58 AM org.apache.beam.runners.dataflow.DataflowRunner validateSdkContainerImageOptions
    WARNING: Prefer --sdkContainerImage over deprecated legacy option --****HarnessContainerImage.
    Sep 13, 2021 12:21:59 AM org.apache.beam.sdk.extensions.gcp.options.GcpOptions$GcpTempLocationFactory tryCreateDefaultBucket
    INFO: No tempLocation specified, attempting to use default bucket: dataflow-staging-us-central1-844138762903
    Sep 13, 2021 12:22:00 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
    WARNING: Request failed with code 409, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/storage/v1/b?predefinedAcl=projectPrivate&predefinedDefaultObjectAcl=projectPrivate&project=apache-beam-testing. 
    Sep 13, 2021 12:22:00 AM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
    INFO: No stagingLocation provided, falling back to gcpTempLocation
    Sep 13, 2021 12:22:01 AM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
    INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 317 files. Enable logging at DEBUG level to see which files will be staged.
    Sep 13, 2021 12:22:01 AM org.apache.beam.runners.dataflow.DataflowRunner run
    INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
    Sep 13, 2021 12:22:03 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
    INFO: Uploading 318 files from PipelineOptions.filesToStage to staging location to prepare for execution.
    Sep 13, 2021 12:22:04 AM org.apache.beam.runners.dataflow.util.PackageUtil$PackageAttributes forFileToStage
    INFO: Staging custom dataflow-****.jar as beam-runners-google-cloud-dataflow-java-legacy-****-2.34.0-SNAPSHOT-YtvkOIsfQYfFzG89aVBO_8oNHoiij9SynwUJLdMkn5A.jar
    Sep 13, 2021 12:22:04 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
    INFO: Uploading /tmp/test4853974046509510044.zip to gs://dataflow-staging-us-central1-844138762903/temp/staging/test-W6zEZYHrEh76zsPNy0TTDbfGzQjj4qPhKGDDhtXsamI.jar
    Sep 13, 2021 12:22:04 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
    INFO: Staging files complete: 317 files cached, 1 files newly uploaded in 0 seconds
    Sep 13, 2021 12:22:04 AM org.apache.beam.runners.dataflow.DataflowRunner run
    INFO: Staging portable pipeline proto to gs://dataflow-staging-us-central1-844138762903/temp/staging/
    Sep 13, 2021 12:22:04 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
    INFO: Uploading <205290 bytes, hash c6becc7616e0bba127b325fa046767c61db3491c4c4453188475f59e18f3205b> to gs://dataflow-staging-us-central1-844138762903/temp/staging/pipeline-xr7Mdhbgu6EnsyX6BGdnxh2zSRxMRFMYhHX1nhjzIFs.pb
    Sep 13, 2021 12:22:07 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Generate sequence/Read(BoundedCountingSource) as step s1
    Sep 13, 2021 12:22:07 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Produce text lines as step s2
    Sep 13, 2021 12:22:07 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Transform strings to bytes/Map as step s3
    Sep 13, 2021 12:22:07 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Record time before writing as step s4
    Sep 13, 2021 12:22:07 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/RewindowIntoGlobal/Window.Assign as step s5
    Sep 13, 2021 12:22:07 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles as step s6
    Sep 13, 2021 12:22:07 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten as step s7
    Sep 13, 2021 12:22:07 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/WriteUnshardedBundlesToTempFiles/WriteUnwritten as step s8
    Sep 13, 2021 12:22:07 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/WriteUnshardedBundlesToTempFiles/DropShardNum as step s9
    Sep 13, 2021 12:22:07 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/WriteUnshardedBundlesToTempFiles/Flatten.PCollections as step s10
    Sep 13, 2021 12:22:07 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow) as step s11
    Sep 13, 2021 12:22:07 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/GatherTempFileResults/View.AsList/CreateDataflowView as step s12
    Sep 13, 2021 12:22:07 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Create.Values/Read(CreateSource) as step s13
    Sep 13, 2021 12:22:07 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Reify.ReifyView/ParDo(Anonymous) as step s14
    Sep 13, 2021 12:22:07 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Values/Values/Map as step s15
    Sep 13, 2021 12:22:07 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Finalize as step s16
    Sep 13, 2021 12:22:07 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random key as step s17
    Sep 13, 2021 12:22:07 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign as step s18
    Sep 13, 2021 12:22:07 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey as step s19
    Sep 13, 2021 12:22:07 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable as step s20
    Sep 13, 2021 12:22:07 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Values/Values/Map as step s21
    Sep 13, 2021 12:22:07 AM org.apache.beam.runners.dataflow.DataflowRunner run
    INFO: Dataflow SDK version: 2.34.0-SNAPSHOT
    Sep 13, 2021 12:22:08 AM org.apache.beam.runners.dataflow.DataflowRunner run
    INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-09-12_17_22_07-13751808236222957809?project=apache-beam-testing
    Sep 13, 2021 12:22:08 AM org.apache.beam.runners.dataflow.DataflowRunner run
    INFO: Submitted job: 2021-09-12_17_22_07-13751808236222957809
    Sep 13, 2021 12:22:08 AM org.apache.beam.runners.dataflow.DataflowRunner run
    INFO: To cancel the job using the 'gcloud' tool, run:
    > gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-09-12_17_22_07-13751808236222957809
    Sep 13, 2021 12:22:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-13T00:22:22.844Z: Worker configuration: e2-standard-2 in us-central1-a.
    Sep 13, 2021 12:22:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    SEVERE: 2021-09-13T00:22:23.645Z: Workflow failed. Causes: Project apache-beam-testing has insufficient quota(s) to execute this workflow with 1 instances in region us-central1. Quota summary (required/available): 1/24386 instances, 2/0 CPUs, 25/183266 disk GB, 0/2397 SSD disk GB, 1/277 instance groups, 1/280 managed instance groups, 1/499 instance templates, 1/615 in-use IP addresses.

    Please see https://cloud.google.com/compute/docs/resource-quotas about requesting more quota.
    Sep 13, 2021 12:22:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-13T00:22:23.704Z: Cleaning up.
    Sep 13, 2021 12:22:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-13T00:22:23.755Z: Worker pool stopped.
    Sep 13, 2021 12:22:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-13T00:22:24.925Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
    Sep 13, 2021 12:22:29 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
    INFO: Job 2021-09-12_17_22_07-13751808236222957809 failed with status FAILED.

Gradle Test Executor 1 finished executing tests.

> Task :sdks:java:io:file-based-io-tests:integrationTest FAILED

org.apache.beam.sdk.io.tfrecord.TFRecordIOIT > writeThenReadAll FAILED
    java.lang.IllegalStateException: Unable to find any files matching gs://temp-storage-for-perf-tests/beam_PerformanceTests_TFRecordIOIT/5316/_1631492517764*
        at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:588)
        at org.apache.beam.sdk.io.TFRecordIO$Read.expand(TFRecordIO.java:191)
        at org.apache.beam.sdk.io.TFRecordIO$Read.expand(TFRecordIO.java:112)
        at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:548)
        at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:482)
        at org.apache.beam.sdk.values.PBegin.apply(PBegin.java:44)
        at org.apache.beam.sdk.Pipeline.apply(Pipeline.java:177)
        at org.apache.beam.sdk.io.tfrecord.TFRecordIOIT.writeThenReadAll(TFRecordIOIT.java:151)

1 test completed, 1 failed
Finished generating test XML results (0.021 secs) into: <https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/file-based-io-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.028 secs) into: <https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/file-based-io-tests/build/reports/tests/integrationTest>
:sdks:java:io:file-based-io-tests:integrationTest (Thread[Execution **** for ':' Thread 2,5,main]) completed. Took 35.809 secs.

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:io:file-based-io-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/file-based-io-tests/build/reports/tests/integrationTest/index.html>

* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 1m 6s
129 actionable tasks: 79 executed, 50 from cache

Publishing build scan...
https://gradle.com/s/2iz2rtmrjwvba

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_PerformanceTests_TFRecordIOIT #5315

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/5315/display/redirect>

Changes:


------------------------------------------
[...truncated 285.27 KB...]
  No history is available.
Loaded cache entry for task ':runners:google-cloud-dataflow-java:****:legacy-****:shadowJar' with cache key b03e31546baded1569b9e22785f539a5
:runners:google-cloud-dataflow-java:****:legacy-****:shadowJar (Thread[Execution **** for ':' Thread 9,5,main]) completed. Took 0.167 secs.

> Task :runners:google-cloud-dataflow-java:compileTestJava FROM-CACHE
Custom actions are attached to task ':runners:google-cloud-dataflow-java:compileTestJava'.
Build cache key for task ':runners:google-cloud-dataflow-java:compileTestJava' is 861c7435ffd549666858392e9bbd7747
Task ':runners:google-cloud-dataflow-java:compileTestJava' is not up-to-date because:
  No history is available.
Loaded cache entry for task ':runners:google-cloud-dataflow-java:compileTestJava' with cache key 861c7435ffd549666858392e9bbd7747
:runners:google-cloud-dataflow-java:compileTestJava (Thread[Execution **** for ':' Thread 2,5,main]) completed. Took 0.189 secs.
:runners:google-cloud-dataflow-java:testClasses (Thread[Execution **** for ':' Thread 2,5,main]) started.

> Task :runners:google-cloud-dataflow-java:testClasses UP-TO-DATE
Skipping task ':runners:google-cloud-dataflow-java:testClasses' as it has no actions.
:runners:google-cloud-dataflow-java:testClasses (Thread[Execution **** for ':' Thread 2,5,main]) completed. Took 0.0 secs.
:runners:google-cloud-dataflow-java:testJar (Thread[Execution **** for ':' Thread 2,5,main]) started.

> Task :runners:google-cloud-dataflow-java:testJar
Caching disabled for task ':runners:google-cloud-dataflow-java:testJar' because:
  Caching has not been enabled for the task
Task ':runners:google-cloud-dataflow-java:testJar' is not up-to-date because:
  No history is available.
file or directory '<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/runners/google-cloud-dataflow-java/build/resources/test',> not found
:runners:google-cloud-dataflow-java:testJar (Thread[Execution **** for ':' Thread 2,5,main]) completed. Took 0.034 secs.
:sdks:java:io:file-based-io-tests:integrationTest (Thread[Execution **** for ':' Thread 2,5,main]) started.
Gradle Test Executor 1 started executing tests.

> Task :sdks:java:io:file-based-io-tests:integrationTest
Custom actions are attached to task ':sdks:java:io:file-based-io-tests:integrationTest'.
Build cache key for task ':sdks:java:io:file-based-io-tests:integrationTest' is 2954f27e833e06d9b6038317a4b1cf00
Task ':sdks:java:io:file-based-io-tests:integrationTest' is not up-to-date because:
  Task.upToDateWhen is false.
Starting process 'Gradle Test Executor 1'. Working directory: <https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/file-based-io-tests> Command: /usr/lib/jvm/java-8-openjdk-amd64/bin/java -DbeamTestPipelineOptions=["--runner=DataflowRunner","--project=apache-beam-testing","--tempRoot=gs://temp-storage-for-perf-tests","--filenamePrefix=gs://temp-storage-for-perf-tests/beam_PerformanceTests_TFRecordIOIT/5315/","--bigQueryDataset=beam_performance","--bigQueryTable=tfrecordioit_results","--influxMeasurement=tfrecordioit_results","--numberOfRecords=18000000","--expectedHash=543104423f8b6eb097acb9f111c19fe4","--datasetSize=1019380000","--numWorkers=5","--autoscalingAlgorithm=NONE","--influxDatabase=beam_test_metrics","--influxHost=http://10.128.0.96:8086","--****HarnessContainerImage=","--dataflowWorkerJar=<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/runners/google-cloud-dataflow-java/****/legacy-****/build/libs/beam-runners-google-cloud-dataflow-java-legacy-****-2.34.0-SNAPSHOT.jar","--region=us-central1"]> -Djava.security.manager=****.org.gradle.process.internal.****.child.BootstrapSecurityManager -Dorg.gradle.native=false -Xmx2g -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant -ea -cp /home/jenkins/.gradle/caches/6.8.3/****Main/gradle-****.jar ****.org.gradle.process.internal.****.GradleWorkerMain 'Gradle Test Executor 1'
Successfully started process 'Gradle Test Executor 1'

org.apache.beam.sdk.io.tfrecord.TFRecordIOIT STANDARD_ERROR
    SLF4J: Class path contains multiple SLF4J bindings.
    SLF4J: Found binding in [jar:<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/runners/google-cloud-dataflow-java/****/legacy-****/build/libs/beam-runners-google-cloud-dataflow-java-legacy-****-2.34.0-SNAPSHOT.jar!/org/slf4j/impl/StaticLoggerBinder.class]>
    SLF4J: Found binding in [jar:file:/home/jenkins/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-log4j12/1.7.30/c21f55139d8141d2231214fb1feaf50a1edca95e/slf4j-log4j12-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class]
    SLF4J: Found binding in [jar:file:/home/jenkins/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-jdk14/1.7.30/d35953dd2fe54ebe39fdf18cfd82fe6eb35b25ed/slf4j-jdk14-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class]
    SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
    SLF4J: Actual binding is of type [org.slf4j.impl.JDK14LoggerFactory]

org.apache.beam.sdk.io.tfrecord.TFRecordIOIT > writeThenReadAll STANDARD_ERROR
    Sep 12, 2021 6:21:55 PM org.apache.beam.runners.dataflow.DataflowRunner validateSdkContainerImageOptions
    WARNING: Prefer --sdkContainerImage over deprecated legacy option --****HarnessContainerImage.
    Sep 12, 2021 6:21:56 PM org.apache.beam.sdk.extensions.gcp.options.GcpOptions$GcpTempLocationFactory tryCreateDefaultBucket
    INFO: No tempLocation specified, attempting to use default bucket: dataflow-staging-us-central1-844138762903
    Sep 12, 2021 6:21:56 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
    WARNING: Request failed with code 409, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/storage/v1/b?predefinedAcl=projectPrivate&predefinedDefaultObjectAcl=projectPrivate&project=apache-beam-testing. 
    Sep 12, 2021 6:21:57 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
    INFO: No stagingLocation provided, falling back to gcpTempLocation
    Sep 12, 2021 6:21:57 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
    INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 317 files. Enable logging at DEBUG level to see which files will be staged.
    Sep 12, 2021 6:21:57 PM org.apache.beam.runners.dataflow.DataflowRunner run
    INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
    Sep 12, 2021 6:22:00 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
    INFO: Uploading 318 files from PipelineOptions.filesToStage to staging location to prepare for execution.
    Sep 12, 2021 6:22:00 PM org.apache.beam.runners.dataflow.util.PackageUtil$PackageAttributes forFileToStage
    INFO: Staging custom dataflow-****.jar as beam-runners-google-cloud-dataflow-java-legacy-****-2.34.0-SNAPSHOT-YtvkOIsfQYfFzG89aVBO_8oNHoiij9SynwUJLdMkn5A.jar
    Sep 12, 2021 6:22:00 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
    INFO: Uploading /tmp/test4636084343195834462.zip to gs://dataflow-staging-us-central1-844138762903/temp/staging/test-SfKgTolV9uAlHr48wzNsM_D5JMVkHrA47Ff4EKQp1AE.jar
    Sep 12, 2021 6:22:01 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
    INFO: Staging files complete: 317 files cached, 1 files newly uploaded in 0 seconds
    Sep 12, 2021 6:22:01 PM org.apache.beam.runners.dataflow.DataflowRunner run
    INFO: Staging portable pipeline proto to gs://dataflow-staging-us-central1-844138762903/temp/staging/
    Sep 12, 2021 6:22:01 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
    INFO: Uploading <205292 bytes, hash b1449c885b1916c664f06e773a10d89ca4f130f617621263f8663c240cd77b42> to gs://dataflow-staging-us-central1-844138762903/temp/staging/pipeline-sUSciFsZFsZk8G53OhDYnKTxMPYXYhJj-GY8JAzXe0I.pb
    Sep 12, 2021 6:22:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Generate sequence/Read(BoundedCountingSource) as step s1
    Sep 12, 2021 6:22:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Produce text lines as step s2
    Sep 12, 2021 6:22:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Transform strings to bytes/Map as step s3
    Sep 12, 2021 6:22:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Record time before writing as step s4
    Sep 12, 2021 6:22:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/RewindowIntoGlobal/Window.Assign as step s5
    Sep 12, 2021 6:22:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles as step s6
    Sep 12, 2021 6:22:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten as step s7
    Sep 12, 2021 6:22:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/WriteUnshardedBundlesToTempFiles/WriteUnwritten as step s8
    Sep 12, 2021 6:22:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/WriteUnshardedBundlesToTempFiles/DropShardNum as step s9
    Sep 12, 2021 6:22:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/WriteUnshardedBundlesToTempFiles/Flatten.PCollections as step s10
    Sep 12, 2021 6:22:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow) as step s11
    Sep 12, 2021 6:22:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/GatherTempFileResults/View.AsList/CreateDataflowView as step s12
    Sep 12, 2021 6:22:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Create.Values/Read(CreateSource) as step s13
    Sep 12, 2021 6:22:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Reify.ReifyView/ParDo(Anonymous) as step s14
    Sep 12, 2021 6:22:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Values/Values/Map as step s15
    Sep 12, 2021 6:22:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Finalize as step s16
    Sep 12, 2021 6:22:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random key as step s17
    Sep 12, 2021 6:22:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign as step s18
    Sep 12, 2021 6:22:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey as step s19
    Sep 12, 2021 6:22:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable as step s20
    Sep 12, 2021 6:22:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Values/Values/Map as step s21
    Sep 12, 2021 6:22:03 PM org.apache.beam.runners.dataflow.DataflowRunner run
    INFO: Dataflow SDK version: 2.34.0-SNAPSHOT
    Sep 12, 2021 6:22:04 PM org.apache.beam.runners.dataflow.DataflowRunner run
    INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-09-12_11_22_03-11495571717971727843?project=apache-beam-testing
    Sep 12, 2021 6:22:04 PM org.apache.beam.runners.dataflow.DataflowRunner run
    INFO: Submitted job: 2021-09-12_11_22_03-11495571717971727843
    Sep 12, 2021 6:22:04 PM org.apache.beam.runners.dataflow.DataflowRunner run
    INFO: To cancel the job using the 'gcloud' tool, run:
    > gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-09-12_11_22_03-11495571717971727843
    Sep 12, 2021 6:22:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-12T18:22:27.431Z: Worker configuration: e2-standard-2 in us-central1-c.
    Sep 12, 2021 6:22:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-12T18:22:28.177Z: Expanding CoGroupByKey operations into optimizable parts.
    Sep 12, 2021 6:22:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-12T18:22:28.283Z: Expanding GroupByKey operations into optimizable parts.
    Sep 12, 2021 6:22:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-12T18:22:28.315Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
    Sep 12, 2021 6:22:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-12T18:22:28.461Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
    Sep 12, 2021 6:22:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-12T18:22:28.496Z: Unzipping flatten s10 for input s6.writtenRecords
    Sep 12, 2021 6:22:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-12T18:22:28.528Z: Fusing unzipped copy of Write content to files/Write/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow), through flatten Write content to files/Write/WriteUnshardedBundlesToTempFiles/Flatten.PCollections, into producer Write content to files/Write/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles
    Sep 12, 2021 6:22:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-12T18:22:28.550Z: Fusing consumer Write content to files/Write/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow) into Write content to files/Write/WriteUnshardedBundlesToTempFiles/DropShardNum
    Sep 12, 2021 6:22:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-12T18:22:28.571Z: Fusing consumer Produce text lines into Generate sequence/Read(BoundedCountingSource)
    Sep 12, 2021 6:22:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-12T18:22:28.609Z: Fusing consumer Transform strings to bytes/Map into Produce text lines
    Sep 12, 2021 6:22:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-12T18:22:28.641Z: Fusing consumer Record time before writing into Transform strings to bytes/Map
    Sep 12, 2021 6:22:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-12T18:22:28.675Z: Fusing consumer Write content to files/Write/RewindowIntoGlobal/Window.Assign into Record time before writing
    Sep 12, 2021 6:22:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-12T18:22:28.709Z: Fusing consumer Write content to files/Write/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles into Write content to files/Write/RewindowIntoGlobal/Window.Assign
    Sep 12, 2021 6:22:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-12T18:22:28.741Z: Fusing consumer Write content to files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify into Write content to files/Write/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles
    Sep 12, 2021 6:22:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-12T18:22:28.763Z: Fusing consumer Write content to files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Write into Write content to files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify
    Sep 12, 2021 6:22:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-12T18:22:28.798Z: Fusing consumer Write content to files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/GroupByWindow into Write content to files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Read
    Sep 12, 2021 6:22:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-12T18:22:28.833Z: Fusing consumer Write content to files/Write/WriteUnshardedBundlesToTempFiles/WriteUnwritten into Write content to files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/GroupByWindow
    Sep 12, 2021 6:22:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-12T18:22:28.868Z: Fusing consumer Write content to files/Write/WriteUnshardedBundlesToTempFiles/DropShardNum into Write content to files/Write/WriteUnshardedBundlesToTempFiles/WriteUnwritten
    Sep 12, 2021 6:22:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-12T18:22:28.899Z: Fusing consumer Write content to files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Reify.ReifyView/ParDo(Anonymous) into Write content to files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Create.Values/Read(CreateSource)
    Sep 12, 2021 6:22:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-12T18:22:28.935Z: Fusing consumer Write content to files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Values/Values/Map into Write content to files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Reify.ReifyView/ParDo(Anonymous)
    Sep 12, 2021 6:22:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-12T18:22:28.978Z: Fusing consumer Write content to files/Write/FinalizeTempFileBundles/Finalize into Write content to files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Values/Values/Map
    Sep 12, 2021 6:22:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-12T18:22:29.012Z: Fusing consumer Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random key into Write content to files/Write/FinalizeTempFileBundles/Finalize
    Sep 12, 2021 6:22:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-12T18:22:29.056Z: Fusing consumer Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign into Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random key
    Sep 12, 2021 6:22:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-12T18:22:29.090Z: Fusing consumer Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify into Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign
    Sep 12, 2021 6:22:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-12T18:22:29.124Z: Fusing consumer Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write into Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify
    Sep 12, 2021 6:22:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-12T18:22:29.158Z: Fusing consumer Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow into Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read
    Sep 12, 2021 6:22:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-12T18:22:29.182Z: Fusing consumer Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable into Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow
    Sep 12, 2021 6:22:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-12T18:22:29.210Z: Fusing consumer Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Values/Values/Map into Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable
    Sep 12, 2021 6:22:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-12T18:22:29.564Z: Executing operation Write content to files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Create
    Sep 12, 2021 6:22:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-12T18:22:29.629Z: Starting 5 ****s in us-central1-c...
    Sep 12, 2021 6:22:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-12T18:22:30.111Z: Finished operation Write content to files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Create
    Sep 12, 2021 6:22:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-12T18:22:30.244Z: Executing operation Generate sequence/Read(BoundedCountingSource)+Produce text lines+Transform strings to bytes/Map+Record time before writing+Write content to files/Write/RewindowIntoGlobal/Window.Assign+Write content to files/Write/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles+Write content to files/Write/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow)+Write content to files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify+Write content to files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Write
    Sep 12, 2021 6:22:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-12T18:22:37.633Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
    Sep 12, 2021 6:23:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    WARNING: 2021-09-12T18:23:09.826Z: Autoscaling: Startup of the **** pool in zone us-central1-c reached 1 ****s, but the goal was 5 ****s. The service will retry. QUOTA_EXCEEDED: Quota 'CPUS' exceeded.  Limit: 2500.0 in region us-central1.
    Sep 12, 2021 6:23:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    SEVERE: 2021-09-12T18:23:21.918Z: Startup of the **** pool in zone us-central1-c failed to bring up any of the desired 5 ****s. QUOTA_EXCEEDED: Quota 'CPUS' exceeded.  Limit: 2500.0 in region us-central1.
    Sep 12, 2021 6:23:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    SEVERE: 2021-09-12T18:23:21.949Z: Workflow failed.
    Sep 12, 2021 6:23:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    WARNING: 2021-09-12T18:23:22.318Z: S02:Generate sequence/Read(BoundedCountingSource)+Produce text lines+Transform strings to bytes/Map+Record time before writing+Write content to files/Write/RewindowIntoGlobal/Window.Assign+Write content to files/Write/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles+Write content to files/Write/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow)+Write content to files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify+Write content to files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Write failed.
    Sep 12, 2021 6:23:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-12T18:23:22.351Z: Finished operation Generate sequence/Read(BoundedCountingSource)+Produce text lines+Transform strings to bytes/Map+Record time before writing+Write content to files/Write/RewindowIntoGlobal/Window.Assign+Write content to files/Write/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles+Write content to files/Write/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow)+Write content to files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify+Write content to files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Write
    Sep 12, 2021 6:23:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-12T18:23:22.431Z: Cleaning up.
    Sep 12, 2021 6:23:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-12T18:23:22.522Z: Stopping **** pool...
    Sep 12, 2021 6:23:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-12T18:23:49.561Z: Worker pool stopped.
    Sep 12, 2021 6:24:25 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
    INFO: Job 2021-09-12_11_22_03-11495571717971727843 failed with status FAILED.

Gradle Test Executor 1 finished executing tests.

> Task :sdks:java:io:file-based-io-tests:integrationTest FAILED

org.apache.beam.sdk.io.tfrecord.TFRecordIOIT > writeThenReadAll FAILED
    java.lang.IllegalStateException: Unable to find any files matching gs://temp-storage-for-perf-tests/beam_PerformanceTests_TFRecordIOIT/5315/_1631470914414*
        at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:588)
        at org.apache.beam.sdk.io.TFRecordIO$Read.expand(TFRecordIO.java:191)
        at org.apache.beam.sdk.io.TFRecordIO$Read.expand(TFRecordIO.java:112)
        at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:548)
        at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:482)
        at org.apache.beam.sdk.values.PBegin.apply(PBegin.java:44)
        at org.apache.beam.sdk.Pipeline.apply(Pipeline.java:177)
        at org.apache.beam.sdk.io.tfrecord.TFRecordIOIT.writeThenReadAll(TFRecordIOIT.java:151)

1 test completed, 1 failed
Finished generating test XML results (0.023 secs) into: <https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/file-based-io-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.029 secs) into: <https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/file-based-io-tests/build/reports/tests/integrationTest>
:sdks:java:io:file-based-io-tests:integrationTest (Thread[Execution **** for ':' Thread 2,5,main]) completed. Took 2 mins 34.628 secs.

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:io:file-based-io-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/file-based-io-tests/build/reports/tests/integrationTest/index.html>

* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3m 4s
129 actionable tasks: 79 executed, 50 from cache

Publishing build scan...
https://gradle.com/s/s2pwijegztydi

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_PerformanceTests_TFRecordIOIT #5314

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/5314/display/redirect>

Changes:


------------------------------------------
[...truncated 273.41 KB...]
Task ':sdks:java:io:google-cloud-platform:jar' is not up-to-date because:
  No history is available.
file or directory '<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/google-cloud-platform/build/resources/main',> not found
:sdks:java:io:google-cloud-platform:jar (Thread[Daemon ****,5,main]) completed. Took 0.238 secs.
:runners:google-cloud-dataflow-java:compileJava (Thread[Daemon ****,5,main]) started.
:sdks:java:io:google-cloud-platform:compileTestJava (Thread[Execution **** for ':' Thread 8,5,main]) started.

> Task :runners:google-cloud-dataflow-java:compileJava FROM-CACHE
Custom actions are attached to task ':runners:google-cloud-dataflow-java:compileJava'.
Build cache key for task ':runners:google-cloud-dataflow-java:compileJava' is f4d45dae85b41e621c919183804b6afa
Task ':runners:google-cloud-dataflow-java:compileJava' is not up-to-date because:
  No history is available.
Loaded cache entry for task ':runners:google-cloud-dataflow-java:compileJava' with cache key f4d45dae85b41e621c919183804b6afa
:runners:google-cloud-dataflow-java:compileJava (Thread[Daemon ****,5,main]) completed. Took 0.24 secs.
:runners:google-cloud-dataflow-java:classes (Thread[Daemon ****,5,main]) started.

> Task :runners:google-cloud-dataflow-java:classes
Skipping task ':runners:google-cloud-dataflow-java:classes' as it has no actions.
:runners:google-cloud-dataflow-java:classes (Thread[Daemon ****,5,main]) completed. Took 0.0 secs.
:runners:google-cloud-dataflow-java:jar (Thread[Daemon ****,5,main]) started.

> Task :runners:google-cloud-dataflow-java:jar
Caching disabled for task ':runners:google-cloud-dataflow-java:jar' because:
  Caching has not been enabled for the task
Task ':runners:google-cloud-dataflow-java:jar' is not up-to-date because:
  No history is available.
:runners:google-cloud-dataflow-java:jar (Thread[Daemon ****,5,main]) completed. Took 0.089 secs.
:runners:google-cloud-dataflow-java:****:legacy-****:compileJava (Thread[Daemon ****,5,main]) started.

> Task :sdks:java:io:google-cloud-platform:compileTestJava FROM-CACHE
Custom actions are attached to task ':sdks:java:io:google-cloud-platform:compileTestJava'.
Build cache key for task ':sdks:java:io:google-cloud-platform:compileTestJava' is 99462c52f9bbbb5d949b676d07ee3bc7
Task ':sdks:java:io:google-cloud-platform:compileTestJava' is not up-to-date because:
  No history is available.
Loaded cache entry for task ':sdks:java:io:google-cloud-platform:compileTestJava' with cache key 99462c52f9bbbb5d949b676d07ee3bc7
:sdks:java:io:google-cloud-platform:compileTestJava (Thread[Execution **** for ':' Thread 8,5,main]) completed. Took 0.524 secs.
:sdks:java:io:google-cloud-platform:testClasses (Thread[Execution **** for ':' Thread 8,5,main]) started.

> Task :sdks:java:io:google-cloud-platform:testClasses
Skipping task ':sdks:java:io:google-cloud-platform:testClasses' as it has no actions.
:sdks:java:io:google-cloud-platform:testClasses (Thread[Execution **** for ':' Thread 8,5,main]) completed. Took 0.0 secs.
:sdks:java:io:google-cloud-platform:testJar (Thread[Execution **** for ':' Thread 8,5,main]) started.

> Task :runners:google-cloud-dataflow-java:****:legacy-****:compileJava FROM-CACHE
file or directory '<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/runners/google-cloud-dataflow-java/****/legacy-****/src/main/java',> not found
Custom actions are attached to task ':runners:google-cloud-dataflow-java:****:legacy-****:compileJava'.
Build cache key for task ':runners:google-cloud-dataflow-java:****:legacy-****:compileJava' is cf672da70ce14149e9f2f602b34780e4
Task ':runners:google-cloud-dataflow-java:****:legacy-****:compileJava' is not up-to-date because:
  No history is available.
Loaded cache entry for task ':runners:google-cloud-dataflow-java:****:legacy-****:compileJava' with cache key cf672da70ce14149e9f2f602b34780e4
:runners:google-cloud-dataflow-java:****:legacy-****:compileJava (Thread[Daemon ****,5,main]) completed. Took 0.397 secs.
:runners:google-cloud-dataflow-java:****:legacy-****:classes (Thread[Daemon ****,5,main]) started.

> Task :runners:google-cloud-dataflow-java:****:legacy-****:classes UP-TO-DATE
Skipping task ':runners:google-cloud-dataflow-java:****:legacy-****:classes' as it has no actions.
:runners:google-cloud-dataflow-java:****:legacy-****:classes (Thread[Daemon ****,5,main]) completed. Took 0.0 secs.
:runners:google-cloud-dataflow-java:****:legacy-****:shadowJar (Thread[Daemon ****,5,main]) started.

> Task :sdks:java:io:google-cloud-platform:testJar
Caching disabled for task ':sdks:java:io:google-cloud-platform:testJar' because:
  Caching has not been enabled for the task
Task ':sdks:java:io:google-cloud-platform:testJar' is not up-to-date because:
  No history is available.
:sdks:java:io:google-cloud-platform:testJar (Thread[Execution **** for ':' Thread 8,5,main]) completed. Took 0.262 secs.
:runners:google-cloud-dataflow-java:compileTestJava (Thread[Execution **** for ':' Thread 8,5,main]) started.

> Task :runners:google-cloud-dataflow-java:****:legacy-****:shadowJar FROM-CACHE
Custom actions are attached to task ':runners:google-cloud-dataflow-java:****:legacy-****:shadowJar'.
Build cache key for task ':runners:google-cloud-dataflow-java:****:legacy-****:shadowJar' is b03e31546baded1569b9e22785f539a5
Task ':runners:google-cloud-dataflow-java:****:legacy-****:shadowJar' is not up-to-date because:
  No history is available.
Loaded cache entry for task ':runners:google-cloud-dataflow-java:****:legacy-****:shadowJar' with cache key b03e31546baded1569b9e22785f539a5
:runners:google-cloud-dataflow-java:****:legacy-****:shadowJar (Thread[Daemon ****,5,main]) completed. Took 0.3 secs.

> Task :runners:google-cloud-dataflow-java:compileTestJava FROM-CACHE
Custom actions are attached to task ':runners:google-cloud-dataflow-java:compileTestJava'.
Build cache key for task ':runners:google-cloud-dataflow-java:compileTestJava' is 861c7435ffd549666858392e9bbd7747
Task ':runners:google-cloud-dataflow-java:compileTestJava' is not up-to-date because:
  No history is available.
Loaded cache entry for task ':runners:google-cloud-dataflow-java:compileTestJava' with cache key 861c7435ffd549666858392e9bbd7747
:runners:google-cloud-dataflow-java:compileTestJava (Thread[Execution **** for ':' Thread 8,5,main]) completed. Took 0.277 secs.
:runners:google-cloud-dataflow-java:testClasses (Thread[Execution **** for ':' Thread 8,5,main]) started.

> Task :runners:google-cloud-dataflow-java:testClasses UP-TO-DATE
Skipping task ':runners:google-cloud-dataflow-java:testClasses' as it has no actions.
:runners:google-cloud-dataflow-java:testClasses (Thread[Execution **** for ':' Thread 8,5,main]) completed. Took 0.0 secs.
:runners:google-cloud-dataflow-java:testJar (Thread[Execution **** for ':' Thread 8,5,main]) started.

> Task :runners:google-cloud-dataflow-java:testJar
Caching disabled for task ':runners:google-cloud-dataflow-java:testJar' because:
  Caching has not been enabled for the task
Task ':runners:google-cloud-dataflow-java:testJar' is not up-to-date because:
  No history is available.
file or directory '<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/runners/google-cloud-dataflow-java/build/resources/test',> not found
:runners:google-cloud-dataflow-java:testJar (Thread[Execution **** for ':' Thread 8,5,main]) completed. Took 0.031 secs.
:sdks:java:io:file-based-io-tests:integrationTest (Thread[Execution **** for ':' Thread 8,5,main]) started.
Gradle Test Executor 1 started executing tests.

> Task :sdks:java:io:file-based-io-tests:integrationTest
Custom actions are attached to task ':sdks:java:io:file-based-io-tests:integrationTest'.
Build cache key for task ':sdks:java:io:file-based-io-tests:integrationTest' is 2954f27e833e06d9b6038317a4b1cf00
Task ':sdks:java:io:file-based-io-tests:integrationTest' is not up-to-date because:
  Task.upToDateWhen is false.
Starting process 'Gradle Test Executor 1'. Working directory: <https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/file-based-io-tests> Command: /usr/lib/jvm/java-8-openjdk-amd64/bin/java -DbeamTestPipelineOptions=["--runner=DataflowRunner","--project=apache-beam-testing","--tempRoot=gs://temp-storage-for-perf-tests","--filenamePrefix=gs://temp-storage-for-perf-tests/beam_PerformanceTests_TFRecordIOIT/5314/","--bigQueryDataset=beam_performance","--bigQueryTable=tfrecordioit_results","--influxMeasurement=tfrecordioit_results","--numberOfRecords=18000000","--expectedHash=543104423f8b6eb097acb9f111c19fe4","--datasetSize=1019380000","--numWorkers=5","--autoscalingAlgorithm=NONE","--influxDatabase=beam_test_metrics","--influxHost=http://10.128.0.96:8086","--****HarnessContainerImage=","--dataflowWorkerJar=<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/runners/google-cloud-dataflow-java/****/legacy-****/build/libs/beam-runners-google-cloud-dataflow-java-legacy-****-2.34.0-SNAPSHOT.jar","--region=us-central1"]> -Djava.security.manager=****.org.gradle.process.internal.****.child.BootstrapSecurityManager -Dorg.gradle.native=false -Xmx2g -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant -ea -cp /home/jenkins/.gradle/caches/6.8.3/****Main/gradle-****.jar ****.org.gradle.process.internal.****.GradleWorkerMain 'Gradle Test Executor 1'
Successfully started process 'Gradle Test Executor 1'

org.apache.beam.sdk.io.tfrecord.TFRecordIOIT STANDARD_ERROR
    SLF4J: Class path contains multiple SLF4J bindings.
    SLF4J: Found binding in [jar:<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/runners/google-cloud-dataflow-java/****/legacy-****/build/libs/beam-runners-google-cloud-dataflow-java-legacy-****-2.34.0-SNAPSHOT.jar!/org/slf4j/impl/StaticLoggerBinder.class]>
    SLF4J: Found binding in [jar:file:/home/jenkins/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-log4j12/1.7.30/c21f55139d8141d2231214fb1feaf50a1edca95e/slf4j-log4j12-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class]
    SLF4J: Found binding in [jar:file:/home/jenkins/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-jdk14/1.7.30/d35953dd2fe54ebe39fdf18cfd82fe6eb35b25ed/slf4j-jdk14-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class]
    SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
    SLF4J: Actual binding is of type [org.slf4j.impl.JDK14LoggerFactory]

org.apache.beam.sdk.io.tfrecord.TFRecordIOIT > writeThenReadAll STANDARD_ERROR
    Sep 12, 2021 12:21:54 PM org.apache.beam.runners.dataflow.DataflowRunner validateSdkContainerImageOptions
    WARNING: Prefer --sdkContainerImage over deprecated legacy option --****HarnessContainerImage.
    Sep 12, 2021 12:21:55 PM org.apache.beam.sdk.extensions.gcp.options.GcpOptions$GcpTempLocationFactory tryCreateDefaultBucket
    INFO: No tempLocation specified, attempting to use default bucket: dataflow-staging-us-central1-844138762903
    Sep 12, 2021 12:21:56 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
    WARNING: Request failed with code 409, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/storage/v1/b?predefinedAcl=projectPrivate&predefinedDefaultObjectAcl=projectPrivate&project=apache-beam-testing. 
    Sep 12, 2021 12:21:56 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
    INFO: No stagingLocation provided, falling back to gcpTempLocation
    Sep 12, 2021 12:21:57 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
    INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 317 files. Enable logging at DEBUG level to see which files will be staged.
    Sep 12, 2021 12:21:57 PM org.apache.beam.runners.dataflow.DataflowRunner run
    INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
    Sep 12, 2021 12:22:00 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
    INFO: Uploading 318 files from PipelineOptions.filesToStage to staging location to prepare for execution.
    Sep 12, 2021 12:22:00 PM org.apache.beam.runners.dataflow.util.PackageUtil$PackageAttributes forFileToStage
    INFO: Staging custom dataflow-****.jar as beam-runners-google-cloud-dataflow-java-legacy-****-2.34.0-SNAPSHOT-YtvkOIsfQYfFzG89aVBO_8oNHoiij9SynwUJLdMkn5A.jar
    Sep 12, 2021 12:22:00 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
    INFO: Uploading /tmp/test2674221705647583389.zip to gs://dataflow-staging-us-central1-844138762903/temp/staging/test-ka0iskoFXTG_zNnDaDnnjcs3oJFMRG7LROJH98bqfaI.jar
    Sep 12, 2021 12:22:01 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
    INFO: Staging files complete: 317 files cached, 1 files newly uploaded in 0 seconds
    Sep 12, 2021 12:22:01 PM org.apache.beam.runners.dataflow.DataflowRunner run
    INFO: Staging portable pipeline proto to gs://dataflow-staging-us-central1-844138762903/temp/staging/
    Sep 12, 2021 12:22:01 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
    INFO: Uploading <205290 bytes, hash 00eea2457c72a94ac02c74f3d7a0670e8c066a316652301b8c5c4c70166163c4> to gs://dataflow-staging-us-central1-844138762903/temp/staging/pipeline-AO6iRXxyqUrALHTz16BnDowGajFmUjAbjFxMcBZhY8Q.pb
    Sep 12, 2021 12:22:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Generate sequence/Read(BoundedCountingSource) as step s1
    Sep 12, 2021 12:22:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Produce text lines as step s2
    Sep 12, 2021 12:22:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Transform strings to bytes/Map as step s3
    Sep 12, 2021 12:22:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Record time before writing as step s4
    Sep 12, 2021 12:22:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/RewindowIntoGlobal/Window.Assign as step s5
    Sep 12, 2021 12:22:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles as step s6
    Sep 12, 2021 12:22:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten as step s7
    Sep 12, 2021 12:22:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/WriteUnshardedBundlesToTempFiles/WriteUnwritten as step s8
    Sep 12, 2021 12:22:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/WriteUnshardedBundlesToTempFiles/DropShardNum as step s9
    Sep 12, 2021 12:22:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/WriteUnshardedBundlesToTempFiles/Flatten.PCollections as step s10
    Sep 12, 2021 12:22:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow) as step s11
    Sep 12, 2021 12:22:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/GatherTempFileResults/View.AsList/CreateDataflowView as step s12
    Sep 12, 2021 12:22:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Create.Values/Read(CreateSource) as step s13
    Sep 12, 2021 12:22:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Reify.ReifyView/ParDo(Anonymous) as step s14
    Sep 12, 2021 12:22:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Values/Values/Map as step s15
    Sep 12, 2021 12:22:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Finalize as step s16
    Sep 12, 2021 12:22:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random key as step s17
    Sep 12, 2021 12:22:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign as step s18
    Sep 12, 2021 12:22:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey as step s19
    Sep 12, 2021 12:22:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable as step s20
    Sep 12, 2021 12:22:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Values/Values/Map as step s21
    Sep 12, 2021 12:22:03 PM org.apache.beam.runners.dataflow.DataflowRunner run
    INFO: Dataflow SDK version: 2.34.0-SNAPSHOT
    Sep 12, 2021 12:22:04 PM org.apache.beam.runners.dataflow.DataflowRunner run
    INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-09-12_05_22_03-15783473270550792703?project=apache-beam-testing
    Sep 12, 2021 12:22:04 PM org.apache.beam.runners.dataflow.DataflowRunner run
    INFO: Submitted job: 2021-09-12_05_22_03-15783473270550792703
    Sep 12, 2021 12:22:04 PM org.apache.beam.runners.dataflow.DataflowRunner run
    INFO: To cancel the job using the 'gcloud' tool, run:
    > gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-09-12_05_22_03-15783473270550792703
    Sep 12, 2021 12:22:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-12T12:22:30.230Z: Worker configuration: e2-standard-2 in us-central1-c.
    Sep 12, 2021 12:22:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    SEVERE: 2021-09-12T12:22:30.967Z: Workflow failed. Causes: Project apache-beam-testing has insufficient quota(s) to execute this workflow with 1 instances in region us-central1. Quota summary (required/available): 1/24379 instances, 2/0 CPUs, 25/186631 disk GB, 0/2397 SSD disk GB, 1/275 instance groups, 1/280 managed instance groups, 1/501 instance templates, 1/608 in-use IP addresses.

    Please see https://cloud.google.com/compute/docs/resource-quotas about requesting more quota.
    Sep 12, 2021 12:22:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-12T12:22:31.012Z: Cleaning up.
    Sep 12, 2021 12:22:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-12T12:22:31.073Z: Worker pool stopped.
    Sep 12, 2021 12:22:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-12T12:22:32.263Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
    Sep 12, 2021 12:22:36 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
    INFO: Job 2021-09-12_05_22_03-15783473270550792703 failed with status FAILED.

Gradle Test Executor 1 finished executing tests.

> Task :sdks:java:io:file-based-io-tests:integrationTest FAILED

org.apache.beam.sdk.io.tfrecord.TFRecordIOIT > writeThenReadAll FAILED
    java.lang.IllegalStateException: Unable to find any files matching gs://temp-storage-for-perf-tests/beam_PerformanceTests_TFRecordIOIT/5314/_1631449313784*
        at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:588)
        at org.apache.beam.sdk.io.TFRecordIO$Read.expand(TFRecordIO.java:191)
        at org.apache.beam.sdk.io.TFRecordIO$Read.expand(TFRecordIO.java:112)
        at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:548)
        at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:482)
        at org.apache.beam.sdk.values.PBegin.apply(PBegin.java:44)
        at org.apache.beam.sdk.Pipeline.apply(Pipeline.java:177)
        at org.apache.beam.sdk.io.tfrecord.TFRecordIOIT.writeThenReadAll(TFRecordIOIT.java:151)

1 test completed, 1 failed
Finished generating test XML results (0.022 secs) into: <https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/file-based-io-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.032 secs) into: <https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/file-based-io-tests/build/reports/tests/integrationTest>
:sdks:java:io:file-based-io-tests:integrationTest (Thread[Execution **** for ':' Thread 8,5,main]) completed. Took 46.353 secs.

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:io:file-based-io-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/file-based-io-tests/build/reports/tests/integrationTest/index.html>

* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 1m 18s
129 actionable tasks: 79 executed, 50 from cache

Publishing build scan...
https://gradle.com/s/425itavdyeank

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_PerformanceTests_TFRecordIOIT #5313

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/5313/display/redirect>

Changes:


------------------------------------------
[...truncated 286.15 KB...]
Build cache key for task ':runners:google-cloud-dataflow-java:compileTestJava' is 861c7435ffd549666858392e9bbd7747
Task ':runners:google-cloud-dataflow-java:compileTestJava' is not up-to-date because:
  No history is available.
Loaded cache entry for task ':runners:google-cloud-dataflow-java:compileTestJava' with cache key 861c7435ffd549666858392e9bbd7747
:runners:google-cloud-dataflow-java:compileTestJava (Thread[Execution **** for ':',5,main]) completed. Took 0.256 secs.
:runners:google-cloud-dataflow-java:testClasses (Thread[Execution **** for ':',5,main]) started.

> Task :runners:google-cloud-dataflow-java:testClasses UP-TO-DATE
Skipping task ':runners:google-cloud-dataflow-java:testClasses' as it has no actions.
:runners:google-cloud-dataflow-java:testClasses (Thread[Execution **** for ':',5,main]) completed. Took 0.0 secs.
:runners:google-cloud-dataflow-java:testJar (Thread[Execution **** for ':',5,main]) started.

> Task :runners:google-cloud-dataflow-java:testJar
Caching disabled for task ':runners:google-cloud-dataflow-java:testJar' because:
  Caching has not been enabled for the task
Task ':runners:google-cloud-dataflow-java:testJar' is not up-to-date because:
  No history is available.
file or directory '<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/runners/google-cloud-dataflow-java/build/resources/test',> not found
:runners:google-cloud-dataflow-java:testJar (Thread[Execution **** for ':',5,main]) completed. Took 0.042 secs.
:sdks:java:io:file-based-io-tests:integrationTest (Thread[Execution **** for ':',5,main]) started.

> Task :sdks:java:io:file-based-io-tests:integrationTest
Custom actions are attached to task ':sdks:java:io:file-based-io-tests:integrationTest'.
Build cache key for task ':sdks:java:io:file-based-io-tests:integrationTest' is 2954f27e833e06d9b6038317a4b1cf00
Task ':sdks:java:io:file-based-io-tests:integrationTest' is not up-to-date because:
  Task.upToDateWhen is false.
Starting process 'Gradle Test Executor 1'. Working directory: <https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/file-based-io-tests> Command: /usr/lib/jvm/java-8-openjdk-amd64/bin/java -DbeamTestPipelineOptions=["--runner=DataflowRunner","--project=apache-beam-testing","--tempRoot=gs://temp-storage-for-perf-tests","--filenamePrefix=gs://temp-storage-for-perf-tests/beam_PerformanceTests_TFRecordIOIT/5313/","--bigQueryDataset=beam_performance","--bigQueryTable=tfrecordioit_results","--influxMeasurement=tfrecordioit_results","--numberOfRecords=18000000","--expectedHash=543104423f8b6eb097acb9f111c19fe4","--datasetSize=1019380000","--numWorkers=5","--autoscalingAlgorithm=NONE","--influxDatabase=beam_test_metrics","--influxHost=http://10.128.0.96:8086","--****HarnessContainerImage=","--dataflowWorkerJar=<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/runners/google-cloud-dataflow-java/****/legacy-****/build/libs/beam-runners-google-cloud-dataflow-java-legacy-****-2.34.0-SNAPSHOT.jar","--region=us-central1"]> -Djava.security.manager=****.org.gradle.process.internal.****.child.BootstrapSecurityManager -Dorg.gradle.native=false -Xmx2g -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant -ea -cp /home/jenkins/.gradle/caches/6.8.3/****Main/gradle-****.jar ****.org.gradle.process.internal.****.GradleWorkerMain 'Gradle Test Executor 1'
Successfully started process 'Gradle Test Executor 1'

Gradle Test Executor 1 started executing tests.

> Task :sdks:java:io:file-based-io-tests:integrationTest

org.apache.beam.sdk.io.tfrecord.TFRecordIOIT STANDARD_ERROR
    SLF4J: Class path contains multiple SLF4J bindings.
    SLF4J: Found binding in [jar:<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/runners/google-cloud-dataflow-java/****/legacy-****/build/libs/beam-runners-google-cloud-dataflow-java-legacy-****-2.34.0-SNAPSHOT.jar!/org/slf4j/impl/StaticLoggerBinder.class]>
    SLF4J: Found binding in [jar:file:/home/jenkins/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-log4j12/1.7.30/c21f55139d8141d2231214fb1feaf50a1edca95e/slf4j-log4j12-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class]
    SLF4J: Found binding in [jar:file:/home/jenkins/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-jdk14/1.7.30/d35953dd2fe54ebe39fdf18cfd82fe6eb35b25ed/slf4j-jdk14-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class]
    SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
    SLF4J: Actual binding is of type [org.slf4j.impl.JDK14LoggerFactory]

org.apache.beam.sdk.io.tfrecord.TFRecordIOIT > writeThenReadAll STANDARD_ERROR
    Sep 12, 2021 6:21:56 AM org.apache.beam.runners.dataflow.DataflowRunner validateSdkContainerImageOptions
    WARNING: Prefer --sdkContainerImage over deprecated legacy option --****HarnessContainerImage.
    Sep 12, 2021 6:21:57 AM org.apache.beam.sdk.extensions.gcp.options.GcpOptions$GcpTempLocationFactory tryCreateDefaultBucket
    INFO: No tempLocation specified, attempting to use default bucket: dataflow-staging-us-central1-844138762903
    Sep 12, 2021 6:21:58 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
    WARNING: Request failed with code 409, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/storage/v1/b?predefinedAcl=projectPrivate&predefinedDefaultObjectAcl=projectPrivate&project=apache-beam-testing. 
    Sep 12, 2021 6:21:58 AM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
    INFO: No stagingLocation provided, falling back to gcpTempLocation
    Sep 12, 2021 6:21:59 AM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
    INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 317 files. Enable logging at DEBUG level to see which files will be staged.
    Sep 12, 2021 6:21:59 AM org.apache.beam.runners.dataflow.DataflowRunner run
    INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
    Sep 12, 2021 6:22:01 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
    INFO: Uploading 318 files from PipelineOptions.filesToStage to staging location to prepare for execution.
    Sep 12, 2021 6:22:01 AM org.apache.beam.runners.dataflow.util.PackageUtil$PackageAttributes forFileToStage
    INFO: Staging custom dataflow-****.jar as beam-runners-google-cloud-dataflow-java-legacy-****-2.34.0-SNAPSHOT-YtvkOIsfQYfFzG89aVBO_8oNHoiij9SynwUJLdMkn5A.jar
    Sep 12, 2021 6:22:02 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
    INFO: Uploading /tmp/test1107205975081295650.zip to gs://dataflow-staging-us-central1-844138762903/temp/staging/test-JuEQQQ6xKqI9vEuZmgroy5GA5BQPhiAhefqu2AL2rNY.jar
    Sep 12, 2021 6:22:02 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
    INFO: Staging files complete: 317 files cached, 1 files newly uploaded in 0 seconds
    Sep 12, 2021 6:22:02 AM org.apache.beam.runners.dataflow.DataflowRunner run
    INFO: Staging portable pipeline proto to gs://dataflow-staging-us-central1-844138762903/temp/staging/
    Sep 12, 2021 6:22:02 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
    INFO: Uploading <205290 bytes, hash 6390f0c398c565c5c59ce13a003756e6949a5fe4db53326a3270d9486f9b7721> to gs://dataflow-staging-us-central1-844138762903/temp/staging/pipeline-Y5Dww5jFZcXFnOE6ADdW5pSaX-TbUzJqMnDZSG-bdyE.pb
    Sep 12, 2021 6:22:05 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Generate sequence/Read(BoundedCountingSource) as step s1
    Sep 12, 2021 6:22:05 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Produce text lines as step s2
    Sep 12, 2021 6:22:05 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Transform strings to bytes/Map as step s3
    Sep 12, 2021 6:22:05 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Record time before writing as step s4
    Sep 12, 2021 6:22:05 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/RewindowIntoGlobal/Window.Assign as step s5
    Sep 12, 2021 6:22:05 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles as step s6
    Sep 12, 2021 6:22:05 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten as step s7
    Sep 12, 2021 6:22:05 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/WriteUnshardedBundlesToTempFiles/WriteUnwritten as step s8
    Sep 12, 2021 6:22:05 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/WriteUnshardedBundlesToTempFiles/DropShardNum as step s9
    Sep 12, 2021 6:22:05 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/WriteUnshardedBundlesToTempFiles/Flatten.PCollections as step s10
    Sep 12, 2021 6:22:05 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow) as step s11
    Sep 12, 2021 6:22:05 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/GatherTempFileResults/View.AsList/CreateDataflowView as step s12
    Sep 12, 2021 6:22:05 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Create.Values/Read(CreateSource) as step s13
    Sep 12, 2021 6:22:05 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Reify.ReifyView/ParDo(Anonymous) as step s14
    Sep 12, 2021 6:22:05 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Values/Values/Map as step s15
    Sep 12, 2021 6:22:05 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Finalize as step s16
    Sep 12, 2021 6:22:05 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random key as step s17
    Sep 12, 2021 6:22:05 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign as step s18
    Sep 12, 2021 6:22:05 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey as step s19
    Sep 12, 2021 6:22:05 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable as step s20
    Sep 12, 2021 6:22:05 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Values/Values/Map as step s21
    Sep 12, 2021 6:22:05 AM org.apache.beam.runners.dataflow.DataflowRunner run
    INFO: Dataflow SDK version: 2.34.0-SNAPSHOT
    Sep 12, 2021 6:22:06 AM org.apache.beam.runners.dataflow.DataflowRunner run
    INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-09-11_23_22_05-13023652060249267279?project=apache-beam-testing
    Sep 12, 2021 6:22:06 AM org.apache.beam.runners.dataflow.DataflowRunner run
    INFO: Submitted job: 2021-09-11_23_22_05-13023652060249267279
    Sep 12, 2021 6:22:06 AM org.apache.beam.runners.dataflow.DataflowRunner run
    INFO: To cancel the job using the 'gcloud' tool, run:
    > gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-09-11_23_22_05-13023652060249267279
    Sep 12, 2021 6:22:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-12T06:22:23.354Z: Worker configuration: e2-standard-2 in us-central1-c.
    Sep 12, 2021 6:22:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-12T06:22:24.109Z: Expanding CoGroupByKey operations into optimizable parts.
    Sep 12, 2021 6:22:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-12T06:22:24.197Z: Expanding GroupByKey operations into optimizable parts.
    Sep 12, 2021 6:22:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-12T06:22:24.227Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
    Sep 12, 2021 6:22:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-12T06:22:24.367Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
    Sep 12, 2021 6:22:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-12T06:22:24.400Z: Unzipping flatten s10 for input s6.writtenRecords
    Sep 12, 2021 6:22:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-12T06:22:24.430Z: Fusing unzipped copy of Write content to files/Write/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow), through flatten Write content to files/Write/WriteUnshardedBundlesToTempFiles/Flatten.PCollections, into producer Write content to files/Write/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles
    Sep 12, 2021 6:22:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-12T06:22:24.462Z: Fusing consumer Write content to files/Write/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow) into Write content to files/Write/WriteUnshardedBundlesToTempFiles/DropShardNum
    Sep 12, 2021 6:22:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-12T06:22:24.492Z: Fusing consumer Produce text lines into Generate sequence/Read(BoundedCountingSource)
    Sep 12, 2021 6:22:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-12T06:22:24.526Z: Fusing consumer Transform strings to bytes/Map into Produce text lines
    Sep 12, 2021 6:22:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-12T06:22:24.558Z: Fusing consumer Record time before writing into Transform strings to bytes/Map
    Sep 12, 2021 6:22:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-12T06:22:24.590Z: Fusing consumer Write content to files/Write/RewindowIntoGlobal/Window.Assign into Record time before writing
    Sep 12, 2021 6:22:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-12T06:22:24.622Z: Fusing consumer Write content to files/Write/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles into Write content to files/Write/RewindowIntoGlobal/Window.Assign
    Sep 12, 2021 6:22:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-12T06:22:24.655Z: Fusing consumer Write content to files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify into Write content to files/Write/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles
    Sep 12, 2021 6:22:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-12T06:22:24.685Z: Fusing consumer Write content to files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Write into Write content to files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify
    Sep 12, 2021 6:22:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-12T06:22:24.716Z: Fusing consumer Write content to files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/GroupByWindow into Write content to files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Read
    Sep 12, 2021 6:22:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-12T06:22:24.749Z: Fusing consumer Write content to files/Write/WriteUnshardedBundlesToTempFiles/WriteUnwritten into Write content to files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/GroupByWindow
    Sep 12, 2021 6:22:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-12T06:22:24.776Z: Fusing consumer Write content to files/Write/WriteUnshardedBundlesToTempFiles/DropShardNum into Write content to files/Write/WriteUnshardedBundlesToTempFiles/WriteUnwritten
    Sep 12, 2021 6:22:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-12T06:22:24.803Z: Fusing consumer Write content to files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Reify.ReifyView/ParDo(Anonymous) into Write content to files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Create.Values/Read(CreateSource)
    Sep 12, 2021 6:22:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-12T06:22:24.836Z: Fusing consumer Write content to files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Values/Values/Map into Write content to files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Reify.ReifyView/ParDo(Anonymous)
    Sep 12, 2021 6:22:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-12T06:22:24.867Z: Fusing consumer Write content to files/Write/FinalizeTempFileBundles/Finalize into Write content to files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Values/Values/Map
    Sep 12, 2021 6:22:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-12T06:22:24.899Z: Fusing consumer Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random key into Write content to files/Write/FinalizeTempFileBundles/Finalize
    Sep 12, 2021 6:22:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-12T06:22:24.921Z: Fusing consumer Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign into Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random key
    Sep 12, 2021 6:22:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-12T06:22:24.952Z: Fusing consumer Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify into Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign
    Sep 12, 2021 6:22:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-12T06:22:24.975Z: Fusing consumer Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write into Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify
    Sep 12, 2021 6:22:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-12T06:22:25.008Z: Fusing consumer Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow into Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read
    Sep 12, 2021 6:22:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-12T06:22:25.040Z: Fusing consumer Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable into Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow
    Sep 12, 2021 6:22:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-12T06:22:25.063Z: Fusing consumer Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Values/Values/Map into Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable
    Sep 12, 2021 6:22:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-12T06:22:25.525Z: Executing operation Write content to files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Create
    Sep 12, 2021 6:22:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-12T06:22:25.593Z: Starting 5 ****s in us-central1-c...
    Sep 12, 2021 6:22:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-12T06:22:25.895Z: Finished operation Write content to files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Create
    Sep 12, 2021 6:22:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-12T06:22:26.040Z: Executing operation Generate sequence/Read(BoundedCountingSource)+Produce text lines+Transform strings to bytes/Map+Record time before writing+Write content to files/Write/RewindowIntoGlobal/Window.Assign+Write content to files/Write/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles+Write content to files/Write/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow)+Write content to files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify+Write content to files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Write
    Sep 12, 2021 6:22:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-12T06:22:37.552Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
    Sep 12, 2021 6:23:04 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    WARNING: 2021-09-12T06:23:03.376Z: Autoscaling: Startup of the **** pool in zone us-central1-c reached 4 ****s, but the goal was 5 ****s. The service will retry. QUOTA_EXCEEDED: Quota 'CPUS' exceeded.  Limit: 2500.0 in region us-central1.
    Sep 12, 2021 6:23:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    SEVERE: 2021-09-12T06:23:15.291Z: Startup of the **** pool in zone us-central1-c failed to bring up any of the desired 5 ****s. QUOTA_EXCEEDED: Quota 'CPUS' exceeded.  Limit: 2500.0 in region us-central1.
    Sep 12, 2021 6:23:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    SEVERE: 2021-09-12T06:23:15.347Z: Workflow failed.
    Sep 12, 2021 6:23:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    WARNING: 2021-09-12T06:23:15.635Z: S02:Generate sequence/Read(BoundedCountingSource)+Produce text lines+Transform strings to bytes/Map+Record time before writing+Write content to files/Write/RewindowIntoGlobal/Window.Assign+Write content to files/Write/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles+Write content to files/Write/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow)+Write content to files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify+Write content to files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Write failed.
    Sep 12, 2021 6:23:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-12T06:23:15.671Z: Finished operation Generate sequence/Read(BoundedCountingSource)+Produce text lines+Transform strings to bytes/Map+Record time before writing+Write content to files/Write/RewindowIntoGlobal/Window.Assign+Write content to files/Write/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles+Write content to files/Write/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow)+Write content to files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify+Write content to files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Write
    Sep 12, 2021 6:23:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-12T06:23:15.733Z: Cleaning up.
    Sep 12, 2021 6:23:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-12T06:23:15.831Z: Stopping **** pool...
    Sep 12, 2021 6:23:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-12T06:23:43.209Z: Worker pool stopped.
    Sep 12, 2021 6:24:20 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
    INFO: Job 2021-09-11_23_22_05-13023652060249267279 failed with status FAILED.

Gradle Test Executor 1 finished executing tests.

> Task :sdks:java:io:file-based-io-tests:integrationTest FAILED

org.apache.beam.sdk.io.tfrecord.TFRecordIOIT > writeThenReadAll FAILED
    java.lang.IllegalStateException: Unable to find any files matching gs://temp-storage-for-perf-tests/beam_PerformanceTests_TFRecordIOIT/5313/_1631427715839*
        at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:588)
        at org.apache.beam.sdk.io.TFRecordIO$Read.expand(TFRecordIO.java:191)
        at org.apache.beam.sdk.io.TFRecordIO$Read.expand(TFRecordIO.java:112)
        at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:548)
        at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:482)
        at org.apache.beam.sdk.values.PBegin.apply(PBegin.java:44)
        at org.apache.beam.sdk.Pipeline.apply(Pipeline.java:177)
        at org.apache.beam.sdk.io.tfrecord.TFRecordIOIT.writeThenReadAll(TFRecordIOIT.java:151)

1 test completed, 1 failed
Finished generating test XML results (0.025 secs) into: <https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/file-based-io-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.027 secs) into: <https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/file-based-io-tests/build/reports/tests/integrationTest>
:sdks:java:io:file-based-io-tests:integrationTest (Thread[Execution **** for ':',5,main]) completed. Took 2 mins 30.685 secs.

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:io:file-based-io-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/file-based-io-tests/build/reports/tests/integrationTest/index.html>

* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3m 0s
129 actionable tasks: 79 executed, 50 from cache

Publishing build scan...
https://gradle.com/s/ugtgcn6zoaomk

Build cache (/home/jenkins/.gradle/caches/build-cache-1) removing files not accessed on or after Sun Sep 05 06:21:24 UTC 2021.
Invalidating in-memory cache of /home/jenkins/.gradle/caches/journal-1/file-access.bin
Build cache (/home/jenkins/.gradle/caches/build-cache-1) cleaned up in 0.398 secs.
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_PerformanceTests_TFRecordIOIT #5312

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/5312/display/redirect>

Changes:


------------------------------------------
[...truncated 274.31 KB...]
Task ':sdks:java:io:google-cloud-platform:jar' is not up-to-date because:
  No history is available.
file or directory '<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/google-cloud-platform/build/resources/main',> not found
:sdks:java:io:google-cloud-platform:jar (Thread[Execution **** for ':' Thread 10,5,main]) completed. Took 0.233 secs.
:runners:google-cloud-dataflow-java:compileJava (Thread[Execution **** for ':' Thread 10,5,main]) started.
:sdks:java:io:google-cloud-platform:compileTestJava (Thread[Execution **** for ':',5,main]) started.

> Task :runners:google-cloud-dataflow-java:compileJava FROM-CACHE
Custom actions are attached to task ':runners:google-cloud-dataflow-java:compileJava'.
Build cache key for task ':runners:google-cloud-dataflow-java:compileJava' is f4d45dae85b41e621c919183804b6afa
Task ':runners:google-cloud-dataflow-java:compileJava' is not up-to-date because:
  No history is available.
Loaded cache entry for task ':runners:google-cloud-dataflow-java:compileJava' with cache key f4d45dae85b41e621c919183804b6afa
:runners:google-cloud-dataflow-java:compileJava (Thread[Execution **** for ':' Thread 10,5,main]) completed. Took 0.221 secs.
:runners:google-cloud-dataflow-java:classes (Thread[Execution **** for ':' Thread 10,5,main]) started.

> Task :runners:google-cloud-dataflow-java:classes
Skipping task ':runners:google-cloud-dataflow-java:classes' as it has no actions.
:runners:google-cloud-dataflow-java:classes (Thread[Execution **** for ':' Thread 10,5,main]) completed. Took 0.0 secs.
:runners:google-cloud-dataflow-java:jar (Thread[Execution **** for ':' Thread 10,5,main]) started.

> Task :runners:google-cloud-dataflow-java:jar
Caching disabled for task ':runners:google-cloud-dataflow-java:jar' because:
  Caching has not been enabled for the task
Task ':runners:google-cloud-dataflow-java:jar' is not up-to-date because:
  No history is available.
:runners:google-cloud-dataflow-java:jar (Thread[Execution **** for ':' Thread 10,5,main]) completed. Took 0.091 secs.
:runners:google-cloud-dataflow-java:****:legacy-****:compileJava (Thread[Execution **** for ':' Thread 10,5,main]) started.

> Task :sdks:java:io:google-cloud-platform:compileTestJava FROM-CACHE
Custom actions are attached to task ':sdks:java:io:google-cloud-platform:compileTestJava'.
Build cache key for task ':sdks:java:io:google-cloud-platform:compileTestJava' is 99462c52f9bbbb5d949b676d07ee3bc7
Task ':sdks:java:io:google-cloud-platform:compileTestJava' is not up-to-date because:
  No history is available.
Loaded cache entry for task ':sdks:java:io:google-cloud-platform:compileTestJava' with cache key 99462c52f9bbbb5d949b676d07ee3bc7
:sdks:java:io:google-cloud-platform:compileTestJava (Thread[Execution **** for ':',5,main]) completed. Took 0.506 secs.
:sdks:java:io:google-cloud-platform:testClasses (Thread[Execution **** for ':',5,main]) started.

> Task :sdks:java:io:google-cloud-platform:testClasses
Skipping task ':sdks:java:io:google-cloud-platform:testClasses' as it has no actions.
:sdks:java:io:google-cloud-platform:testClasses (Thread[Execution **** for ':',5,main]) completed. Took 0.0 secs.
:sdks:java:io:google-cloud-platform:testJar (Thread[Execution **** for ':',5,main]) started.

> Task :runners:google-cloud-dataflow-java:****:legacy-****:compileJava FROM-CACHE
file or directory '<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/runners/google-cloud-dataflow-java/****/legacy-****/src/main/java',> not found
Custom actions are attached to task ':runners:google-cloud-dataflow-java:****:legacy-****:compileJava'.
Build cache key for task ':runners:google-cloud-dataflow-java:****:legacy-****:compileJava' is cf672da70ce14149e9f2f602b34780e4
Task ':runners:google-cloud-dataflow-java:****:legacy-****:compileJava' is not up-to-date because:
  No history is available.
Loaded cache entry for task ':runners:google-cloud-dataflow-java:****:legacy-****:compileJava' with cache key cf672da70ce14149e9f2f602b34780e4
:runners:google-cloud-dataflow-java:****:legacy-****:compileJava (Thread[Execution **** for ':' Thread 10,5,main]) completed. Took 0.39 secs.
:runners:google-cloud-dataflow-java:****:legacy-****:classes (Thread[Execution **** for ':' Thread 10,5,main]) started.

> Task :runners:google-cloud-dataflow-java:****:legacy-****:classes UP-TO-DATE
Skipping task ':runners:google-cloud-dataflow-java:****:legacy-****:classes' as it has no actions.
:runners:google-cloud-dataflow-java:****:legacy-****:classes (Thread[Execution **** for ':' Thread 10,5,main]) completed. Took 0.0 secs.
:runners:google-cloud-dataflow-java:****:legacy-****:shadowJar (Thread[Execution **** for ':' Thread 10,5,main]) started.

> Task :sdks:java:io:google-cloud-platform:testJar
Caching disabled for task ':sdks:java:io:google-cloud-platform:testJar' because:
  Caching has not been enabled for the task
Task ':sdks:java:io:google-cloud-platform:testJar' is not up-to-date because:
  No history is available.
:sdks:java:io:google-cloud-platform:testJar (Thread[Execution **** for ':',5,main]) completed. Took 0.363 secs.
:runners:google-cloud-dataflow-java:compileTestJava (Thread[Execution **** for ':',5,main]) started.

> Task :runners:google-cloud-dataflow-java:****:legacy-****:shadowJar FROM-CACHE
Custom actions are attached to task ':runners:google-cloud-dataflow-java:****:legacy-****:shadowJar'.
Build cache key for task ':runners:google-cloud-dataflow-java:****:legacy-****:shadowJar' is b03e31546baded1569b9e22785f539a5
Task ':runners:google-cloud-dataflow-java:****:legacy-****:shadowJar' is not up-to-date because:
  No history is available.
Loaded cache entry for task ':runners:google-cloud-dataflow-java:****:legacy-****:shadowJar' with cache key b03e31546baded1569b9e22785f539a5
:runners:google-cloud-dataflow-java:****:legacy-****:shadowJar (Thread[Execution **** for ':' Thread 10,5,main]) completed. Took 0.197 secs.

> Task :runners:google-cloud-dataflow-java:compileTestJava FROM-CACHE
Custom actions are attached to task ':runners:google-cloud-dataflow-java:compileTestJava'.
Build cache key for task ':runners:google-cloud-dataflow-java:compileTestJava' is 861c7435ffd549666858392e9bbd7747
Task ':runners:google-cloud-dataflow-java:compileTestJava' is not up-to-date because:
  No history is available.
Loaded cache entry for task ':runners:google-cloud-dataflow-java:compileTestJava' with cache key 861c7435ffd549666858392e9bbd7747
:runners:google-cloud-dataflow-java:compileTestJava (Thread[Execution **** for ':',5,main]) completed. Took 0.22 secs.
:runners:google-cloud-dataflow-java:testClasses (Thread[Execution **** for ':',5,main]) started.

> Task :runners:google-cloud-dataflow-java:testClasses UP-TO-DATE
Skipping task ':runners:google-cloud-dataflow-java:testClasses' as it has no actions.
:runners:google-cloud-dataflow-java:testClasses (Thread[Execution **** for ':',5,main]) completed. Took 0.0 secs.
:runners:google-cloud-dataflow-java:testJar (Thread[Execution **** for ':' Thread 7,5,main]) started.

> Task :runners:google-cloud-dataflow-java:testJar
Caching disabled for task ':runners:google-cloud-dataflow-java:testJar' because:
  Caching has not been enabled for the task
Task ':runners:google-cloud-dataflow-java:testJar' is not up-to-date because:
  No history is available.
file or directory '<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/runners/google-cloud-dataflow-java/build/resources/test',> not found
:runners:google-cloud-dataflow-java:testJar (Thread[Execution **** for ':' Thread 7,5,main]) completed. Took 0.041 secs.
:sdks:java:io:file-based-io-tests:integrationTest (Thread[Execution **** for ':' Thread 7,5,main]) started.
Gradle Test Executor 1 started executing tests.

> Task :sdks:java:io:file-based-io-tests:integrationTest
Custom actions are attached to task ':sdks:java:io:file-based-io-tests:integrationTest'.
Build cache key for task ':sdks:java:io:file-based-io-tests:integrationTest' is 2954f27e833e06d9b6038317a4b1cf00
Task ':sdks:java:io:file-based-io-tests:integrationTest' is not up-to-date because:
  Task.upToDateWhen is false.
Starting process 'Gradle Test Executor 1'. Working directory: <https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/file-based-io-tests> Command: /usr/lib/jvm/java-8-openjdk-amd64/bin/java -DbeamTestPipelineOptions=["--runner=DataflowRunner","--project=apache-beam-testing","--tempRoot=gs://temp-storage-for-perf-tests","--filenamePrefix=gs://temp-storage-for-perf-tests/beam_PerformanceTests_TFRecordIOIT/5312/","--bigQueryDataset=beam_performance","--bigQueryTable=tfrecordioit_results","--influxMeasurement=tfrecordioit_results","--numberOfRecords=18000000","--expectedHash=543104423f8b6eb097acb9f111c19fe4","--datasetSize=1019380000","--numWorkers=5","--autoscalingAlgorithm=NONE","--influxDatabase=beam_test_metrics","--influxHost=http://10.128.0.96:8086","--****HarnessContainerImage=","--dataflowWorkerJar=<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/runners/google-cloud-dataflow-java/****/legacy-****/build/libs/beam-runners-google-cloud-dataflow-java-legacy-****-2.34.0-SNAPSHOT.jar","--region=us-central1"]> -Djava.security.manager=****.org.gradle.process.internal.****.child.BootstrapSecurityManager -Dorg.gradle.native=false -Xmx2g -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant -ea -cp /home/jenkins/.gradle/caches/6.8.3/****Main/gradle-****.jar ****.org.gradle.process.internal.****.GradleWorkerMain 'Gradle Test Executor 1'
Successfully started process 'Gradle Test Executor 1'

org.apache.beam.sdk.io.tfrecord.TFRecordIOIT STANDARD_ERROR
    SLF4J: Class path contains multiple SLF4J bindings.
    SLF4J: Found binding in [jar:<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/runners/google-cloud-dataflow-java/****/legacy-****/build/libs/beam-runners-google-cloud-dataflow-java-legacy-****-2.34.0-SNAPSHOT.jar!/org/slf4j/impl/StaticLoggerBinder.class]>
    SLF4J: Found binding in [jar:file:/home/jenkins/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-log4j12/1.7.30/c21f55139d8141d2231214fb1feaf50a1edca95e/slf4j-log4j12-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class]
    SLF4J: Found binding in [jar:file:/home/jenkins/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-jdk14/1.7.30/d35953dd2fe54ebe39fdf18cfd82fe6eb35b25ed/slf4j-jdk14-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class]
    SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
    SLF4J: Actual binding is of type [org.slf4j.impl.JDK14LoggerFactory]

org.apache.beam.sdk.io.tfrecord.TFRecordIOIT > writeThenReadAll STANDARD_ERROR
    Sep 12, 2021 12:21:52 AM org.apache.beam.runners.dataflow.DataflowRunner validateSdkContainerImageOptions
    WARNING: Prefer --sdkContainerImage over deprecated legacy option --****HarnessContainerImage.
    Sep 12, 2021 12:21:53 AM org.apache.beam.sdk.extensions.gcp.options.GcpOptions$GcpTempLocationFactory tryCreateDefaultBucket
    INFO: No tempLocation specified, attempting to use default bucket: dataflow-staging-us-central1-844138762903
    Sep 12, 2021 12:21:54 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
    WARNING: Request failed with code 409, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/storage/v1/b?predefinedAcl=projectPrivate&predefinedDefaultObjectAcl=projectPrivate&project=apache-beam-testing. 
    Sep 12, 2021 12:21:54 AM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
    INFO: No stagingLocation provided, falling back to gcpTempLocation
    Sep 12, 2021 12:21:54 AM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
    INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 317 files. Enable logging at DEBUG level to see which files will be staged.
    Sep 12, 2021 12:21:54 AM org.apache.beam.runners.dataflow.DataflowRunner run
    INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
    Sep 12, 2021 12:21:57 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
    INFO: Uploading 318 files from PipelineOptions.filesToStage to staging location to prepare for execution.
    Sep 12, 2021 12:21:57 AM org.apache.beam.runners.dataflow.util.PackageUtil$PackageAttributes forFileToStage
    INFO: Staging custom dataflow-****.jar as beam-runners-google-cloud-dataflow-java-legacy-****-2.34.0-SNAPSHOT-YtvkOIsfQYfFzG89aVBO_8oNHoiij9SynwUJLdMkn5A.jar
    Sep 12, 2021 12:21:57 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
    INFO: Uploading /tmp/test4844225839052219753.zip to gs://dataflow-staging-us-central1-844138762903/temp/staging/test-0bUaTfKzP-CFWyMIv9IRgZu_fYmhBmFpNIU6GrDQTB0.jar
    Sep 12, 2021 12:21:58 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
    INFO: Staging files complete: 317 files cached, 1 files newly uploaded in 0 seconds
    Sep 12, 2021 12:21:58 AM org.apache.beam.runners.dataflow.DataflowRunner run
    INFO: Staging portable pipeline proto to gs://dataflow-staging-us-central1-844138762903/temp/staging/
    Sep 12, 2021 12:21:58 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
    INFO: Uploading <205290 bytes, hash faaa764196c770244bacf9945e022babf7f632b586257de722336314fbfd0c50> to gs://dataflow-staging-us-central1-844138762903/temp/staging/pipeline--qp2QZbHcCRLrPmUXgIrq_f2MrWGJX3nIjNjFPv9DFA.pb
    Sep 12, 2021 12:22:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Generate sequence/Read(BoundedCountingSource) as step s1
    Sep 12, 2021 12:22:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Produce text lines as step s2
    Sep 12, 2021 12:22:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Transform strings to bytes/Map as step s3
    Sep 12, 2021 12:22:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Record time before writing as step s4
    Sep 12, 2021 12:22:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/RewindowIntoGlobal/Window.Assign as step s5
    Sep 12, 2021 12:22:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles as step s6
    Sep 12, 2021 12:22:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten as step s7
    Sep 12, 2021 12:22:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/WriteUnshardedBundlesToTempFiles/WriteUnwritten as step s8
    Sep 12, 2021 12:22:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/WriteUnshardedBundlesToTempFiles/DropShardNum as step s9
    Sep 12, 2021 12:22:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/WriteUnshardedBundlesToTempFiles/Flatten.PCollections as step s10
    Sep 12, 2021 12:22:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow) as step s11
    Sep 12, 2021 12:22:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/GatherTempFileResults/View.AsList/CreateDataflowView as step s12
    Sep 12, 2021 12:22:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Create.Values/Read(CreateSource) as step s13
    Sep 12, 2021 12:22:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Reify.ReifyView/ParDo(Anonymous) as step s14
    Sep 12, 2021 12:22:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Values/Values/Map as step s15
    Sep 12, 2021 12:22:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Finalize as step s16
    Sep 12, 2021 12:22:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random key as step s17
    Sep 12, 2021 12:22:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign as step s18
    Sep 12, 2021 12:22:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey as step s19
    Sep 12, 2021 12:22:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable as step s20
    Sep 12, 2021 12:22:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Values/Values/Map as step s21
    Sep 12, 2021 12:22:00 AM org.apache.beam.runners.dataflow.DataflowRunner run
    INFO: Dataflow SDK version: 2.34.0-SNAPSHOT
    Sep 12, 2021 12:22:01 AM org.apache.beam.runners.dataflow.DataflowRunner run
    INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-09-11_17_22_00-10989387687034536051?project=apache-beam-testing
    Sep 12, 2021 12:22:01 AM org.apache.beam.runners.dataflow.DataflowRunner run
    INFO: Submitted job: 2021-09-11_17_22_00-10989387687034536051
    Sep 12, 2021 12:22:01 AM org.apache.beam.runners.dataflow.DataflowRunner run
    INFO: To cancel the job using the 'gcloud' tool, run:
    > gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-09-11_17_22_00-10989387687034536051
    Sep 12, 2021 12:22:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-12T00:22:19.428Z: Worker configuration: e2-standard-2 in us-central1-c.
    Sep 12, 2021 12:22:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    SEVERE: 2021-09-12T00:22:20.078Z: Workflow failed. Causes: Project apache-beam-testing has insufficient quota(s) to execute this workflow with 1 instances in region us-central1. Quota summary (required/available): 1/24365 instances, 2/0 CPUs, 25/176446 disk GB, 0/2397 SSD disk GB, 1/213 instance groups, 1/216 managed instance groups, 1/439 instance templates, 1/594 in-use IP addresses.

    Please see https://cloud.google.com/compute/docs/resource-quotas about requesting more quota.
    Sep 12, 2021 12:22:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-12T00:22:20.119Z: Cleaning up.
    Sep 12, 2021 12:22:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-12T00:22:20.172Z: Worker pool stopped.
    Sep 12, 2021 12:22:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-12T00:22:20.903Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
    Sep 12, 2021 12:22:27 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
    INFO: Job 2021-09-11_17_22_00-10989387687034536051 failed with status FAILED.

Gradle Test Executor 1 finished executing tests.

> Task :sdks:java:io:file-based-io-tests:integrationTest FAILED

org.apache.beam.sdk.io.tfrecord.TFRecordIOIT > writeThenReadAll FAILED
    java.lang.IllegalStateException: Unable to find any files matching gs://temp-storage-for-perf-tests/beam_PerformanceTests_TFRecordIOIT/5312/_1631406111632*
        at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:588)
        at org.apache.beam.sdk.io.TFRecordIO$Read.expand(TFRecordIO.java:191)
        at org.apache.beam.sdk.io.TFRecordIO$Read.expand(TFRecordIO.java:112)
        at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:548)
        at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:482)
        at org.apache.beam.sdk.values.PBegin.apply(PBegin.java:44)
        at org.apache.beam.sdk.Pipeline.apply(Pipeline.java:177)
        at org.apache.beam.sdk.io.tfrecord.TFRecordIOIT.writeThenReadAll(TFRecordIOIT.java:151)

1 test completed, 1 failed
Finished generating test XML results (0.021 secs) into: <https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/file-based-io-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.028 secs) into: <https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/file-based-io-tests/build/reports/tests/integrationTest>
:sdks:java:io:file-based-io-tests:integrationTest (Thread[Execution **** for ':' Thread 7,5,main]) completed. Took 39.925 secs.

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:io:file-based-io-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/file-based-io-tests/build/reports/tests/integrationTest/index.html>

* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 1m 9s
129 actionable tasks: 79 executed, 50 from cache

Publishing build scan...
https://gradle.com/s/awh3ehxtvyeks

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_PerformanceTests_TFRecordIOIT #5311

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/5311/display/redirect>

Changes:


------------------------------------------
[...truncated 274.16 KB...]
Task ':sdks:java:io:google-cloud-platform:jar' is not up-to-date because:
  No history is available.
file or directory '<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/google-cloud-platform/build/resources/main',> not found
:sdks:java:io:google-cloud-platform:jar (Thread[Daemon ****,5,main]) completed. Took 0.281 secs.
:runners:google-cloud-dataflow-java:compileJava (Thread[Daemon ****,5,main]) started.
:sdks:java:io:google-cloud-platform:compileTestJava (Thread[Execution **** for ':' Thread 5,5,main]) started.

> Task :runners:google-cloud-dataflow-java:compileJava FROM-CACHE
Custom actions are attached to task ':runners:google-cloud-dataflow-java:compileJava'.
Build cache key for task ':runners:google-cloud-dataflow-java:compileJava' is f4d45dae85b41e621c919183804b6afa
Task ':runners:google-cloud-dataflow-java:compileJava' is not up-to-date because:
  No history is available.
Loaded cache entry for task ':runners:google-cloud-dataflow-java:compileJava' with cache key f4d45dae85b41e621c919183804b6afa
:runners:google-cloud-dataflow-java:compileJava (Thread[Daemon ****,5,main]) completed. Took 0.234 secs.
:runners:google-cloud-dataflow-java:classes (Thread[Execution **** for ':' Thread 6,5,main]) started.

> Task :runners:google-cloud-dataflow-java:classes
Skipping task ':runners:google-cloud-dataflow-java:classes' as it has no actions.
:runners:google-cloud-dataflow-java:classes (Thread[Execution **** for ':' Thread 6,5,main]) completed. Took 0.0 secs.
:runners:google-cloud-dataflow-java:jar (Thread[Execution **** for ':' Thread 6,5,main]) started.

> Task :runners:google-cloud-dataflow-java:jar
Caching disabled for task ':runners:google-cloud-dataflow-java:jar' because:
  Caching has not been enabled for the task
Task ':runners:google-cloud-dataflow-java:jar' is not up-to-date because:
  No history is available.
:runners:google-cloud-dataflow-java:jar (Thread[Execution **** for ':' Thread 6,5,main]) completed. Took 0.097 secs.
:runners:google-cloud-dataflow-java:****:legacy-****:compileJava (Thread[Execution **** for ':' Thread 6,5,main]) started.

> Task :sdks:java:io:google-cloud-platform:compileTestJava FROM-CACHE
Custom actions are attached to task ':sdks:java:io:google-cloud-platform:compileTestJava'.
Build cache key for task ':sdks:java:io:google-cloud-platform:compileTestJava' is 99462c52f9bbbb5d949b676d07ee3bc7
Task ':sdks:java:io:google-cloud-platform:compileTestJava' is not up-to-date because:
  No history is available.
Loaded cache entry for task ':sdks:java:io:google-cloud-platform:compileTestJava' with cache key 99462c52f9bbbb5d949b676d07ee3bc7
:sdks:java:io:google-cloud-platform:compileTestJava (Thread[Execution **** for ':' Thread 5,5,main]) completed. Took 0.513 secs.
:sdks:java:io:google-cloud-platform:testClasses (Thread[Execution **** for ':' Thread 5,5,main]) started.

> Task :sdks:java:io:google-cloud-platform:testClasses
Skipping task ':sdks:java:io:google-cloud-platform:testClasses' as it has no actions.
:sdks:java:io:google-cloud-platform:testClasses (Thread[Execution **** for ':' Thread 5,5,main]) completed. Took 0.0 secs.
:sdks:java:io:google-cloud-platform:testJar (Thread[Execution **** for ':' Thread 5,5,main]) started.

> Task :runners:google-cloud-dataflow-java:****:legacy-****:compileJava FROM-CACHE
file or directory '<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/runners/google-cloud-dataflow-java/****/legacy-****/src/main/java',> not found
Custom actions are attached to task ':runners:google-cloud-dataflow-java:****:legacy-****:compileJava'.
Build cache key for task ':runners:google-cloud-dataflow-java:****:legacy-****:compileJava' is cf672da70ce14149e9f2f602b34780e4
Task ':runners:google-cloud-dataflow-java:****:legacy-****:compileJava' is not up-to-date because:
  No history is available.
Loaded cache entry for task ':runners:google-cloud-dataflow-java:****:legacy-****:compileJava' with cache key cf672da70ce14149e9f2f602b34780e4
:runners:google-cloud-dataflow-java:****:legacy-****:compileJava (Thread[Execution **** for ':' Thread 6,5,main]) completed. Took 0.334 secs.
:runners:google-cloud-dataflow-java:****:legacy-****:classes (Thread[Execution **** for ':' Thread 6,5,main]) started.

> Task :runners:google-cloud-dataflow-java:****:legacy-****:classes UP-TO-DATE
Skipping task ':runners:google-cloud-dataflow-java:****:legacy-****:classes' as it has no actions.
:runners:google-cloud-dataflow-java:****:legacy-****:classes (Thread[Execution **** for ':' Thread 6,5,main]) completed. Took 0.0 secs.
:runners:google-cloud-dataflow-java:****:legacy-****:shadowJar (Thread[Execution **** for ':' Thread 6,5,main]) started.

> Task :sdks:java:io:google-cloud-platform:testJar
Caching disabled for task ':sdks:java:io:google-cloud-platform:testJar' because:
  Caching has not been enabled for the task
Task ':sdks:java:io:google-cloud-platform:testJar' is not up-to-date because:
  No history is available.
:sdks:java:io:google-cloud-platform:testJar (Thread[Execution **** for ':' Thread 5,5,main]) completed. Took 0.324 secs.
:runners:google-cloud-dataflow-java:compileTestJava (Thread[Execution **** for ':' Thread 5,5,main]) started.

> Task :runners:google-cloud-dataflow-java:****:legacy-****:shadowJar FROM-CACHE
Custom actions are attached to task ':runners:google-cloud-dataflow-java:****:legacy-****:shadowJar'.
Build cache key for task ':runners:google-cloud-dataflow-java:****:legacy-****:shadowJar' is 165b57b3de0480886088cb77a0fc5516
Task ':runners:google-cloud-dataflow-java:****:legacy-****:shadowJar' is not up-to-date because:
  No history is available.
Loaded cache entry for task ':runners:google-cloud-dataflow-java:****:legacy-****:shadowJar' with cache key 165b57b3de0480886088cb77a0fc5516
:runners:google-cloud-dataflow-java:****:legacy-****:shadowJar (Thread[Execution **** for ':' Thread 6,5,main]) completed. Took 0.205 secs.

> Task :runners:google-cloud-dataflow-java:compileTestJava FROM-CACHE
Custom actions are attached to task ':runners:google-cloud-dataflow-java:compileTestJava'.
Build cache key for task ':runners:google-cloud-dataflow-java:compileTestJava' is 861c7435ffd549666858392e9bbd7747
Task ':runners:google-cloud-dataflow-java:compileTestJava' is not up-to-date because:
  No history is available.
Loaded cache entry for task ':runners:google-cloud-dataflow-java:compileTestJava' with cache key 861c7435ffd549666858392e9bbd7747
:runners:google-cloud-dataflow-java:compileTestJava (Thread[Execution **** for ':' Thread 5,5,main]) completed. Took 0.199 secs.
:runners:google-cloud-dataflow-java:testClasses (Thread[Execution **** for ':' Thread 5,5,main]) started.

> Task :runners:google-cloud-dataflow-java:testClasses UP-TO-DATE
Skipping task ':runners:google-cloud-dataflow-java:testClasses' as it has no actions.
:runners:google-cloud-dataflow-java:testClasses (Thread[Execution **** for ':' Thread 5,5,main]) completed. Took 0.0 secs.
:runners:google-cloud-dataflow-java:testJar (Thread[Execution **** for ':' Thread 5,5,main]) started.

> Task :runners:google-cloud-dataflow-java:testJar
Caching disabled for task ':runners:google-cloud-dataflow-java:testJar' because:
  Caching has not been enabled for the task
Task ':runners:google-cloud-dataflow-java:testJar' is not up-to-date because:
  No history is available.
file or directory '<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/runners/google-cloud-dataflow-java/build/resources/test',> not found
:runners:google-cloud-dataflow-java:testJar (Thread[Execution **** for ':' Thread 5,5,main]) completed. Took 0.038 secs.
:sdks:java:io:file-based-io-tests:integrationTest (Thread[Execution **** for ':' Thread 5,5,main]) started.
Gradle Test Executor 1 started executing tests.

> Task :sdks:java:io:file-based-io-tests:integrationTest
Custom actions are attached to task ':sdks:java:io:file-based-io-tests:integrationTest'.
Build cache key for task ':sdks:java:io:file-based-io-tests:integrationTest' is d113188b16b5dc9e6e52b7033c284315
Task ':sdks:java:io:file-based-io-tests:integrationTest' is not up-to-date because:
  Task.upToDateWhen is false.
Starting process 'Gradle Test Executor 1'. Working directory: <https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/file-based-io-tests> Command: /usr/lib/jvm/java-8-openjdk-amd64/bin/java -DbeamTestPipelineOptions=["--runner=DataflowRunner","--project=apache-beam-testing","--tempRoot=gs://temp-storage-for-perf-tests","--filenamePrefix=gs://temp-storage-for-perf-tests/beam_PerformanceTests_TFRecordIOIT/5311/","--bigQueryDataset=beam_performance","--bigQueryTable=tfrecordioit_results","--influxMeasurement=tfrecordioit_results","--numberOfRecords=18000000","--expectedHash=543104423f8b6eb097acb9f111c19fe4","--datasetSize=1019380000","--numWorkers=5","--autoscalingAlgorithm=NONE","--influxDatabase=beam_test_metrics","--influxHost=http://10.128.0.96:8086","--****HarnessContainerImage=","--dataflowWorkerJar=<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/runners/google-cloud-dataflow-java/****/legacy-****/build/libs/beam-runners-google-cloud-dataflow-java-legacy-****-2.34.0-SNAPSHOT.jar","--region=us-central1"]> -Djava.security.manager=****.org.gradle.process.internal.****.child.BootstrapSecurityManager -Dorg.gradle.native=false -Xmx2g -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant -ea -cp /home/jenkins/.gradle/caches/6.8.3/****Main/gradle-****.jar ****.org.gradle.process.internal.****.GradleWorkerMain 'Gradle Test Executor 1'
Successfully started process 'Gradle Test Executor 1'

org.apache.beam.sdk.io.tfrecord.TFRecordIOIT STANDARD_ERROR
    SLF4J: Class path contains multiple SLF4J bindings.
    SLF4J: Found binding in [jar:<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/runners/google-cloud-dataflow-java/****/legacy-****/build/libs/beam-runners-google-cloud-dataflow-java-legacy-****-2.34.0-SNAPSHOT.jar!/org/slf4j/impl/StaticLoggerBinder.class]>
    SLF4J: Found binding in [jar:file:/home/jenkins/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-log4j12/1.7.30/c21f55139d8141d2231214fb1feaf50a1edca95e/slf4j-log4j12-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class]
    SLF4J: Found binding in [jar:file:/home/jenkins/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-jdk14/1.7.30/d35953dd2fe54ebe39fdf18cfd82fe6eb35b25ed/slf4j-jdk14-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class]
    SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
    SLF4J: Actual binding is of type [org.slf4j.impl.JDK14LoggerFactory]

org.apache.beam.sdk.io.tfrecord.TFRecordIOIT > writeThenReadAll STANDARD_ERROR
    Sep 11, 2021 6:21:59 PM org.apache.beam.runners.dataflow.DataflowRunner validateSdkContainerImageOptions
    WARNING: Prefer --sdkContainerImage over deprecated legacy option --****HarnessContainerImage.
    Sep 11, 2021 6:22:00 PM org.apache.beam.sdk.extensions.gcp.options.GcpOptions$GcpTempLocationFactory tryCreateDefaultBucket
    INFO: No tempLocation specified, attempting to use default bucket: dataflow-staging-us-central1-844138762903
    Sep 11, 2021 6:22:00 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
    WARNING: Request failed with code 409, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/storage/v1/b?predefinedAcl=projectPrivate&predefinedDefaultObjectAcl=projectPrivate&project=apache-beam-testing. 
    Sep 11, 2021 6:22:01 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
    INFO: No stagingLocation provided, falling back to gcpTempLocation
    Sep 11, 2021 6:22:01 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
    INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 317 files. Enable logging at DEBUG level to see which files will be staged.
    Sep 11, 2021 6:22:01 PM org.apache.beam.runners.dataflow.DataflowRunner run
    INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
    Sep 11, 2021 6:22:04 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
    INFO: Uploading 318 files from PipelineOptions.filesToStage to staging location to prepare for execution.
    Sep 11, 2021 6:22:04 PM org.apache.beam.runners.dataflow.util.PackageUtil$PackageAttributes forFileToStage
    INFO: Staging custom dataflow-****.jar as beam-runners-google-cloud-dataflow-java-legacy-****-2.34.0-SNAPSHOT-YtvkOIsfQYfFzG89aVBO_8oNHoiij9SynwUJLdMkn5A.jar
    Sep 11, 2021 6:22:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
    INFO: Uploading /tmp/test7839630842594335631.zip to gs://dataflow-staging-us-central1-844138762903/temp/staging/test-EjEnhFZm60qK1ofux94K8GBewuZa4N2BNWjnsZJ4SRw.jar
    Sep 11, 2021 6:22:05 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
    INFO: Staging files complete: 317 files cached, 1 files newly uploaded in 0 seconds
    Sep 11, 2021 6:22:05 PM org.apache.beam.runners.dataflow.DataflowRunner run
    INFO: Staging portable pipeline proto to gs://dataflow-staging-us-central1-844138762903/temp/staging/
    Sep 11, 2021 6:22:05 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
    INFO: Uploading <205290 bytes, hash 41f133ec36858e9ff94e2281a4804de6e71c0af6c55b60f1921d11e2a84809c6> to gs://dataflow-staging-us-central1-844138762903/temp/staging/pipeline-QfEz7DaFjp_5TiKBpIBN5uccCvbFW2Dxkh0R4qhICcY.pb
    Sep 11, 2021 6:22:07 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Generate sequence/Read(BoundedCountingSource) as step s1
    Sep 11, 2021 6:22:07 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Produce text lines as step s2
    Sep 11, 2021 6:22:07 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Transform strings to bytes/Map as step s3
    Sep 11, 2021 6:22:07 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Record time before writing as step s4
    Sep 11, 2021 6:22:07 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/RewindowIntoGlobal/Window.Assign as step s5
    Sep 11, 2021 6:22:07 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles as step s6
    Sep 11, 2021 6:22:07 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten as step s7
    Sep 11, 2021 6:22:07 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/WriteUnshardedBundlesToTempFiles/WriteUnwritten as step s8
    Sep 11, 2021 6:22:07 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/WriteUnshardedBundlesToTempFiles/DropShardNum as step s9
    Sep 11, 2021 6:22:07 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/WriteUnshardedBundlesToTempFiles/Flatten.PCollections as step s10
    Sep 11, 2021 6:22:07 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow) as step s11
    Sep 11, 2021 6:22:07 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/GatherTempFileResults/View.AsList/CreateDataflowView as step s12
    Sep 11, 2021 6:22:07 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Create.Values/Read(CreateSource) as step s13
    Sep 11, 2021 6:22:07 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Reify.ReifyView/ParDo(Anonymous) as step s14
    Sep 11, 2021 6:22:07 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Values/Values/Map as step s15
    Sep 11, 2021 6:22:07 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Finalize as step s16
    Sep 11, 2021 6:22:07 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random key as step s17
    Sep 11, 2021 6:22:07 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign as step s18
    Sep 11, 2021 6:22:07 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey as step s19
    Sep 11, 2021 6:22:07 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable as step s20
    Sep 11, 2021 6:22:07 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Values/Values/Map as step s21
    Sep 11, 2021 6:22:07 PM org.apache.beam.runners.dataflow.DataflowRunner run
    INFO: Dataflow SDK version: 2.34.0-SNAPSHOT
    Sep 11, 2021 6:22:08 PM org.apache.beam.runners.dataflow.DataflowRunner run
    INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-09-11_11_22_07-9167581997166830178?project=apache-beam-testing
    Sep 11, 2021 6:22:08 PM org.apache.beam.runners.dataflow.DataflowRunner run
    INFO: Submitted job: 2021-09-11_11_22_07-9167581997166830178
    Sep 11, 2021 6:22:08 PM org.apache.beam.runners.dataflow.DataflowRunner run
    INFO: To cancel the job using the 'gcloud' tool, run:
    > gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-09-11_11_22_07-9167581997166830178
    Sep 11, 2021 6:22:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-11T18:22:25.828Z: Worker configuration: e2-standard-2 in us-central1-a.
    Sep 11, 2021 6:22:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    SEVERE: 2021-09-11T18:22:26.523Z: Workflow failed. Causes: Project apache-beam-testing has insufficient quota(s) to execute this workflow with 1 instances in region us-central1. Quota summary (required/available): 1/24366 instances, 2/0 CPUs, 25/188696 disk GB, 0/2397 SSD disk GB, 1/249 instance groups, 1/253 managed instance groups, 1/476 instance templates, 1/595 in-use IP addresses.

    Please see https://cloud.google.com/compute/docs/resource-quotas about requesting more quota.
    Sep 11, 2021 6:22:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-11T18:22:26.565Z: Cleaning up.
    Sep 11, 2021 6:22:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-11T18:22:26.617Z: Worker pool stopped.
    Sep 11, 2021 6:22:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-11T18:22:27.801Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
    Sep 11, 2021 6:22:31 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
    INFO: Job 2021-09-11_11_22_07-9167581997166830178 failed with status FAILED.

Gradle Test Executor 1 finished executing tests.

> Task :sdks:java:io:file-based-io-tests:integrationTest FAILED

org.apache.beam.sdk.io.tfrecord.TFRecordIOIT > writeThenReadAll FAILED
    java.lang.IllegalStateException: Unable to find any files matching gs://temp-storage-for-perf-tests/beam_PerformanceTests_TFRecordIOIT/5311/_1631384518411*
        at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:588)
        at org.apache.beam.sdk.io.TFRecordIO$Read.expand(TFRecordIO.java:191)
        at org.apache.beam.sdk.io.TFRecordIO$Read.expand(TFRecordIO.java:112)
        at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:548)
        at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:482)
        at org.apache.beam.sdk.values.PBegin.apply(PBegin.java:44)
        at org.apache.beam.sdk.Pipeline.apply(Pipeline.java:177)
        at org.apache.beam.sdk.io.tfrecord.TFRecordIOIT.writeThenReadAll(TFRecordIOIT.java:151)

1 test completed, 1 failed
Finished generating test XML results (0.02 secs) into: <https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/file-based-io-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.03 secs) into: <https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/file-based-io-tests/build/reports/tests/integrationTest>
:sdks:java:io:file-based-io-tests:integrationTest (Thread[Execution **** for ':' Thread 5,5,main]) completed. Took 37.238 secs.

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:io:file-based-io-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/file-based-io-tests/build/reports/tests/integrationTest/index.html>

* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 1m 11s
129 actionable tasks: 79 executed, 50 from cache

Publishing build scan...
https://gradle.com/s/m4ivvg3jxc42g

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_PerformanceTests_TFRecordIOIT #5310

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/5310/display/redirect>

Changes:


------------------------------------------
[...truncated 292.45 KB...]
> Task :sdks:java:io:parquet:testJar
Caching disabled for task ':sdks:java:io:parquet:testJar' because:
  Caching has not been enabled for the task
Task ':sdks:java:io:parquet:testJar' is not up-to-date because:
  No history is available.
:sdks:java:io:parquet:testJar (Thread[Execution **** for ':' Thread 9,5,main]) completed. Took 0.009 secs.

> Task :sdks:java:io:xml:compileJava
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Invalidating in-memory cache of /home/jenkins/.gradle/caches/6.8.3/javaCompile/jarAnalysis.bin
Created classpath snapshot for incremental compilation in 0.115 secs. 1 duplicate classes found in classpath (see all with --debug).
Invalidating in-memory cache of /home/jenkins/.gradle/caches/journal-1/file-access.bin
Stored cache entry for task ':sdks:java:io:xml:compileJava' with cache key 8dab0cbeb3a5b05abb991284acd12372
:sdks:java:io:xml:compileJava (Thread[Execution **** for ':' Thread 7,5,main]) completed. Took 1 mins 13.228 secs.
:sdks:java:io:xml:classes (Thread[Execution **** for ':' Thread 7,5,main]) started.

> Task :sdks:java:io:xml:classes
Skipping task ':sdks:java:io:xml:classes' as it has no actions.
:sdks:java:io:xml:classes (Thread[Execution **** for ':' Thread 7,5,main]) completed. Took 0.0 secs.
:sdks:java:io:xml:jar (Thread[Execution **** for ':' Thread 7,5,main]) started.

> Task :sdks:java:io:xml:jar
Caching disabled for task ':sdks:java:io:xml:jar' because:
  Caching has not been enabled for the task
Task ':sdks:java:io:xml:jar' is not up-to-date because:
  No history is available.
file or directory '<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/xml/build/resources/main',> not found
:sdks:java:io:xml:jar (Thread[Execution **** for ':' Thread 7,5,main]) completed. Took 0.031 secs.
:sdks:java:io:xml:compileTestJava (Thread[Execution **** for ':' Thread 4,5,main]) started.

> Task :sdks:java:io:xml:compileTestJava
Invalidating in-memory cache of /home/jenkins/.gradle/caches/6.8.3/fileHashes/fileHashes.bin
Invalidating in-memory cache of /home/jenkins/.gradle/caches/6.8.3/fileHashes/resourceHashesCache.bin
Custom actions are attached to task ':sdks:java:io:xml:compileTestJava'.
Build cache key for task ':sdks:java:io:xml:compileTestJava' is a9c21598404b4d5d5967a46a3a47333b
Task ':sdks:java:io:xml:compileTestJava' is not up-to-date because:
  No history is available.
The input changes require a full rebuild for incremental task ':sdks:java:io:xml:compileTestJava'.
Full recompilation is required because no incremental change information is available. This is usually caused by clean builds or changing compiler arguments.
Invalidating in-memory cache of /home/jenkins/.gradle/caches/6.8.3/fileContent/java-modules.bin
Invalidating in-memory cache of /home/jenkins/.gradle/caches/6.8.3/fileContent/annotation-processors.bin
Compiling with JDK Java compiler API.
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Invalidating in-memory cache of /home/jenkins/.gradle/caches/6.8.3/javaCompile/jarAnalysis.bin
Created classpath snapshot for incremental compilation in 0.255 secs. 1075 duplicate classes found in classpath (see all with --debug).
Invalidating in-memory cache of /home/jenkins/.gradle/caches/journal-1/file-access.bin
Stored cache entry for task ':sdks:java:io:xml:compileTestJava' with cache key a9c21598404b4d5d5967a46a3a47333b
:sdks:java:io:xml:compileTestJava (Thread[Execution **** for ':' Thread 4,5,main]) completed. Took 9.566 secs.
:sdks:java:io:xml:testClasses (Thread[Execution **** for ':' Thread 4,5,main]) started.

> Task :sdks:java:io:xml:testClasses
Skipping task ':sdks:java:io:xml:testClasses' as it has no actions.
:sdks:java:io:xml:testClasses (Thread[Execution **** for ':' Thread 4,5,main]) completed. Took 0.0 secs.
:sdks:java:io:xml:testJar (Thread[Execution **** for ':' Thread 4,5,main]) started.

> Task :sdks:java:io:xml:testJar
Caching disabled for task ':sdks:java:io:xml:testJar' because:
  Caching has not been enabled for the task
Task ':sdks:java:io:xml:testJar' is not up-to-date because:
  No history is available.
file or directory '<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/xml/build/resources/test',> not found
:sdks:java:io:xml:testJar (Thread[Execution **** for ':' Thread 4,5,main]) completed. Took 0.038 secs.
:sdks:java:io:file-based-io-tests:compileTestJava (Thread[Execution **** for ':' Thread 9,5,main]) started.

> Task :sdks:java:io:file-based-io-tests:compileTestJava
Invalidating in-memory cache of /home/jenkins/.gradle/caches/6.8.3/fileHashes/fileHashes.bin
Invalidating in-memory cache of /home/jenkins/.gradle/caches/6.8.3/fileHashes/resourceHashesCache.bin
Custom actions are attached to task ':sdks:java:io:file-based-io-tests:compileTestJava'.
Build cache key for task ':sdks:java:io:file-based-io-tests:compileTestJava' is 338c1e5cd51c0d1cb4c8e1b2aabeba08
Task ':sdks:java:io:file-based-io-tests:compileTestJava' is not up-to-date because:
  No history is available.
The input changes require a full rebuild for incremental task ':sdks:java:io:file-based-io-tests:compileTestJava'.
Full recompilation is required because no incremental change information is available. This is usually caused by clean builds or changing compiler arguments.
Invalidating in-memory cache of /home/jenkins/.gradle/caches/6.8.3/fileContent/java-modules.bin
Invalidating in-memory cache of /home/jenkins/.gradle/caches/6.8.3/fileContent/annotation-processors.bin
Compiling with JDK Java compiler API.
Note: <https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/file-based-io-tests/src/test/java/org/apache/beam/sdk/io/tfrecord/TFRecordIOIT.java> uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Created classpath snapshot for incremental compilation in 0.499 secs. 1748 duplicate classes found in classpath (see all with --debug).
Invalidating in-memory cache of /home/jenkins/.gradle/caches/journal-1/file-access.bin
Stored cache entry for task ':sdks:java:io:file-based-io-tests:compileTestJava' with cache key 338c1e5cd51c0d1cb4c8e1b2aabeba08
:sdks:java:io:file-based-io-tests:compileTestJava (Thread[Execution **** for ':' Thread 9,5,main]) completed. Took 12.892 secs.
:sdks:java:io:file-based-io-tests:testClasses (Thread[Execution **** for ':' Thread 9,5,main]) started.

> Task :sdks:java:io:file-based-io-tests:testClasses
Skipping task ':sdks:java:io:file-based-io-tests:testClasses' as it has no actions.
:sdks:java:io:file-based-io-tests:testClasses (Thread[Execution **** for ':' Thread 9,5,main]) completed. Took 0.0 secs.
:sdks:java:io:file-based-io-tests:integrationTest (Thread[Execution **** for ':' Thread 9,5,main]) started.

> Task :sdks:java:io:file-based-io-tests:integrationTest
Invalidating in-memory cache of /home/jenkins/.gradle/caches/6.8.3/fileHashes/fileHashes.bin
Invalidating in-memory cache of /home/jenkins/.gradle/caches/6.8.3/fileHashes/resourceHashesCache.bin
Custom actions are attached to task ':sdks:java:io:file-based-io-tests:integrationTest'.
Build cache key for task ':sdks:java:io:file-based-io-tests:integrationTest' is d113188b16b5dc9e6e52b7033c284315
Task ':sdks:java:io:file-based-io-tests:integrationTest' is not up-to-date because:
  Task.upToDateWhen is false.
Starting process 'Gradle Test Executor 3'. Working directory: <https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/file-based-io-tests> Command: /usr/lib/jvm/java-8-openjdk-amd64/bin/java -DbeamTestPipelineOptions=["--runner=DataflowRunner","--project=apache-beam-testing","--tempRoot=gs://temp-storage-for-perf-tests","--filenamePrefix=gs://temp-storage-for-perf-tests/beam_PerformanceTests_TFRecordIOIT/5310/","--bigQueryDataset=beam_performance","--bigQueryTable=tfrecordioit_results","--influxMeasurement=tfrecordioit_results","--numberOfRecords=18000000","--expectedHash=543104423f8b6eb097acb9f111c19fe4","--datasetSize=1019380000","--numWorkers=5","--autoscalingAlgorithm=NONE","--influxDatabase=beam_test_metrics","--influxHost=http://10.128.0.96:8086","--****HarnessContainerImage=","--dataflowWorkerJar=<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/runners/google-cloud-dataflow-java/****/legacy-****/build/libs/beam-runners-google-cloud-dataflow-java-legacy-****-2.34.0-SNAPSHOT.jar","--region=us-central1"]> -Djava.security.manager=****.org.gradle.process.internal.****.child.BootstrapSecurityManager -Dorg.gradle.native=false -Xmx2g -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant -ea -cp /home/jenkins/.gradle/caches/6.8.3/****Main/gradle-****.jar ****.org.gradle.process.internal.****.GradleWorkerMain 'Gradle Test Executor 3'
Successfully started process 'Gradle Test Executor 3'

Gradle Test Executor 3 started executing tests.

> Task :sdks:java:io:file-based-io-tests:integrationTest

org.apache.beam.sdk.io.tfrecord.TFRecordIOIT STANDARD_ERROR
    SLF4J: Class path contains multiple SLF4J bindings.
    SLF4J: Found binding in [jar:<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/runners/google-cloud-dataflow-java/****/legacy-****/build/libs/beam-runners-google-cloud-dataflow-java-legacy-****-2.34.0-SNAPSHOT.jar!/org/slf4j/impl/StaticLoggerBinder.class]>
    SLF4J: Found binding in [jar:file:/home/jenkins/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-log4j12/1.7.30/c21f55139d8141d2231214fb1feaf50a1edca95e/slf4j-log4j12-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class]
    SLF4J: Found binding in [jar:file:/home/jenkins/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-jdk14/1.7.30/d35953dd2fe54ebe39fdf18cfd82fe6eb35b25ed/slf4j-jdk14-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class]
    SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
    SLF4J: Actual binding is of type [org.slf4j.impl.JDK14LoggerFactory]

org.apache.beam.sdk.io.tfrecord.TFRecordIOIT > writeThenReadAll STANDARD_ERROR
    Sep 11, 2021 12:26:44 PM org.apache.beam.runners.dataflow.DataflowRunner validateSdkContainerImageOptions
    WARNING: Prefer --sdkContainerImage over deprecated legacy option --****HarnessContainerImage.
    Sep 11, 2021 12:26:49 PM org.apache.beam.sdk.extensions.gcp.options.GcpOptions$GcpTempLocationFactory tryCreateDefaultBucket
    INFO: No tempLocation specified, attempting to use default bucket: dataflow-staging-us-central1-844138762903
    Sep 11, 2021 12:26:50 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
    WARNING: Request failed with code 409, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/storage/v1/b?predefinedAcl=projectPrivate&predefinedDefaultObjectAcl=projectPrivate&project=apache-beam-testing. 
    Sep 11, 2021 12:26:50 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
    INFO: No stagingLocation provided, falling back to gcpTempLocation
    Sep 11, 2021 12:26:53 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
    INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 317 files. Enable logging at DEBUG level to see which files will be staged.
    Sep 11, 2021 12:26:53 PM org.apache.beam.runners.dataflow.DataflowRunner run
    INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
    Sep 11, 2021 12:27:04 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
    INFO: Uploading 318 files from PipelineOptions.filesToStage to staging location to prepare for execution.
    Sep 11, 2021 12:27:04 PM org.apache.beam.runners.dataflow.util.PackageUtil$PackageAttributes forFileToStage
    INFO: Staging custom dataflow-****.jar as beam-runners-google-cloud-dataflow-java-legacy-****-2.34.0-SNAPSHOT-YtvkOIsfQYfFzG89aVBO_8oNHoiij9SynwUJLdMkn5A.jar
    Sep 11, 2021 12:27:06 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
    INFO: Uploading /tmp/test2787157726218378138.zip to gs://dataflow-staging-us-central1-844138762903/temp/staging/test-9_j75KCBsKQVUvFeA7CC8aaqh462lEsYIskGvyKobWI.jar
    Sep 11, 2021 12:27:12 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
    INFO: Staging files complete: 317 files cached, 1 files newly uploaded in 8 seconds
    Sep 11, 2021 12:27:13 PM org.apache.beam.runners.dataflow.DataflowRunner run
    INFO: Staging portable pipeline proto to gs://dataflow-staging-us-central1-844138762903/temp/staging/
    Sep 11, 2021 12:27:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
    INFO: Uploading <205290 bytes, hash 3e0644f0578a52252a9a5420d9ac9071800c4843c82dc806a876aaead5619515> to gs://dataflow-staging-us-central1-844138762903/temp/staging/pipeline-PgZE8FeKUiUqmlQg2ayQcYAMSEPILcgGqHaq6tVhlRU.pb
    Sep 11, 2021 12:27:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Generate sequence/Read(BoundedCountingSource) as step s1
    Sep 11, 2021 12:27:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Produce text lines as step s2
    Sep 11, 2021 12:27:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Transform strings to bytes/Map as step s3
    Sep 11, 2021 12:27:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Record time before writing as step s4
    Sep 11, 2021 12:27:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/RewindowIntoGlobal/Window.Assign as step s5
    Sep 11, 2021 12:27:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles as step s6
    Sep 11, 2021 12:27:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten as step s7
    Sep 11, 2021 12:27:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/WriteUnshardedBundlesToTempFiles/WriteUnwritten as step s8
    Sep 11, 2021 12:27:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/WriteUnshardedBundlesToTempFiles/DropShardNum as step s9
    Sep 11, 2021 12:27:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/WriteUnshardedBundlesToTempFiles/Flatten.PCollections as step s10
    Sep 11, 2021 12:27:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow) as step s11
    Sep 11, 2021 12:27:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/GatherTempFileResults/View.AsList/CreateDataflowView as step s12
    Sep 11, 2021 12:27:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Create.Values/Read(CreateSource) as step s13
    Sep 11, 2021 12:27:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Reify.ReifyView/ParDo(Anonymous) as step s14
    Sep 11, 2021 12:27:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Values/Values/Map as step s15
    Sep 11, 2021 12:27:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Finalize as step s16
    Sep 11, 2021 12:27:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random key as step s17
    Sep 11, 2021 12:27:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign as step s18
    Sep 11, 2021 12:27:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey as step s19
    Sep 11, 2021 12:27:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable as step s20
    Sep 11, 2021 12:27:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Values/Values/Map as step s21
    Sep 11, 2021 12:27:22 PM org.apache.beam.runners.dataflow.DataflowRunner run
    INFO: Dataflow SDK version: 2.34.0-SNAPSHOT
    Sep 11, 2021 12:27:23 PM org.apache.beam.runners.dataflow.DataflowRunner run
    INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-09-11_05_27_22-15014109773693637146?project=apache-beam-testing
    Sep 11, 2021 12:27:23 PM org.apache.beam.runners.dataflow.DataflowRunner run
    INFO: Submitted job: 2021-09-11_05_27_22-15014109773693637146
    Sep 11, 2021 12:27:23 PM org.apache.beam.runners.dataflow.DataflowRunner run
    INFO: To cancel the job using the 'gcloud' tool, run:
    > gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-09-11_05_27_22-15014109773693637146
    Sep 11, 2021 12:27:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-11T12:27:44.838Z: Worker configuration: e2-standard-2 in us-central1-c.
    Sep 11, 2021 12:27:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    SEVERE: 2021-09-11T12:27:45.416Z: Workflow failed. Causes: Project apache-beam-testing has insufficient quota(s) to execute this workflow with 1 instances in region us-central1. Quota summary (required/available): 1/24361 instances, 2/0 CPUs, 25/182106 disk GB, 0/2397 SSD disk GB, 1/208 instance groups, 1/212 managed instance groups, 1/436 instance templates, 1/590 in-use IP addresses.

    Please see https://cloud.google.com/compute/docs/resource-quotas about requesting more quota.
    Sep 11, 2021 12:27:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-11T12:27:45.457Z: Cleaning up.
    Sep 11, 2021 12:27:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-11T12:27:45.524Z: Worker pool stopped.
    Sep 11, 2021 12:27:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-11T12:27:46.696Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
    Sep 11, 2021 12:27:51 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
    INFO: Job 2021-09-11_05_27_22-15014109773693637146 failed with status FAILED.

org.apache.beam.sdk.io.tfrecord.TFRecordIOIT > writeThenReadAll FAILED
    java.lang.IllegalStateException: Unable to find any files matching gs://temp-storage-for-perf-tests/beam_PerformanceTests_TFRecordIOIT/5310/_1631363198956*
        at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:588)
        at org.apache.beam.sdk.io.TFRecordIO$Read.expand(TFRecordIO.java:191)
        at org.apache.beam.sdk.io.TFRecordIO$Read.expand(TFRecordIO.java:112)
        at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:548)
        at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:482)
        at org.apache.beam.sdk.values.PBegin.apply(PBegin.java:44)
        at org.apache.beam.sdk.Pipeline.apply(Pipeline.java:177)
        at org.apache.beam.sdk.io.tfrecord.TFRecordIOIT.writeThenReadAll(TFRecordIOIT.java:151)

Gradle Test Executor 3 finished executing tests.

> Task :sdks:java:io:file-based-io-tests:integrationTest FAILED

1 test completed, 1 failed
Finished generating test XML results (0.111 secs) into: <https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/file-based-io-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.094 secs) into: <https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/file-based-io-tests/build/reports/tests/integrationTest>
:sdks:java:io:file-based-io-tests:integrationTest (Thread[Execution **** for ':' Thread 9,5,main]) completed. Took 1 mins 34.446 secs.

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:io:file-based-io-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/file-based-io-tests/build/reports/tests/integrationTest/index.html>

* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 5m 29s
129 actionable tasks: 83 executed, 46 from cache

Publishing build scan...
https://gradle.com/s/hoskryqrahgka

Stopped 2 **** daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_PerformanceTests_TFRecordIOIT #5309

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/5309/display/redirect>

Changes:


------------------------------------------
[...truncated 285.46 KB...]
Loaded cache entry for task ':runners:google-cloud-dataflow-java:****:legacy-****:shadowJar' with cache key 165b57b3de0480886088cb77a0fc5516
:runners:google-cloud-dataflow-java:****:legacy-****:shadowJar (Thread[Execution **** for ':' Thread 9,5,main]) completed. Took 0.204 secs.

> Task :runners:google-cloud-dataflow-java:compileTestJava FROM-CACHE
Custom actions are attached to task ':runners:google-cloud-dataflow-java:compileTestJava'.
Build cache key for task ':runners:google-cloud-dataflow-java:compileTestJava' is 861c7435ffd549666858392e9bbd7747
Task ':runners:google-cloud-dataflow-java:compileTestJava' is not up-to-date because:
  No history is available.
Loaded cache entry for task ':runners:google-cloud-dataflow-java:compileTestJava' with cache key 861c7435ffd549666858392e9bbd7747
:runners:google-cloud-dataflow-java:compileTestJava (Thread[Execution **** for ':',5,main]) completed. Took 0.305 secs.
:runners:google-cloud-dataflow-java:testClasses (Thread[Daemon ****,5,main]) started.

> Task :runners:google-cloud-dataflow-java:testClasses UP-TO-DATE
Skipping task ':runners:google-cloud-dataflow-java:testClasses' as it has no actions.
:runners:google-cloud-dataflow-java:testClasses (Thread[Daemon ****,5,main]) completed. Took 0.107 secs.
:runners:google-cloud-dataflow-java:testJar (Thread[Daemon ****,5,main]) started.

> Task :runners:google-cloud-dataflow-java:testJar
Caching disabled for task ':runners:google-cloud-dataflow-java:testJar' because:
  Caching has not been enabled for the task
Task ':runners:google-cloud-dataflow-java:testJar' is not up-to-date because:
  No history is available.
file or directory '<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/runners/google-cloud-dataflow-java/build/resources/test',> not found
:runners:google-cloud-dataflow-java:testJar (Thread[Daemon ****,5,main]) completed. Took 0.05 secs.
:sdks:java:io:file-based-io-tests:integrationTest (Thread[Daemon ****,5,main]) started.

> Task :sdks:java:io:file-based-io-tests:integrationTest
Custom actions are attached to task ':sdks:java:io:file-based-io-tests:integrationTest'.
Build cache key for task ':sdks:java:io:file-based-io-tests:integrationTest' is d113188b16b5dc9e6e52b7033c284315
Task ':sdks:java:io:file-based-io-tests:integrationTest' is not up-to-date because:
  Task.upToDateWhen is false.
Starting process 'Gradle Test Executor 1'. Working directory: <https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/file-based-io-tests> Command: /usr/lib/jvm/java-8-openjdk-amd64/bin/java -DbeamTestPipelineOptions=["--runner=DataflowRunner","--project=apache-beam-testing","--tempRoot=gs://temp-storage-for-perf-tests","--filenamePrefix=gs://temp-storage-for-perf-tests/beam_PerformanceTests_TFRecordIOIT/5309/","--bigQueryDataset=beam_performance","--bigQueryTable=tfrecordioit_results","--influxMeasurement=tfrecordioit_results","--numberOfRecords=18000000","--expectedHash=543104423f8b6eb097acb9f111c19fe4","--datasetSize=1019380000","--numWorkers=5","--autoscalingAlgorithm=NONE","--influxDatabase=beam_test_metrics","--influxHost=http://10.128.0.96:8086","--****HarnessContainerImage=","--dataflowWorkerJar=<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/runners/google-cloud-dataflow-java/****/legacy-****/build/libs/beam-runners-google-cloud-dataflow-java-legacy-****-2.34.0-SNAPSHOT.jar","--region=us-central1"]> -Djava.security.manager=****.org.gradle.process.internal.****.child.BootstrapSecurityManager -Dorg.gradle.native=false -Xmx2g -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant -ea -cp /home/jenkins/.gradle/caches/6.8.3/****Main/gradle-****.jar ****.org.gradle.process.internal.****.GradleWorkerMain 'Gradle Test Executor 1'
Successfully started process 'Gradle Test Executor 1'

Gradle Test Executor 1 started executing tests.

> Task :sdks:java:io:file-based-io-tests:integrationTest

org.apache.beam.sdk.io.tfrecord.TFRecordIOIT STANDARD_ERROR
    SLF4J: Class path contains multiple SLF4J bindings.
    SLF4J: Found binding in [jar:<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/runners/google-cloud-dataflow-java/****/legacy-****/build/libs/beam-runners-google-cloud-dataflow-java-legacy-****-2.34.0-SNAPSHOT.jar!/org/slf4j/impl/StaticLoggerBinder.class]>
    SLF4J: Found binding in [jar:file:/home/jenkins/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-log4j12/1.7.30/c21f55139d8141d2231214fb1feaf50a1edca95e/slf4j-log4j12-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class]
    SLF4J: Found binding in [jar:file:/home/jenkins/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-jdk14/1.7.30/d35953dd2fe54ebe39fdf18cfd82fe6eb35b25ed/slf4j-jdk14-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class]
    SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
    SLF4J: Actual binding is of type [org.slf4j.impl.JDK14LoggerFactory]

org.apache.beam.sdk.io.tfrecord.TFRecordIOIT > writeThenReadAll STANDARD_ERROR
    Sep 11, 2021 6:22:28 AM org.apache.beam.runners.dataflow.DataflowRunner validateSdkContainerImageOptions
    WARNING: Prefer --sdkContainerImage over deprecated legacy option --****HarnessContainerImage.
    Sep 11, 2021 6:22:30 AM org.apache.beam.sdk.extensions.gcp.options.GcpOptions$GcpTempLocationFactory tryCreateDefaultBucket
    INFO: No tempLocation specified, attempting to use default bucket: dataflow-staging-us-central1-844138762903
    Sep 11, 2021 6:22:30 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
    WARNING: Request failed with code 409, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/storage/v1/b?predefinedAcl=projectPrivate&predefinedDefaultObjectAcl=projectPrivate&project=apache-beam-testing. 
    Sep 11, 2021 6:22:30 AM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
    INFO: No stagingLocation provided, falling back to gcpTempLocation
    Sep 11, 2021 6:22:31 AM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
    INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 317 files. Enable logging at DEBUG level to see which files will be staged.
    Sep 11, 2021 6:22:31 AM org.apache.beam.runners.dataflow.DataflowRunner run
    INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services.
    Sep 11, 2021 6:22:35 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
    INFO: Uploading 318 files from PipelineOptions.filesToStage to staging location to prepare for execution.
    Sep 11, 2021 6:22:35 AM org.apache.beam.runners.dataflow.util.PackageUtil$PackageAttributes forFileToStage
    INFO: Staging custom dataflow-****.jar as beam-runners-google-cloud-dataflow-java-legacy-****-2.34.0-SNAPSHOT-YtvkOIsfQYfFzG89aVBO_8oNHoiij9SynwUJLdMkn5A.jar
    Sep 11, 2021 6:22:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
    INFO: Uploading /tmp/test8771101010197680558.zip to gs://dataflow-staging-us-central1-844138762903/temp/staging/test-7fiVfVx6yD8QDpxuJ_06ZpYgUPVPbONkMDR2NF5PBmQ.jar
    Sep 11, 2021 6:22:37 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
    INFO: Staging files complete: 317 files cached, 1 files newly uploaded in 2 seconds
    Sep 11, 2021 6:22:38 AM org.apache.beam.runners.dataflow.DataflowRunner run
    INFO: Staging portable pipeline proto to gs://dataflow-staging-us-central1-844138762903/temp/staging/
    Sep 11, 2021 6:22:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
    INFO: Uploading <205290 bytes, hash 0eec860f76e48c8ffb019d474fe8476644322ace20e8fa15f4a61bf748aa1240> to gs://dataflow-staging-us-central1-844138762903/temp/staging/pipeline-DuyGD3bkjI_7AZ1HT-hHZkQyKs4g6PoV9KYb90iqEkA.pb
    Sep 11, 2021 6:22:40 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Generate sequence/Read(BoundedCountingSource) as step s1
    Sep 11, 2021 6:22:41 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Produce text lines as step s2
    Sep 11, 2021 6:22:41 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Transform strings to bytes/Map as step s3
    Sep 11, 2021 6:22:41 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Record time before writing as step s4
    Sep 11, 2021 6:22:41 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/RewindowIntoGlobal/Window.Assign as step s5
    Sep 11, 2021 6:22:41 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles as step s6
    Sep 11, 2021 6:22:41 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten as step s7
    Sep 11, 2021 6:22:41 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/WriteUnshardedBundlesToTempFiles/WriteUnwritten as step s8
    Sep 11, 2021 6:22:41 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/WriteUnshardedBundlesToTempFiles/DropShardNum as step s9
    Sep 11, 2021 6:22:41 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/WriteUnshardedBundlesToTempFiles/Flatten.PCollections as step s10
    Sep 11, 2021 6:22:41 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow) as step s11
    Sep 11, 2021 6:22:41 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/GatherTempFileResults/View.AsList/CreateDataflowView as step s12
    Sep 11, 2021 6:22:41 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Create.Values/Read(CreateSource) as step s13
    Sep 11, 2021 6:22:41 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Reify.ReifyView/ParDo(Anonymous) as step s14
    Sep 11, 2021 6:22:41 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Values/Values/Map as step s15
    Sep 11, 2021 6:22:41 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Finalize as step s16
    Sep 11, 2021 6:22:41 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random key as step s17
    Sep 11, 2021 6:22:41 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign as step s18
    Sep 11, 2021 6:22:41 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey as step s19
    Sep 11, 2021 6:22:41 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable as step s20
    Sep 11, 2021 6:22:41 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
    INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Values/Values/Map as step s21
    Sep 11, 2021 6:22:41 AM org.apache.beam.runners.dataflow.DataflowRunner run
    INFO: Dataflow SDK version: 2.34.0-SNAPSHOT
    Sep 11, 2021 6:22:42 AM org.apache.beam.runners.dataflow.DataflowRunner run
    INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-09-10_23_22_41-2690919654692164700?project=apache-beam-testing
    Sep 11, 2021 6:22:42 AM org.apache.beam.runners.dataflow.DataflowRunner run
    INFO: Submitted job: 2021-09-10_23_22_41-2690919654692164700
    Sep 11, 2021 6:22:42 AM org.apache.beam.runners.dataflow.DataflowRunner run
    INFO: To cancel the job using the 'gcloud' tool, run:
    > gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-09-10_23_22_41-2690919654692164700
    Sep 11, 2021 6:23:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-11T06:23:01.453Z: Worker configuration: e2-standard-2 in us-central1-c.
    Sep 11, 2021 6:23:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-11T06:23:02.244Z: Expanding CoGroupByKey operations into optimizable parts.
    Sep 11, 2021 6:23:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-11T06:23:02.351Z: Expanding GroupByKey operations into optimizable parts.
    Sep 11, 2021 6:23:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-11T06:23:02.393Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
    Sep 11, 2021 6:23:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-11T06:23:02.524Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
    Sep 11, 2021 6:23:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-11T06:23:02.569Z: Unzipping flatten s10 for input s6.writtenRecords
    Sep 11, 2021 6:23:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-11T06:23:02.604Z: Fusing unzipped copy of Write content to files/Write/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow), through flatten Write content to files/Write/WriteUnshardedBundlesToTempFiles/Flatten.PCollections, into producer Write content to files/Write/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles
    Sep 11, 2021 6:23:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-11T06:23:02.631Z: Fusing consumer Write content to files/Write/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow) into Write content to files/Write/WriteUnshardedBundlesToTempFiles/DropShardNum
    Sep 11, 2021 6:23:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-11T06:23:02.674Z: Fusing consumer Produce text lines into Generate sequence/Read(BoundedCountingSource)
    Sep 11, 2021 6:23:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-11T06:23:02.711Z: Fusing consumer Transform strings to bytes/Map into Produce text lines
    Sep 11, 2021 6:23:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-11T06:23:02.757Z: Fusing consumer Record time before writing into Transform strings to bytes/Map
    Sep 11, 2021 6:23:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-11T06:23:02.795Z: Fusing consumer Write content to files/Write/RewindowIntoGlobal/Window.Assign into Record time before writing
    Sep 11, 2021 6:23:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-11T06:23:02.818Z: Fusing consumer Write content to files/Write/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles into Write content to files/Write/RewindowIntoGlobal/Window.Assign
    Sep 11, 2021 6:23:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-11T06:23:02.845Z: Fusing consumer Write content to files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify into Write content to files/Write/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles
    Sep 11, 2021 6:23:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-11T06:23:02.876Z: Fusing consumer Write content to files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Write into Write content to files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify
    Sep 11, 2021 6:23:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-11T06:23:02.914Z: Fusing consumer Write content to files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/GroupByWindow into Write content to files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Read
    Sep 11, 2021 6:23:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-11T06:23:02.947Z: Fusing consumer Write content to files/Write/WriteUnshardedBundlesToTempFiles/WriteUnwritten into Write content to files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/GroupByWindow
    Sep 11, 2021 6:23:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-11T06:23:02.983Z: Fusing consumer Write content to files/Write/WriteUnshardedBundlesToTempFiles/DropShardNum into Write content to files/Write/WriteUnshardedBundlesToTempFiles/WriteUnwritten
    Sep 11, 2021 6:23:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-11T06:23:03.014Z: Fusing consumer Write content to files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Reify.ReifyView/ParDo(Anonymous) into Write content to files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Create.Values/Read(CreateSource)
    Sep 11, 2021 6:23:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-11T06:23:03.052Z: Fusing consumer Write content to files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Values/Values/Map into Write content to files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Reify.ReifyView/ParDo(Anonymous)
    Sep 11, 2021 6:23:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-11T06:23:03.094Z: Fusing consumer Write content to files/Write/FinalizeTempFileBundles/Finalize into Write content to files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Values/Values/Map
    Sep 11, 2021 6:23:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-11T06:23:03.115Z: Fusing consumer Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random key into Write content to files/Write/FinalizeTempFileBundles/Finalize
    Sep 11, 2021 6:23:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-11T06:23:03.143Z: Fusing consumer Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign into Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random key
    Sep 11, 2021 6:23:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-11T06:23:03.186Z: Fusing consumer Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify into Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign
    Sep 11, 2021 6:23:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-11T06:23:03.215Z: Fusing consumer Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write into Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify
    Sep 11, 2021 6:23:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-11T06:23:03.247Z: Fusing consumer Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow into Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read
    Sep 11, 2021 6:23:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-11T06:23:03.279Z: Fusing consumer Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable into Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow
    Sep 11, 2021 6:23:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-11T06:23:03.304Z: Fusing consumer Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Values/Values/Map into Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable
    Sep 11, 2021 6:23:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-11T06:23:03.702Z: Executing operation Write content to files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Create
    Sep 11, 2021 6:23:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-11T06:23:03.805Z: Starting 5 ****s in us-central1-c...
    Sep 11, 2021 6:23:05 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-11T06:23:04.209Z: Finished operation Write content to files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Create
    Sep 11, 2021 6:23:05 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-11T06:23:04.348Z: Executing operation Generate sequence/Read(BoundedCountingSource)+Produce text lines+Transform strings to bytes/Map+Record time before writing+Write content to files/Write/RewindowIntoGlobal/Window.Assign+Write content to files/Write/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles+Write content to files/Write/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow)+Write content to files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify+Write content to files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Write
    Sep 11, 2021 6:23:05 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-11T06:23:04.354Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
    Sep 11, 2021 6:23:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    SEVERE: 2021-09-11T06:23:39.395Z: Startup of the **** pool in zone us-central1-c failed to bring up any of the desired 5 ****s. QUOTA_EXCEEDED: Quota 'CPUS' exceeded.  Limit: 2500.0 in region us-central1.
    Sep 11, 2021 6:23:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    SEVERE: 2021-09-11T06:23:39.423Z: Workflow failed.
    Sep 11, 2021 6:23:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    WARNING: 2021-09-11T06:23:39.677Z: S02:Generate sequence/Read(BoundedCountingSource)+Produce text lines+Transform strings to bytes/Map+Record time before writing+Write content to files/Write/RewindowIntoGlobal/Window.Assign+Write content to files/Write/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles+Write content to files/Write/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow)+Write content to files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify+Write content to files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Write failed.
    Sep 11, 2021 6:23:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-11T06:23:39.710Z: Finished operation Generate sequence/Read(BoundedCountingSource)+Produce text lines+Transform strings to bytes/Map+Record time before writing+Write content to files/Write/RewindowIntoGlobal/Window.Assign+Write content to files/Write/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles+Write content to files/Write/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow)+Write content to files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify+Write content to files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Write
    Sep 11, 2021 6:23:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-11T06:23:39.787Z: Cleaning up.
    Sep 11, 2021 6:23:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-11T06:23:39.877Z: Stopping **** pool...
    Sep 11, 2021 6:24:06 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
    INFO: 2021-09-11T06:24:04.454Z: Worker pool stopped.
    Sep 11, 2021 6:24:43 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
    INFO: Job 2021-09-10_23_22_41-2690919654692164700 failed with status FAILED.

Gradle Test Executor 1 finished executing tests.

> Task :sdks:java:io:file-based-io-tests:integrationTest FAILED

org.apache.beam.sdk.io.tfrecord.TFRecordIOIT > writeThenReadAll FAILED
    java.lang.IllegalStateException: Unable to find any files matching gs://temp-storage-for-perf-tests/beam_PerformanceTests_TFRecordIOIT/5309/_1631341347170*
        at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:588)
        at org.apache.beam.sdk.io.TFRecordIO$Read.expand(TFRecordIO.java:191)
        at org.apache.beam.sdk.io.TFRecordIO$Read.expand(TFRecordIO.java:112)
        at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:548)
        at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:482)
        at org.apache.beam.sdk.values.PBegin.apply(PBegin.java:44)
        at org.apache.beam.sdk.Pipeline.apply(Pipeline.java:177)
        at org.apache.beam.sdk.io.tfrecord.TFRecordIOIT.writeThenReadAll(TFRecordIOIT.java:151)

1 test completed, 1 failed
Finished generating test XML results (0.09 secs) into: <https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/file-based-io-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.091 secs) into: <https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/file-based-io-tests/build/reports/tests/integrationTest>
:sdks:java:io:file-based-io-tests:integrationTest (Thread[Daemon ****,5,main]) completed. Took 2 mins 21.819 secs.

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:io:file-based-io-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/file-based-io-tests/build/reports/tests/integrationTest/index.html>

* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3m 12s
129 actionable tasks: 79 executed, 50 from cache

Publishing build scan...
https://gradle.com/s/fh5dr5hlyeuvm

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org