You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@beam.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2018/11/20 05:56:13 UTC
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Dataflow
#1088
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/1088/display/redirect?page=changes>
Changes:
[cushon] [BEAM-6096] illegal signature attribute when compiling with JDK 11
------------------------------------------
[...truncated 269.74 KB...]
"message" : "The total number of changes to the object temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-JXk4_AkNyZ8UWLy68oYasg.jar exceeds the rate limit. Please reduce the rate of create, update, and delete requests.",
"reason" : "rateLimitExceeded"
} ],
"message" : "The total number of changes to the object temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-JXk4_AkNyZ8UWLy68oYasg.jar exceeds the rate limit. Please reduce the rate of create, update, and delete requests."
}
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:432)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:287)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1626)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 410 Gone
{
"code" : 429,
"errors" : [ {
"domain" : "usageLimits",
"message" : "The total number of changes to the object temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-JXk4_AkNyZ8UWLy68oYasg.jar exceeds the rate limit. Please reduce the rate of create, update, and delete requests.",
"reason" : "rateLimitExceeded"
} ],
"message" : "The total number of changes to the object temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-JXk4_AkNyZ8UWLy68oYasg.jar exceeds the rate limit. Please reduce the rate of create, update, and delete requests."
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:512)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:432)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:549)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:358)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Nov 20, 2018 5:51:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-JXk4_AkNyZ8UWLy68oYasg.jar
Nov 20, 2018 5:51:49 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-JXk4_AkNyZ8UWLy68oYasg.jar
Nov 20, 2018 5:51:53 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 117 files cached, 28 files newly uploaded
Nov 20, 2018 5:51:53 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query3.ReadBounded as step s1
Nov 20, 2018 5:51:53 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query3/Query3.Monitor as step s2
Nov 20, 2018 5:51:53 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query3/Query3.Snoop as step s3
Nov 20, 2018 5:51:53 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query3/Query3/Window.Into()/Window.Assign as step s4
Nov 20, 2018 5:51:53 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query3/Query3/justNewAuctions/IsNewAuction/ParDo(Anonymous) as step s5
Nov 20, 2018 5:51:53 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query3/Query3/justNewAuctions/AsAuction as step s6
Nov 20, 2018 5:51:53 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query3/Query3/Query3.InCategory/ParDo(Anonymous) as step s7
Nov 20, 2018 5:51:53 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query3/Query3/AuctionBySeller as step s8
Nov 20, 2018 5:51:53 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query3/Query3/justNewPersons/IsNewPerson/ParDo(Anonymous) as step s9
Nov 20, 2018 5:51:53 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query3/Query3/justNewPersons/AsPerson as step s10
Nov 20, 2018 5:51:53 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query3/Query3/Query3.InState/ParDo(Anonymous) as step s11
Nov 20, 2018 5:51:53 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query3/Query3/PersonById as step s12
Nov 20, 2018 5:51:53 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query3/Query3/CoGroupByKey/MakeUnionTable0 as step s13
Nov 20, 2018 5:51:53 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query3/Query3/CoGroupByKey/MakeUnionTable1 as step s14
Nov 20, 2018 5:51:53 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query3/Query3/CoGroupByKey/Flatten as step s15
Nov 20, 2018 5:51:53 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query3/Query3/CoGroupByKey/GBK as step s16
Nov 20, 2018 5:51:53 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query3/Query3/CoGroupByKey/ConstructCoGbkResultFn as step s17
Nov 20, 2018 5:51:53 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query3/Query3/Query3.Join/BatchStatefulParDoOverrides.GbkBeforeStatefulParDo/ReifyWindows as step s18
Nov 20, 2018 5:51:53 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query3/Query3/Query3.Join/BatchStatefulParDoOverrides.GbkBeforeStatefulParDo/PartitionKeys as step s19
Nov 20, 2018 5:51:53 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query3/Query3/Query3.Join/ParDo(BatchStateful) as step s20
Nov 20, 2018 5:51:53 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query3/Query3/Query3.Project as step s21
Nov 20, 2018 5:51:53 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query3/Query3.Debug as step s22
Nov 20, 2018 5:51:53 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query3/Query3.Stamp as step s23
Nov 20, 2018 5:51:53 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query3.Format as step s24
Nov 20, 2018 5:51:53 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query3.DevNull as step s25
Nov 20, 2018 5:51:53 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/nexmark/staging/
Nov 20, 2018 5:51:53 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <65289 bytes, hash E5tK16nmdPiAqXp-J5KmiA> to gs://temp-storage-for-perf-tests/nexmark/staging/pipeline-E5tK16nmdPiAqXp-J5KmiA.pb
Dataflow SDK version: 2.9.0-SNAPSHOT
Nov 20, 2018 5:51:55 AM org.apache.beam.runners.dataflow.DataflowRunner run
Submitted job: 2018-11-19_21_51_54-3518111521880910996
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-11-19_21_51_54-3518111521880910996?project=apache-beam-testing
Nov 20, 2018 5:51:55 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2018-11-19_21_51_54-3518111521880910996
Nov 20, 2018 5:52:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2018-11-20T05:51:54.466Z: The requested max number of workers (4) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Nov 20, 2018 5:52:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:51:57.004Z: Checking permissions granted to controller Service Account.
Nov 20, 2018 5:52:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:52:03.349Z: Worker configuration: n1-standard-1 in us-central1-b.
Nov 20, 2018 5:52:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:52:03.997Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 20, 2018 5:52:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:52:04.096Z: Expanding GroupByKey operations into optimizable parts.
Nov 20, 2018 5:52:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:52:04.118Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 20, 2018 5:52:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:52:04.241Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 20, 2018 5:52:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:52:04.281Z: Unzipping flatten s15 for input s13.org.apache.beam.sdk.values.PCollection.<init>:402#9a88db53ad38536b
Nov 20, 2018 5:52:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:52:04.316Z: Fusing unzipped copy of Query3/Query3/CoGroupByKey/GBK/Reify, through flatten Query3/Query3/CoGroupByKey/Flatten, into producer Query3/Query3/CoGroupByKey/MakeUnionTable0
Nov 20, 2018 5:52:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:52:04.351Z: Fusing consumer Query3/Query3.Stamp into Query3/Query3.Debug
Nov 20, 2018 5:52:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:52:04.388Z: Fusing consumer Query3.Format into Query3/Query3.Stamp
Nov 20, 2018 5:52:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:52:04.423Z: Unzipping flatten s15-u18 for input s16-reify-value0-c16
Nov 20, 2018 5:52:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:52:04.451Z: Fusing unzipped copy of Query3/Query3/CoGroupByKey/GBK/Write, through flatten s15-u18, into producer Query3/Query3/CoGroupByKey/GBK/Reify
Nov 20, 2018 5:52:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:52:04.484Z: Fusing consumer Query3.DevNull into Query3.Format
Nov 20, 2018 5:52:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:52:04.511Z: Fusing consumer Query3/Query3/Query3.Join/BatchStatefulParDoOverrides.GbkBeforeStatefulParDo/PartitionKeys/Write into Query3/Query3/Query3.Join/BatchStatefulParDoOverrides.GbkBeforeStatefulParDo/ReifyWindows
Nov 20, 2018 5:52:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:52:04.548Z: Fusing consumer Query3/Query3/Query3.Project into Query3/Query3/Query3.Join/ParDo(BatchStateful)
Nov 20, 2018 5:52:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:52:04.586Z: Fusing consumer Query3/Query3.Debug into Query3/Query3/Query3.Project
Nov 20, 2018 5:52:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:52:04.618Z: Fusing consumer Query3/Query3/Query3.Join/ParDo(BatchStateful) into Query3/Query3/Query3.Join/BatchStatefulParDoOverrides.GbkBeforeStatefulParDo/PartitionKeys/Read
Nov 20, 2018 5:52:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:52:04.656Z: Fusing consumer Query3/Query3/CoGroupByKey/GBK/GroupByWindow into Query3/Query3/CoGroupByKey/GBK/Read
Nov 20, 2018 5:52:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:52:04.693Z: Fusing consumer Query3/Query3/CoGroupByKey/ConstructCoGbkResultFn into Query3/Query3/CoGroupByKey/GBK/GroupByWindow
Nov 20, 2018 5:52:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:52:04.730Z: Fusing consumer Query3/Query3/Query3.Join/BatchStatefulParDoOverrides.GbkBeforeStatefulParDo/ReifyWindows into Query3/Query3/CoGroupByKey/ConstructCoGbkResultFn
Nov 20, 2018 5:52:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:52:04.769Z: Fusing consumer Query3/Query3/CoGroupByKey/GBK/Reify into Query3/Query3/CoGroupByKey/MakeUnionTable1
Nov 20, 2018 5:52:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:52:04.822Z: Fusing consumer Query3/Query3/CoGroupByKey/GBK/Write into Query3/Query3/CoGroupByKey/GBK/Reify
Nov 20, 2018 5:52:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:52:04.850Z: Fusing consumer Query3/Query3/justNewAuctions/IsNewAuction/ParDo(Anonymous) into Query3/Query3/Window.Into()/Window.Assign
Nov 20, 2018 5:52:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:52:04.892Z: Fusing consumer Query3/Query3/justNewPersons/IsNewPerson/ParDo(Anonymous) into Query3/Query3/Window.Into()/Window.Assign
Nov 20, 2018 5:52:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:52:04.926Z: Fusing consumer Query3/Query3/Query3.InCategory/ParDo(Anonymous) into Query3/Query3/justNewAuctions/AsAuction
Nov 20, 2018 5:52:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:52:04.961Z: Fusing consumer Query3/Query3.Snoop into Query3/Query3.Monitor
Nov 20, 2018 5:52:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:52:04.996Z: Fusing consumer Query3/Query3/PersonById into Query3/Query3/Query3.InState/ParDo(Anonymous)
Nov 20, 2018 5:52:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:52:05.032Z: Fusing consumer Query3/Query3/CoGroupByKey/MakeUnionTable1 into Query3/Query3/PersonById
Nov 20, 2018 5:52:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:52:05.072Z: Fusing consumer Query3/Query3/justNewAuctions/AsAuction into Query3/Query3/justNewAuctions/IsNewAuction/ParDo(Anonymous)
Nov 20, 2018 5:52:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:52:05.098Z: Fusing consumer Query3/Query3/Query3.InState/ParDo(Anonymous) into Query3/Query3/justNewPersons/AsPerson
Nov 20, 2018 5:52:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:52:05.134Z: Fusing consumer Query3/Query3/justNewPersons/AsPerson into Query3/Query3/justNewPersons/IsNewPerson/ParDo(Anonymous)
Nov 20, 2018 5:52:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:52:05.171Z: Fusing consumer Query3/Query3/AuctionBySeller into Query3/Query3/Query3.InCategory/ParDo(Anonymous)
Nov 20, 2018 5:52:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:52:05.208Z: Fusing consumer Query3/Query3.Monitor into Query3.ReadBounded
Nov 20, 2018 5:52:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:52:05.243Z: Fusing consumer Query3/Query3/Window.Into()/Window.Assign into Query3/Query3.Snoop
Nov 20, 2018 5:52:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:52:05.281Z: Fusing consumer Query3/Query3/CoGroupByKey/MakeUnionTable0 into Query3/Query3/AuctionBySeller
Nov 20, 2018 5:52:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:52:05.721Z: Executing operation Query3/Query3/CoGroupByKey/GBK/Create
Nov 20, 2018 5:52:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:52:05.766Z: Executing operation Query3/Query3/Query3.Join/BatchStatefulParDoOverrides.GbkBeforeStatefulParDo/PartitionKeys/Create
Nov 20, 2018 5:52:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:52:05.818Z: Starting 4 workers in us-central1-b...
Nov 20, 2018 5:52:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:52:06.004Z: Executing operation Query3.ReadBounded+Query3/Query3.Monitor+Query3/Query3.Snoop+Query3/Query3/Window.Into()/Window.Assign+Query3/Query3/justNewAuctions/IsNewAuction/ParDo(Anonymous)+Query3/Query3/justNewPersons/IsNewPerson/ParDo(Anonymous)+Query3/Query3/justNewAuctions/AsAuction+Query3/Query3/Query3.InCategory/ParDo(Anonymous)+Query3/Query3/justNewPersons/AsPerson+Query3/Query3/Query3.InState/ParDo(Anonymous)+Query3/Query3/PersonById+Query3/Query3/CoGroupByKey/MakeUnionTable1+Query3/Query3/CoGroupByKey/GBK/Reify+Query3/Query3/AuctionBySeller+Query3/Query3/CoGroupByKey/MakeUnionTable0+Query3/Query3/CoGroupByKey/GBK/Reify+Query3/Query3/CoGroupByKey/GBK/Session/Flatten+Query3/Query3/CoGroupByKey/GBK/Write
Nov 20, 2018 5:52:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:52:22.985Z: Autoscaling: Raised the number of workers to 0 based on the rate of progress in the currently running step(s).
Nov 20, 2018 5:52:48 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:52:46.717Z: Autoscaling: Raised the number of workers to 1 based on the rate of progress in the currently running step(s).
Nov 20, 2018 5:52:48 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:52:46.764Z: Resized worker pool to 1, though goal was 4. This could be a quota issue.
Nov 20, 2018 5:52:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:52:49.530Z: Workers have started successfully.
Nov 20, 2018 5:52:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:52:49.568Z: Workers have started successfully.
Nov 20, 2018 5:52:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:52:58.722Z: Autoscaling: Raised the number of workers to 3 based on the rate of progress in the currently running step(s).
Nov 20, 2018 5:52:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:52:58.762Z: Resized worker pool to 3, though goal was 4. This could be a quota issue.
Nov 20, 2018 5:53:06 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:53:05.127Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running step(s).
Nov 20, 2018 5:53:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:53:44.084Z: Executing operation Query3/Query3/CoGroupByKey/GBK/Close
Nov 20, 2018 5:53:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:53:44.183Z: Executing operation Query3/Query3/CoGroupByKey/GBK/Read+Query3/Query3/CoGroupByKey/GBK/GroupByWindow+Query3/Query3/CoGroupByKey/ConstructCoGbkResultFn+Query3/Query3/Query3.Join/BatchStatefulParDoOverrides.GbkBeforeStatefulParDo/ReifyWindows+Query3/Query3/Query3.Join/BatchStatefulParDoOverrides.GbkBeforeStatefulParDo/PartitionKeys/Write
Nov 20, 2018 5:53:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:53:50.103Z: Executing operation Query3/Query3/Query3.Join/BatchStatefulParDoOverrides.GbkBeforeStatefulParDo/PartitionKeys/Close
Nov 20, 2018 5:53:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:53:50.200Z: Executing operation Query3/Query3/Query3.Join/BatchStatefulParDoOverrides.GbkBeforeStatefulParDo/PartitionKeys/Read+Query3/Query3/Query3.Join/ParDo(BatchStateful)+Query3/Query3/Query3.Project+Query3/Query3.Debug+Query3/Query3.Stamp+Query3.Format+Query3.DevNull
Nov 20, 2018 5:53:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:53:53.892Z: Cleaning up.
Nov 20, 2018 5:53:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:53:53.997Z: Stopping worker pool...
Nov 20, 2018 5:55:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:55:53.597Z: Autoscaling: Resized worker pool from 4 to 0.
Nov 20, 2018 5:55:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:55:53.647Z: Worker pool stopped.
Nov 20, 2018 5:55:55 AM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish
2018-11-20T05:55:55.387Z Waiting for main pipeline to 'finish'
WARNING: No terminal state was returned. State value RUNNING
2018-11-20T05:55:55.415Z RUNNING Query3
2018-11-20T05:55:56.633Z new perf {"snapshots":null,"runtimeSec":49.931,"numEvents":10000000,"eventsPerSec":200276.38140634075,"eventBytesPerSec":2.570677328713625E7,"numResults":59507,"resultsPerSec":1191.784662834712,"resultBytesPerSec":38085.918567623325,"startupDelaySec":-171.659,"processingDelaySec":47.128,"shutdownDelaySec":121.728,"timeDilation":-1.0,"errors":null,"jobId":null}
2018-11-20T05:56:11.690Z DONE Query3
> Task :beam-sdks-java-nexmark:run FAILED
:beam-sdks-java-nexmark:run (Thread[Task worker for ':' Thread 3,5,main]) completed. Took 4 mins 46.303 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/4.10.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 8m 18s
76 actionable tasks: 71 executed, 5 from cache
Publishing build scan...
https://gradle.com/s/ko5qqku753iqa
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Jenkins build is back to normal :
beam_PostCommit_Java_Nexmark_Dataflow #1092
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/1092/display/redirect?page=changes>
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Dataflow
#1091
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/1091/display/redirect?page=changes>
Changes:
[echauchot] [BEAM-5969] Fix typo in nexmark web page
[mxm] [website] Add 'builds' mailing list to the contact page
------------------------------------------
[...truncated 381.91 KB...]
Nov 20, 2018 10:04:04 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForKeys as step s11
Nov 20, 2018 10:04:04 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/ParDo(ToIsmMetadataRecordForKey) as step s12
Nov 20, 2018 10:04:04 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/Flatten.PCollections as step s13
Nov 20, 2018 10:04:04 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/CreateDataflowView as step s14
Nov 20, 2018 10:04:04 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/justBids/IsBid/ParDo(Anonymous) as step s15
Nov 20, 2018 10:04:04 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/BidToPrice as step s7
Nov 20, 2018 10:04:04 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map as step s8
Nov 20, 2018 10:04:04 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/justBids/AsBid as step s16
Nov 20, 2018 10:04:04 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce as step s9
Nov 20, 2018 10:04:04 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey as step s10
Nov 20, 2018 10:04:04 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues as step s11
Nov 20, 2018 10:04:04 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/JoinToFiles.JoinToFiles as step s17
Nov 20, 2018 10:04:04 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map as step s12
Nov 20, 2018 10:04:04 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map as step s13
Nov 20, 2018 10:04:04 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles.Debug as step s18
Nov 20, 2018 10:04:04 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map as step s14
Nov 20, 2018 10:04:04 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections as step s15
Nov 20, 2018 10:04:04 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey as step s16
Nov 20, 2018 10:04:04 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues as step s17
Nov 20, 2018 10:04:04 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles.Stamp as step s19
Nov 20, 2018 10:04:04 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map as step s18
Nov 20, 2018 10:04:04 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) as step s19
Nov 20, 2018 10:04:04 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly as step s20
Nov 20, 2018 10:04:04 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) as step s21
Nov 20, 2018 10:04:04 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView as step s22
Nov 20, 2018 10:04:04 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles.Format as step s20
Nov 20, 2018 10:04:04 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Query7.Select as step s23
Nov 20, 2018 10:04:04 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles.DevNull as step s21
Nov 20, 2018 10:04:04 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/nexmark/staging/
Nov 20, 2018 10:04:04 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Debug as step s24
Nov 20, 2018 10:04:04 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Stamp as step s25
Nov 20, 2018 10:04:04 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.Format as step s26
Nov 20, 2018 10:04:04 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.DevNull as step s27
Nov 20, 2018 10:04:04 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/nexmark/staging/
Nov 20, 2018 10:04:04 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <58906 bytes, hash 4k0HKoRdW0mkWLHwM962Ng> to gs://temp-storage-for-perf-tests/nexmark/staging/pipeline-4k0HKoRdW0mkWLHwM962Ng.pb
Nov 20, 2018 10:04:04 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <88194 bytes, hash A5XVO8-mvoxNTyTEHhSaxw> to gs://temp-storage-for-perf-tests/nexmark/staging/pipeline-A5XVO8-mvoxNTyTEHhSaxw.pb
Dataflow SDK version: 2.9.0-SNAPSHOT
Dataflow SDK version: 2.9.0-SNAPSHOT
Submitted job: 2018-11-20_02_04_05-9943991727944894115
Nov 20, 2018 10:04:06 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-11-20_02_04_05-9943991727944894115?project=apache-beam-testing
Nov 20, 2018 10:04:06 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2018-11-20_02_04_05-9943991727944894115
Nov 20, 2018 10:04:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2018-11-20T10:04:05.084Z: The requested max number of workers (4) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Nov 20, 2018 10:04:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:04:08.522Z: Checking permissions granted to controller Service Account.
Nov 20, 2018 10:04:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:04:13.439Z: Worker configuration: n1-standard-1 in us-central1-b.
Nov 20, 2018 10:04:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:04:14.154Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 20, 2018 10:04:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:04:14.238Z: Expanding GroupByKey operations into optimizable parts.
Nov 20, 2018 10:04:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:04:14.288Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 20, 2018 10:04:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:04:14.472Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 20, 2018 10:04:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:04:14.515Z: Unzipping flatten s15 for input s13.org.apache.beam.sdk.values.PCollection.<init>:402#25398a999f74762b
Nov 20, 2018 10:04:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:04:14.557Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial, through flatten Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map
Nov 20, 2018 10:04:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:04:14.608Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract
Nov 20, 2018 10:04:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:04:14.656Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map
Nov 20, 2018 10:04:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:04:14.695Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)
Nov 20, 2018 10:04:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:04:14.730Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read
Nov 20, 2018 10:04:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:04:14.768Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify
Nov 20, 2018 10:04:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:04:14.814Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read
Nov 20, 2018 10:04:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:04:14.861Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues
Nov 20, 2018 10:04:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:04:14.896Z: Unzipping flatten s15-u45 for input partial-s16-out1-c43
Nov 20, 2018 10:04:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:04:14.937Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify, through flatten s15-u45, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 20, 2018 10:04:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:04:14.982Z: Fusing consumer Query7/Query7/BidToPrice into Query7/Query7/Window.Into()/Window.Assign
Nov 20, 2018 10:04:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:04:15.030Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 20, 2018 10:04:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:04:15.083Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map
Nov 20, 2018 10:04:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:04:15.126Z: Fusing consumer Query7.Format into Query7/Query7.Stamp
Nov 20, 2018 10:04:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:04:15.165Z: Fusing consumer Query7/Query7.Stamp into Query7/Query7.Debug
Nov 20, 2018 10:04:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:04:15.201Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 20, 2018 10:04:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:04:15.239Z: Fusing consumer Query7/Query7/justBids/IsBid/ParDo(Anonymous) into Query7/Query7.Snoop
Nov 20, 2018 10:04:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:04:15.288Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read
Nov 20, 2018 10:04:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:04:15.339Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map into Query7/Query7/BidToPrice
Nov 20, 2018 10:04:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:04:15.383Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues
Nov 20, 2018 10:04:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:04:15.417Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial
Nov 20, 2018 10:04:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:04:15.468Z: Fusing consumer Query7/Query7/justBids/AsBid into Query7/Query7/justBids/IsBid/ParDo(Anonymous)
Nov 20, 2018 10:04:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:04:15.511Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map
Nov 20, 2018 10:04:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:04:15.543Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map
Nov 20, 2018 10:04:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:04:15.585Z: Fusing consumer Query7/Query7.Monitor into Query7.ReadBounded
Nov 20, 2018 10:04:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:04:15.632Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract
Nov 20, 2018 10:04:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:04:15.666Z: Fusing consumer Query7/Query7/Window.Into()/Window.Assign into Query7/Query7/justBids/AsBid
Nov 20, 2018 10:04:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:04:15.715Z: Fusing consumer Query7/Query7.Debug into Query7/Query7/Query7.Select
Nov 20, 2018 10:04:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:04:15.758Z: Fusing consumer Query7/Query7.Snoop into Query7/Query7.Monitor
Nov 20, 2018 10:04:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:04:15.800Z: Fusing consumer Query7.DevNull into Query7.Format
Nov 20, 2018 10:04:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:04:15.849Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 20, 2018 10:04:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:04:15.887Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify
Nov 20, 2018 10:04:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:04:16.407Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Create
Nov 20, 2018 10:04:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:04:16.446Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Create
Nov 20, 2018 10:04:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:04:16.484Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
Nov 20, 2018 10:04:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:04:16.517Z: Starting 4 workers in us-central1-b...
Nov 20, 2018 10:04:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:04:16.779Z: Executing operation Query7.ReadBounded+Query7/Query7.Monitor+Query7/Query7.Snoop+Query7/Query7/justBids/IsBid/ParDo(Anonymous)+Query7/Query7/justBids/AsBid+Query7/Query7/Window.Into()/Window.Assign+Query7/Query7/BidToPrice+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write
Nov 20, 2018 10:04:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:04:35.498Z: Autoscaling: Raised the number of workers to 0 based on the rate of progress in the currently running step(s).
Nov 20, 2018 10:05:02 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:05:02.325Z: Workers have started successfully.
Nov 20, 2018 10:05:04 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:05:04.411Z: Autoscaling: Raised the number of workers to 3 based on the rate of progress in the currently running step(s).
Nov 20, 2018 10:05:04 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:05:04.454Z: Resized worker pool to 3, though goal was 4. This could be a quota issue.
Nov 20, 2018 10:05:06 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:05:05.226Z: Workers have started successfully.
Nov 20, 2018 10:05:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:05:21.668Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running step(s).
Nov 20, 2018 10:06:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:06:17.925Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Close
Nov 20, 2018 10:06:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:06:18.021Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write
Nov 20, 2018 10:06:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:06:19.681Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Close
Nov 20, 2018 10:06:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:06:19.777Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
Nov 20, 2018 10:06:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:06:21.212Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
Nov 20, 2018 10:06:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:06:21.308Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow)
Nov 20, 2018 10:06:24 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:06:23.718Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView
Nov 20, 2018 10:06:24 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:06:23.951Z: Executing operation Query7/Query7/Query7.Select+Query7/Query7.Debug+Query7/Query7.Stamp+Query7.Format+Query7.DevNull
Nov 20, 2018 10:06:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:06:51.715Z: Cleaning up.
Nov 20, 2018 10:06:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:06:51.853Z: Stopping worker pool...
Nov 20, 2018 10:08:06 AM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish
2018-11-20T10:08:06.589Z Waiting for main pipeline to 'finish'
WARNING: No terminal state was returned. State value RUNNING
2018-11-20T10:08:06.624Z RUNNING Query7
2018-11-20T10:08:07.781Z new perf {"snapshots":null,"runtimeSec":95.68,"numEvents":10000000,"eventsPerSec":104515.05016722408,"eventBytesPerSec":1.341518496028428E7,"numResults":100,"resultsPerSec":1.0451505016722407,"resultBytesPerSec":105.59155518394648,"startupDelaySec":-171.076,"processingDelaySec":70.044,"shutdownDelaySec":75.396,"timeDilation":10.278010033444815,"errors":null,"jobId":null}
2018-11-20T10:08:22.826Z RUNNING Query7
2018-11-20T10:08:24.042Z no activity
2018-11-20T10:08:39.080Z RUNNING Query7
2018-11-20T10:08:40.100Z no activity
2018-11-20T10:08:55.147Z RUNNING Query7
2018-11-20T10:08:56.208Z no activity
2018-11-20T10:09:11.264Z RUNNING Query7
2018-11-20T10:09:12.311Z no activity
2018-11-20T10:09:27.351Z RUNNING Query7
2018-11-20T10:09:28.445Z no activity
2018-11-20T10:09:43.486Z RUNNING Query7
2018-11-20T10:09:44.495Z no activity
2018-11-20T10:09:59.544Z RUNNING Query7
2018-11-20T10:09:59.705Z no activity
2018-11-20T10:10:14.706Z DONE Query7
> Task :beam-sdks-java-nexmark:run FAILED
:beam-sdks-java-nexmark:run (Thread[Task worker for ':' Thread 10,5,main]) completed. Took 6 mins 39.386 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/4.10.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 10m 55s
76 actionable tasks: 72 executed, 4 from cache
Publishing build scan...
https://gradle.com/s/6l2dekckqdki6
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Dataflow
#1090
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/1090/display/redirect?page=changes>
Changes:
[relax] CoGroup transforms for schemas.
------------------------------------------
[...truncated 233.61 KB...]
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-SJ0iKljRwJqRaVO78HFu2A.jar
Nov 20, 2018 8:50:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-SJ0iKljRwJqRaVO78HFu2A.jar
Nov 20, 2018 8:50:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-SJ0iKljRwJqRaVO78HFu2A.jar
Nov 20, 2018 8:50:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-SJ0iKljRwJqRaVO78HFu2A.jar
Nov 20, 2018 8:50:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-SJ0iKljRwJqRaVO78HFu2A.jar
Nov 20, 2018 8:50:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-SJ0iKljRwJqRaVO78HFu2A.jar
Nov 20, 2018 8:50:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-SJ0iKljRwJqRaVO78HFu2A.jar
Nov 20, 2018 8:50:24 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 117 files cached, 29 files newly uploaded
Nov 20, 2018 8:50:24 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.ReadBounded as step s1
Nov 20, 2018 8:50:24 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Monitor as step s2
Nov 20, 2018 8:50:24 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Snoop as step s3
Nov 20, 2018 8:50:24 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/IsBid/ParDo(Anonymous) as step s4
Nov 20, 2018 8:50:24 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/AsBid as step s5
Nov 20, 2018 8:50:24 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Window.Into()/Window.Assign as step s6
Nov 20, 2018 8:50:24 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/BidToPrice as step s7
Nov 20, 2018 8:50:24 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map as step s8
Nov 20, 2018 8:50:24 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce as step s9
Nov 20, 2018 8:50:24 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey as step s10
Nov 20, 2018 8:50:24 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues as step s11
Nov 20, 2018 8:50:24 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map as step s12
Nov 20, 2018 8:50:24 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map as step s13
Nov 20, 2018 8:50:24 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map as step s14
Nov 20, 2018 8:50:24 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections as step s15
Nov 20, 2018 8:50:24 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey as step s16
Nov 20, 2018 8:50:24 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues as step s17
Nov 20, 2018 8:50:24 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map as step s18
Nov 20, 2018 8:50:24 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) as step s19
Nov 20, 2018 8:50:24 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly as step s20
Nov 20, 2018 8:50:24 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) as step s21
Nov 20, 2018 8:50:24 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView as step s22
Nov 20, 2018 8:50:24 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Query7.Select as step s23
Nov 20, 2018 8:50:24 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Debug as step s24
Nov 20, 2018 8:50:24 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Stamp as step s25
Nov 20, 2018 8:50:25 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.Format as step s26
Nov 20, 2018 8:50:25 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.DevNull as step s27
Nov 20, 2018 8:50:25 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/nexmark/staging/
Nov 20, 2018 8:50:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <88179 bytes, hash a_H6UnUYycCHXUZUuqLt5Q> to gs://temp-storage-for-perf-tests/nexmark/staging/pipeline-a_H6UnUYycCHXUZUuqLt5Q.pb
Dataflow SDK version: 2.9.0-SNAPSHOT
Nov 20, 2018 8:50:29 AM org.apache.beam.runners.dataflow.DataflowRunner run
Submitted job: 2018-11-20_00_50_25-6387333694799286878
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-11-20_00_50_25-6387333694799286878?project=apache-beam-testing
Nov 20, 2018 8:50:29 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2018-11-20_00_50_25-6387333694799286878
Nov 20, 2018 8:50:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2018-11-20T08:50:25.592Z: The requested max number of workers (4) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Nov 20, 2018 8:50:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:50:32.213Z: Checking permissions granted to controller Service Account.
Nov 20, 2018 8:50:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:50:38.389Z: Worker configuration: n1-standard-1 in us-central1-b.
Nov 20, 2018 8:50:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:50:38.872Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 20, 2018 8:50:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:50:38.945Z: Expanding GroupByKey operations into optimizable parts.
Nov 20, 2018 8:50:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:50:38.978Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 20, 2018 8:50:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:50:39.171Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 20, 2018 8:50:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:50:39.207Z: Unzipping flatten s15 for input s13.org.apache.beam.sdk.values.PCollection.<init>:402#403d1808e3d3838
Nov 20, 2018 8:50:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:50:39.243Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial, through flatten Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map
Nov 20, 2018 8:50:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:50:39.277Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract
Nov 20, 2018 8:50:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:50:39.313Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map
Nov 20, 2018 8:50:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:50:39.350Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)
Nov 20, 2018 8:50:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:50:39.386Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read
Nov 20, 2018 8:50:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:50:39.424Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify
Nov 20, 2018 8:50:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:50:39.458Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read
Nov 20, 2018 8:50:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:50:39.483Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues
Nov 20, 2018 8:50:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:50:39.515Z: Unzipping flatten s15-u45 for input partial-s16-out1-c43
Nov 20, 2018 8:50:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:50:39.548Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify, through flatten s15-u45, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 20, 2018 8:50:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:50:39.585Z: Fusing consumer Query7/Query7/BidToPrice into Query7/Query7/Window.Into()/Window.Assign
Nov 20, 2018 8:50:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:50:39.619Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 20, 2018 8:50:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:50:39.656Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map
Nov 20, 2018 8:50:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:50:39.692Z: Fusing consumer Query7.Format into Query7/Query7.Stamp
Nov 20, 2018 8:50:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:50:39.729Z: Fusing consumer Query7/Query7.Stamp into Query7/Query7.Debug
Nov 20, 2018 8:50:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:50:39.764Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 20, 2018 8:50:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:50:39.801Z: Fusing consumer Query7/Query7/justBids/IsBid/ParDo(Anonymous) into Query7/Query7.Snoop
Nov 20, 2018 8:50:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:50:39.838Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read
Nov 20, 2018 8:50:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:50:39.876Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map into Query7/Query7/BidToPrice
Nov 20, 2018 8:50:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:50:39.916Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues
Nov 20, 2018 8:50:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:50:39.949Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial
Nov 20, 2018 8:50:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:50:39.993Z: Fusing consumer Query7/Query7/justBids/AsBid into Query7/Query7/justBids/IsBid/ParDo(Anonymous)
Nov 20, 2018 8:50:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:50:40.027Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map
Nov 20, 2018 8:50:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:50:40.063Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map
Nov 20, 2018 8:50:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:50:40.101Z: Fusing consumer Query7/Query7.Monitor into Query7.ReadBounded
Nov 20, 2018 8:50:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:50:40.132Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract
Nov 20, 2018 8:50:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:50:40.169Z: Fusing consumer Query7/Query7/Window.Into()/Window.Assign into Query7/Query7/justBids/AsBid
Nov 20, 2018 8:50:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:50:40.204Z: Fusing consumer Query7/Query7.Debug into Query7/Query7/Query7.Select
Nov 20, 2018 8:50:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:50:40.242Z: Fusing consumer Query7/Query7.Snoop into Query7/Query7.Monitor
Nov 20, 2018 8:50:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:50:40.280Z: Fusing consumer Query7.DevNull into Query7.Format
Nov 20, 2018 8:50:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:50:40.312Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 20, 2018 8:50:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:50:40.348Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify
Nov 20, 2018 8:50:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:50:40.744Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Create
Nov 20, 2018 8:50:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:50:40.781Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Create
Nov 20, 2018 8:50:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:50:40.822Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
Nov 20, 2018 8:50:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:50:40.829Z: Starting 4 workers in us-central1-b...
Nov 20, 2018 8:50:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:50:41.083Z: Executing operation Query7.ReadBounded+Query7/Query7.Monitor+Query7/Query7.Snoop+Query7/Query7/justBids/IsBid/ParDo(Anonymous)+Query7/Query7/justBids/AsBid+Query7/Query7/Window.Into()/Window.Assign+Query7/Query7/BidToPrice+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write
Nov 20, 2018 8:50:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:50:53.500Z: Autoscaling: Raised the number of workers to 0 based on the rate of progress in the currently running step(s).
Nov 20, 2018 8:51:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:51:23.338Z: Workers have started successfully.
Nov 20, 2018 8:51:29 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:51:27.376Z: Autoscaling: Raised the number of workers to 2 based on the rate of progress in the currently running step(s).
Nov 20, 2018 8:51:29 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:51:27.418Z: Resized worker pool to 2, though goal was 4. This could be a quota issue.
Nov 20, 2018 8:51:30 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:51:30.461Z: Workers have started successfully.
Nov 20, 2018 8:51:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:51:39.489Z: Autoscaling: Raised the number of workers to 3 based on the rate of progress in the currently running step(s).
Nov 20, 2018 8:51:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:51:39.522Z: Resized worker pool to 3, though goal was 4. This could be a quota issue.
Nov 20, 2018 8:51:45 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:51:45.228Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running step(s).
Nov 20, 2018 8:52:45 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:52:44.749Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Close
Nov 20, 2018 8:52:45 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:52:44.868Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write
Nov 20, 2018 8:52:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:52:46.726Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Close
Nov 20, 2018 8:52:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:52:46.843Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
Nov 20, 2018 8:52:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:52:49.417Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
Nov 20, 2018 8:52:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:52:49.539Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow)
Nov 20, 2018 8:52:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:52:53.090Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView
Nov 20, 2018 8:52:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:52:53.340Z: Executing operation Query7/Query7/Query7.Select+Query7/Query7.Debug+Query7/Query7.Stamp+Query7.Format+Query7.DevNull
Nov 20, 2018 8:53:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:53:18.672Z: Cleaning up.
Nov 20, 2018 8:53:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:53:18.778Z: Stopping worker pool...
Nov 20, 2018 8:54:30 AM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish
2018-11-20T08:54:30.056Z Waiting for main pipeline to 'finish'
WARNING: No terminal state was returned. State value RUNNING
2018-11-20T08:54:30.094Z RUNNING Query7
2018-11-20T08:54:31.164Z new perf {"snapshots":null,"runtimeSec":101.228,"numEvents":10000000,"eventsPerSec":98786.89690599439,"eventBytesPerSec":1.2679939315209232E7,"numResults":100,"resultsPerSec":0.9878689690599439,"resultBytesPerSec":99.80440194412614,"startupDelaySec":-172.957,"processingDelaySec":77.905,"shutdownDelaySec":71.729,"timeDilation":9.71470344173549,"errors":null,"jobId":null}
2018-11-20T08:54:46.204Z RUNNING Query7
2018-11-20T08:54:47.255Z no activity
2018-11-20T08:55:02.287Z RUNNING Query7
2018-11-20T08:55:03.369Z no activity
2018-11-20T08:55:18.407Z RUNNING Query7
2018-11-20T08:55:19.524Z no activity
2018-11-20T08:55:34.561Z RUNNING Query7
2018-11-20T08:55:35.599Z no activity
2018-11-20T08:55:50.649Z RUNNING Query7
2018-11-20T08:55:51.681Z no activity
2018-11-20T08:56:06.736Z DONE Query7
> Task :beam-sdks-java-nexmark:run FAILED
:beam-sdks-java-nexmark:run (Thread[Task worker for ':' Thread 3,5,main]) completed. Took 5 mins 49.985 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/4.10.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 9m 35s
76 actionable tasks: 72 executed, 4 from cache
Publishing build scan...
https://gradle.com/s/zqqk2rkc4okkw
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Dataflow
#1089
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/1089/display/redirect>
------------------------------------------
[...truncated 257.91 KB...]
} ],
"message" : "The total number of changes to the object temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-vHRCqotCn6lvBiqXwBTqKQ.jar exceeds the rate limit. Please reduce the rate of create, update, and delete requests."
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:512)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:432)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:549)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:358)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Nov 20, 2018 6:03:54 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-vHRCqotCn6lvBiqXwBTqKQ.jar
Nov 20, 2018 6:03:58 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-vHRCqotCn6lvBiqXwBTqKQ.jar
Nov 20, 2018 6:04:01 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 117 files cached, 29 files newly uploaded
Nov 20, 2018 6:04:01 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.ReadBounded as step s1
Nov 20, 2018 6:04:02 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Monitor as step s2
Nov 20, 2018 6:04:02 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Snoop as step s3
Nov 20, 2018 6:04:02 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/IsBid/ParDo(Anonymous) as step s4
Nov 20, 2018 6:04:02 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/AsBid as step s5
Nov 20, 2018 6:04:02 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Window.Into()/Window.Assign as step s6
Nov 20, 2018 6:04:02 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/BidToPrice as step s7
Nov 20, 2018 6:04:02 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map as step s8
Nov 20, 2018 6:04:02 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce as step s9
Nov 20, 2018 6:04:02 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey as step s10
Nov 20, 2018 6:04:02 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues as step s11
Nov 20, 2018 6:04:02 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map as step s12
Nov 20, 2018 6:04:02 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map as step s13
Nov 20, 2018 6:04:02 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map as step s14
Nov 20, 2018 6:04:02 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections as step s15
Nov 20, 2018 6:04:02 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey as step s16
Nov 20, 2018 6:04:02 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues as step s17
Nov 20, 2018 6:04:02 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map as step s18
Nov 20, 2018 6:04:02 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) as step s19
Nov 20, 2018 6:04:02 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly as step s20
Nov 20, 2018 6:04:02 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) as step s21
Nov 20, 2018 6:04:02 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView as step s22
Nov 20, 2018 6:04:02 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Query7.Select as step s23
Nov 20, 2018 6:04:02 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Debug as step s24
Nov 20, 2018 6:04:02 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Stamp as step s25
Nov 20, 2018 6:04:02 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.Format as step s26
Nov 20, 2018 6:04:02 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.DevNull as step s27
Nov 20, 2018 6:04:02 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/nexmark/staging/
Nov 20, 2018 6:04:02 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <88182 bytes, hash w5soxkiSkC8m5e06Xa6Inw> to gs://temp-storage-for-perf-tests/nexmark/staging/pipeline-w5soxkiSkC8m5e06Xa6Inw.pb
Dataflow SDK version: 2.9.0-SNAPSHOT
Nov 20, 2018 6:04:03 AM org.apache.beam.runners.dataflow.DataflowRunner run
Submitted job: 2018-11-19_22_04_02-14667256735523976504
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-11-19_22_04_02-14667256735523976504?project=apache-beam-testing
Nov 20, 2018 6:04:03 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2018-11-19_22_04_02-14667256735523976504
Nov 20, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2018-11-20T06:04:02.741Z: The requested max number of workers (4) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Nov 20, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:04:05.777Z: Checking permissions granted to controller Service Account.
Nov 20, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:04:10.930Z: Worker configuration: n1-standard-1 in us-central1-b.
Nov 20, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:04:11.444Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 20, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:04:11.534Z: Expanding GroupByKey operations into optimizable parts.
Nov 20, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:04:11.573Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 20, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:04:11.773Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 20, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:04:11.822Z: Unzipping flatten s15 for input s13.org.apache.beam.sdk.values.PCollection.<init>:402#a70ce18896339064
Nov 20, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:04:11.870Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial, through flatten Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map
Nov 20, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:04:11.917Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract
Nov 20, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:04:11.965Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map
Nov 20, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:04:12.010Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)
Nov 20, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:04:12.056Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read
Nov 20, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:04:12.104Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify
Nov 20, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:04:12.146Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read
Nov 20, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:04:12.181Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues
Nov 20, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:04:12.218Z: Unzipping flatten s15-u45 for input partial-s16-out1-c43
Nov 20, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:04:12.257Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify, through flatten s15-u45, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 20, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:04:12.303Z: Fusing consumer Query7/Query7/BidToPrice into Query7/Query7/Window.Into()/Window.Assign
Nov 20, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:04:12.349Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 20, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:04:12.397Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map
Nov 20, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:04:12.435Z: Fusing consumer Query7.Format into Query7/Query7.Stamp
Nov 20, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:04:12.483Z: Fusing consumer Query7/Query7.Stamp into Query7/Query7.Debug
Nov 20, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:04:12.523Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 20, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:04:12.568Z: Fusing consumer Query7/Query7/justBids/IsBid/ParDo(Anonymous) into Query7/Query7.Snoop
Nov 20, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:04:12.616Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read
Nov 20, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:04:12.651Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map into Query7/Query7/BidToPrice
Nov 20, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:04:12.696Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues
Nov 20, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:04:12.742Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial
Nov 20, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:04:12.788Z: Fusing consumer Query7/Query7/justBids/AsBid into Query7/Query7/justBids/IsBid/ParDo(Anonymous)
Nov 20, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:04:12.835Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map
Nov 20, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:04:12.875Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map
Nov 20, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:04:12.922Z: Fusing consumer Query7/Query7.Monitor into Query7.ReadBounded
Nov 20, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:04:12.969Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract
Nov 20, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:04:13.018Z: Fusing consumer Query7/Query7/Window.Into()/Window.Assign into Query7/Query7/justBids/AsBid
Nov 20, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:04:13.054Z: Fusing consumer Query7/Query7.Debug into Query7/Query7/Query7.Select
Nov 20, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:04:13.100Z: Fusing consumer Query7/Query7.Snoop into Query7/Query7.Monitor
Nov 20, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:04:13.149Z: Fusing consumer Query7.DevNull into Query7.Format
Nov 20, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:04:13.186Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 20, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:04:13.219Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify
Nov 20, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:04:13.679Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Create
Nov 20, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:04:13.726Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Create
Nov 20, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:04:13.775Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
Nov 20, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:04:13.786Z: Starting 4 workers in us-central1-b...
Nov 20, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:04:14.095Z: Executing operation Query7.ReadBounded+Query7/Query7.Monitor+Query7/Query7.Snoop+Query7/Query7/justBids/IsBid/ParDo(Anonymous)+Query7/Query7/justBids/AsBid+Query7/Query7/Window.Into()/Window.Assign+Query7/Query7/BidToPrice+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write
Nov 20, 2018 6:04:34 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:04:33.168Z: Autoscaling: Raised the number of workers to 0 based on the rate of progress in the currently running step(s).
Nov 20, 2018 6:04:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:04:56.598Z: Autoscaling: Raised the number of workers to 1 based on the rate of progress in the currently running step(s).
Nov 20, 2018 6:04:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:04:56.634Z: Resized worker pool to 1, though goal was 4. This could be a quota issue.
Nov 20, 2018 6:05:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:05:02.650Z: Workers have started successfully.
Nov 20, 2018 6:05:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:05:02.694Z: Workers have started successfully.
Nov 20, 2018 6:05:14 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:05:14.624Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running step(s).
Nov 20, 2018 6:06:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:06:19.742Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Close
Nov 20, 2018 6:06:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:06:19.846Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write
Nov 20, 2018 6:06:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:06:20.624Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Close
Nov 20, 2018 6:06:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:06:20.728Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
Nov 20, 2018 6:06:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:06:22.462Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
Nov 20, 2018 6:06:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:06:22.564Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow)
Nov 20, 2018 6:06:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:06:25.841Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView
Nov 20, 2018 6:06:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:06:26.031Z: Executing operation Query7/Query7/Query7.Select+Query7/Query7.Debug+Query7/Query7.Stamp+Query7.Format+Query7.DevNull
Nov 20, 2018 6:06:55 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:06:54.183Z: Cleaning up.
Nov 20, 2018 6:06:55 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:06:54.334Z: Stopping worker pool...
Nov 20, 2018 6:08:03 AM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish
2018-11-20T06:08:03.694Z Waiting for main pipeline to 'finish'
WARNING: No terminal state was returned. State value RUNNING
2018-11-20T06:08:03.728Z RUNNING Query7
2018-11-20T06:08:04.798Z new perf {"snapshots":null,"runtimeSec":96.918,"numEvents":10000000,"eventsPerSec":103180.00784168059,"eventBytesPerSec":1.3243823613776594E7,"numResults":100,"resultsPerSec":1.031800078416806,"resultBytesPerSec":104.2427619224499,"startupDelaySec":-167.159,"processingDelaySec":72.389,"shutdownDelaySec":70.241,"timeDilation":10.14672197115087,"errors":null,"jobId":null}
2018-11-20T06:08:19.844Z RUNNING Query7
2018-11-20T06:08:20.815Z no activity
2018-11-20T06:08:35.851Z RUNNING Query7
2018-11-20T06:08:36.929Z no activity
2018-11-20T06:08:51.969Z RUNNING Query7
2018-11-20T06:08:53.016Z no activity
2018-11-20T06:09:08.055Z RUNNING Query7
2018-11-20T06:09:09.152Z no activity
2018-11-20T06:09:24.189Z RUNNING Query7
2018-11-20T06:09:25.222Z no activity
2018-11-20T06:09:40.273Z DONE Query7
> Task :beam-sdks-java-nexmark:run FAILED
:beam-sdks-java-nexmark:run (Thread[Task worker for ':' Thread 4,5,main]) completed. Took 6 mins 0.225 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/4.10.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 9m 33s
76 actionable tasks: 71 executed, 5 from cache
Publishing build scan...
https://gradle.com/s/xqoamggzskfhs
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org